Increasing instability with OpenFire, what do I do?

I’m having to restart the OpenFire service (running on Centos 6) 3 or 4 times a day because it simply stops routing messages during the day.

And there are also several exceptions in the error log, some of which seem to be known and simply accepted (based on my research)

How much memory have you allocated? Which plugins are installed?

Version: Openfire 3.9.3

Java Version: 1.7.0_13 Oracle Corporation – Java HotSpot™ 64-Bit Server VM

Appserver: jetty/7.x.y-SNAPSHOT

Java Memory: 186.55 MB of 989.88 MB (18.8%) used

OS / Hardware: Linux / amd64

Looks like the only installed plugin is “Search”, based on /plugin-admin.jsp

That version of Oracle Java is ancient. Do you care about security? Updating Java may fix this memory issue.

Thanks, I must not have installed Java via yum, I’ll have to clear this install, and start fresh. What’s the best way to “export” the OpenFire settings in a human readable format?

Any idea how well OpenFire works on OpenJDK? If it works well, that would save me from having to manually follow Oracle Java updates

Daryl, he is not going to browse the Internet on that server and the java installed on his server don’t even has a browser plugin, i think. Security is not a big issue, if at all… Most Java security flaws are about Oracle’s browser plugin.

But Java is indeed old. I was using Openfire with 1.7.0_01 or something on Arch linux box. And when i tried to run Openfire 3.9.3 it was stalling with memory issues every day, so i had to switch back to 3.9.1. This is an option for you. Try downgrading to 3.9.1.

Though, I have moved my server to a Windows Server 2008 R2 and have installed Java JRE 1.8.0 the latest version. And now Openfire 3.9.3 runs smoothly. Not sure where the problem was. Maybe it’s Java, maybe it’s linux. Btw, i have modified my vmoptions to use 512-1024 MB, looks like you did it too.

After reading these forums for many years i tend to think, that it is better to use Openfire with Oracle’s Java than OpenJDK. But i can’t provide any facts or statistics. Just a feeling.

Btw, when you look how much memory JVM uses in the Admin Console, try refreshing it every second to see how much it climbs, because it changes dynamically. You can also check this every hour or more often to see if it actually getting close to 989 MB limit. If it does, then you are hitting some memory leak. Also you may be getting “out of memory” errors in the logs.

wroot, you are wrong

Regardless, I uninstalled Java and OpenFire on that machine, stripped it all the way down, upgraded from Centos 6 to Centos 7, installed OpenJDK and reinstalled OpenFire and a re-keyed certificate with MariaDB instead of MySQL (couldn’t figure out how to load the MariaDB Java drivers though).

I’ll see how it holds up for the following week.

Daryl Herzmann wrote:

wroot, you are wrong

About what? The security or OpenJDK?

So now, I have an essentially clean system. 24hrs later, I tried logging in again (with Trillian) and the login process got stuck:

*** Creating connection “apemberton@im.mediagistic.com”.

*** TLS established using ECDHE-RSA-DES-CBC3-SHA (TLSv1/SSLv3).

That’s client side.

I restart the service, and bam everything is working again.

So, there is no “out of memory” errors in the logs? You can try disabling SSL for a while and see if you can login without it, though i doubt. You can also try using 3.9.1 to see if 3.9.3 is the problem.

error.log and info.log had nothing obvious, the JVM had 500MB to play with last I checked, so I can’t see how it could have been out of memory, I’ll have to check the memory usage next time before I restart.

I had turned off SSL (client side) before the re-install, so I doubt that’s the issue.

I am hesitant to downgrade, I believe that older version lacked the property encryption, which would mean that I would need to fresh install it.

I’d like advice on how to further debug this issue.

If you have created new properties since 3.9.3 install, then you can delete them and recreate after you downgrade to 3.9.1. If you haven’t, then there shouldn’t be problems with properties. I have done downgrade from 3.9.3 to 3.9.1 both on production server (linux) and on test server (windows). I don’t like suggesting downgrading, but as this is a community driven project which lacks much attention currently, i doubt the cause will be found soon. Don’t know how to debug such issues. Well, you can create dumps of a freezing server or something, but that’s above my knowledge and then there should be someone with expertize to look through such dumps. As there are only a few volunteers doing some stuff sporadically (basically on the stuff they are interested in), i doubt someone will take a look at them.

But you can try. JVM Settings and Debugging