powered by Jive Software

3.8.2 memory issues causing Openfire to crash repeatedly

Openfire is crashing repeatedly for me (multiple time over the course of the day), despite having very few users on the system (~10, low volume usage).

I’m getting this error in nohup.out:

A fatal error has been detected by the Java Runtime Environment:

java.lang.OutOfMemoryError: pthread_getattr_np

Internal Error (os_linux_x86.cpp:681), pid=32058, tid=140037785782016

Error: pthread_getattr_np

JRE version: 6.0_18-b07

I’m running the server with java options " -Xms128m -Xmx1024m", and I’ve even set xmpp.pep.enabled to ‘false’ (although I understand that shouldn’t be an issue in 3.8.2.

The admin console claims my memory usage is under control (eg., “78.84 MB of 455.12 MB”), but it seems flaky - if I refresh the page, the memory usage jumps by 2-3 Mb each time (regardless of how quickly or slowly I refresh), until it gets to about 130Mb, then resets to about 35… weird.

I’ve also tried using jmap, which claims Openfire is using about 90Mb.

I’m running on 64-bit CentOS, on a machine that has other stuff running on it, but still has about 6Gb of swap available… so there is no way that I am stressing the machine.

Another weird thing - the server was left running last night, and didn’t crash. But this morning, with about 2 users on the system, it went down. So, it does seem tied to usage, but again, the system is new and is hardly used at all.

Any ideas?

This seems to be a native OOM, and I assume that you have enough free physical memory. Otherwise you could set Xmx to a lower value.

Or try to use a more recent 1.6 JVM.

Yeah, there is plenty of physical memory available (over 2Gb free).

I’m not sure if I’m going to be able to upgrade the JVM, but I’ll experiment with a lower Xmx and let you know if that helps…

Matt Hofmann wrote:

The admin console claims my memory usage is under control (eg., “78.84 MB of 455.12 MB”), but it seems flaky - if I refresh the page, the memory usage jumps by 2-3 Mb each time (regardless of how quickly or slowly I refresh), until it gets to about 130Mb, then resets to about 35… weird.

This is a normal JVM behavior. It collects garbage and then releases it periodically.

Changing the Xmx to a lower value seems to have helped. I altered the openfire/bin/openfire script so that I’m running with “-Xms64m -Xmx256m”, and it seems much more stable (it hasn’t crashed in the past 24 hours, whereas it crashed ~5 times yesterday).

I still suspect that Openfire is doing something funky in terms of memory management, because it shouldn’t be spiking so drastically, given the low volume usage that I’m seeing. But, it seems like the issue is probably more on my end, rather than on Openfire’s end.

Thanks for the suggestions!

Sure, but if I keep refreshing every couple of seconds, it keeps jumping each time… then if I wait a minute between refreshes, it still jumps. So it seems like the admin console itself is causing memory usage to spike, until it gets collected.

So, maybe not related to the fact that I am seeing Openfire crash, but still pretty weird.

I see the same thing and this was like that with the older Openfire versions many years ago. JVM is keeping stuff in memory for some time (even if it’s not needed anymore) and after ~30 seconds garbage collection process kicks in and removes all unneeded stuff. So you see it constantly growing and then it drops down at some point. This is normal. Well at least from the Java standpoint Admin Console page refreshes shouldn’t affect this much or at all.

I’m having the same problem as Matt. I just upgraded my server from 3.7.1 to 3.8.2. Before upgrade, with the PEP Property disabled, I wasn’t having this problem anymore. If it’s corrected at this version, whatelse can I do to solve the problem?

I put the Xmx parameter as 256, 768 and even 1024, but none of them solved the issue.

Does anybody know what I can do?

Thanks in advance

@Bruno Figueiredo Saikali, what’s your physical memory available at all on your machine?

Rene,

Since my last post, on Nov 26, my openfire server isn’t showing any problem. I increased the Xmx memory to 768 and the total machine’s physical memory is 1 GB with 862 MB being used.

I think it’s normalized now. So, I believe this question is solved.

Thanks for your reply.

Good for you! But what was the main problem? You said “I put the Xmx parameter as 256, 768 and even 1024, but none of them solved the issue” and now you’re running without any problems while Xmx memory is set to 768.

Did you changed anything else? Xms size maybe?

@wroot we have a serious memory issue with a 32 bits version of open fire…it runs out of memory and than crashes which we catch with a failover…but still it happens to often. Would you know someone who can help us analyse

Upgrade Openfire, Upgrade to 64bit JRE.

Thanks, That is a great advice…can you share a procedure for doing this on a windows VM and how is the configuration upgrade/compatibility. Does an upgrade script exist? @akrherz