Wildfire crashes in FreeBSD 6.0 SMP

Openfire Openfire Support
1

Hi,

I am using wildfire(2.6.2) server for jabber services. Its running on FreeBSD 6.0 with SMP kernel, and Java uses 2GB memory for its operation (Xmx=2048m).

When I connect jabber clients more than 18000(around) wildfire crashes and generates a huge “java.core” file. How to solve this problem? Or I have to tune some wildfire/java parameters ?

Thanks,

LF

2

Hi,

do you also get a java thread dump in nohup.out?

Do you see outofmemory or thread errors in the log files?

Is java.core a binary file?

LG

3

Yes, there was a error saying " OutOfMemory error: Could not create native thread." in the log file.

The same kind of error in nohup.out file also.

java. core is a binary file.

Thanks.

4

LF,

18,000 users is a lot. Wildfire uses the standard Java IO library which is not as scalable as some of the alternatives such as NIO. Until we switch to NIO, you’'re going to have issues scaling to a huge number of users with a single instance. I would recommend looking at using connection managers to help mange the client connection load. Using a couple of servers, you should be able to scale to that many users much more easily. Connection manager support is a part of the upcoming Wildfire 3.0. You can test them out now:

http://www.jivesoftware.org/wildfire/connection_manager.jsp

Regards,

Matt

5

Yes, I agree that 18,000 users is a lot.

But out of curiosity I am asking this question. Why it crashes and generating java.core file?. Ideally after reaching some threshold limit, it should not accept any more new connection and keep alive with current users. (some where I’'ve read like, “pure java applications will not crash by itself. If it crashes then the real cause is the JVM or the c/c++ code which java application is using”. If so, why our wildfire is behaving like this? Is the real cause is JVM or wildfire using c/c++ code creating this problem?

Thanks.

6

Hi,

you could try to set -Xss128k (or one which is not really tested -Xss96k) to decrease the memory every thread allocates. Depending on your kernel and the memory mapping it may help, as I have nealry no BSD experience if it helps.

Is the “OutOfMemory error: Could not create native thread.” the cause for the crash or do you see this error very often?

What should the virtual machine do if the application running in it is trying to start a new thread and this fails? I assume that it is possible to catch such errors within the java application, but one would not really be happy doing this.

LG

7

Yep, LG is exactly right. The reason you’‘re crashing is due to the out of memory. It’‘s one of the very few ways to make Java crash. LG’'s change will help decrease memory usage. Another thing you can do is simply limit the number of users that connect to the server. The CM architecture should really help to distribute load, though.

Regards,

Matt