Out of Memory - Java Heap Space

Martyn_Baker · May 2, 2008, 7:05am

Hi Guys

I’m still getting numerous out of memory errors, even when setting the paramters -Xms128m -Xmx256m -Xss128k -Xoss128k -XX:ThreadStackSize=128 - the server seems to stay up and running for a day or a bit over with around 70 concurrent users. Once the server runs out of memory, I can’t get onto the admin console and no one can login.

2007.05.03 07:44:34 org.jivesoftware.openfire.container.PluginManager$PluginMonitor.run(PluginManage r.java:1027)

java.lang.OutOfMemoryError: Java heap space

The server is a decent spec with 2Gb RAM running CentOS 5.1 and I’m sure this didn’t happen when I was running 3.4.1 - I upgraded straight to 3.5.1

Any thoughts ?

Martyn

Daniel_Henninger · May 2, 2008, 1:03pm

Have you tried increasing the memory available to Openfire? I see you are limiting it to 256MB. It’s possible we hit a threshold where we may need to suggest higher settings. Of course, if it’s a steady 70 concurrent users, I wouldn’t think it would run itself out of memory that fast. It ought to at least stay fairly steady. Would you mind bumping slightly and seeing if all that it accomplishes is making it last a little longer before it runs out?

Martyn_Baker · May 2, 2008, 1:13pm

I’ve tried leaving it without any specific parameters - it then automatically set the Java memory at 500Mb but it still ran out after a couple of days. I’m sure this didn’t happen with 3.4.1 and only started when I upgraded to 3.5.1

I should add here that I run the admin console all day monitoring sessions, refreshing every 90 seconds.

Daniel_Henninger · May 2, 2008, 1:17pm

Hrm. Sounds like some sort of memory leak is getting you. What plugins do you have installed? What Java version?

Martyn_Baker · May 2, 2008, 1:25pm

1.6.0_03 Sun Microsystems Inc. – Java HotSpot™ Server VM plugins are as below:

Content Filter

Scans message packets for defined patterns

1.5.0 Conor Hayes

MotD (Message of the Day)

Allows admins to have a message sent to users each time they log in.

1.0.4 Ryan Graham

Search

Provides support for Jabber Search (XEP-0055)

1.4.2 Ryan Graham

Note that both the Content Filter and MotD plugin, although loaded, are not ‘active’ ie not doing anything. I can try removing them both and see if that makes any difference.

|

Martyn_Baker · May 2, 2008, 3:33pm

As a point of interest, I have just found this in the rror log:

at org.apache.mina.common.support.AbstractIoFilterChain.access$1300(AbstractIoFilt erChain.java:53)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.filterWrite(Ab stractIoFilterChain.java:659)

at org.apache.mina.filter.executor.ExecutorFilter.filterWrite(ExecutorFilter.java: 255)

at org.apache.mina.common.support.AbstractIoFilterChain.callPreviousFilterWrite(Ab stractIoFilterChain.java:361)

at org.apache.mina.common.support.AbstractIoFilterChain.access$1300(AbstractIoFilt erChain.java:53)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.filterWrite(Ab stractIoFilterChain.java:659)

at org.apache.mina.filter.codec.ProtocolCodecFilter.filterWrite(ProtocolCodecFilte r.java:205)

at org.apache.mina.common.support.AbstractIoFilterChain.callPreviousFilterWrite(Ab stractIoFilterChain.java:361)

at org.apache.mina.common.support.AbstractIoFilterChain.access$1300(AbstractIoFilt erChain.java:53)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.filterWrite(Ab stractIoFilterChain.java:659)

at org.apache.mina.common.IoFilterAdapter.filterWrite(IoFilterAdapter.java:90)

at org.jivesoftware.openfire.net.StalledSessionsFilter.filterWrite(StalledSessions Filter.java:49)

at org.apache.mina.common.support.AbstractIoFilterChain.callPreviousFilterWrite(Ab stractIoFilterChain.java:361)

at org.apache.mina.common.support.AbstractIoFilterChain.access$1300(AbstractIoFilt erChain.java:53)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.filterWrite(Ab stractIoFilterChain.java:659)

at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.filterWrite(Abs tractIoFilterChain.java:587)

at org.apache.mina.common.support.AbstractIoFilterChain.callPreviousFilterWrite(Ab stractIoFilterChain.java:361)

at org.apache.mina.common.support.AbstractIoFilterChain.fireFilterWrite(AbstractIo FilterChain.java:355)

at org.apache.mina.transport.socket.nio.SocketSessionImpl.write0(SocketSessionImpl .java:166)

at org.apache.mina.common.support.BaseIoSession.write(BaseIoSession.java:177)

at org.apache.mina.common.support.BaseIoSession.write(BaseIoSession.java:168)

at org.jivesoftware.openfire.nio.NIOConnection.deliver(NIOConnection.java:223)

at org.jivesoftware.openfire.session.LocalClientSession.deliver(LocalClientSession .java:787)

at org.jivesoftware.openfire.session.LocalSession.process(LocalSession.java:262)

at org.jivesoftware.openfire.spi.RoutingTableImpl.routePacket(RoutingTableImpl.jav a:235)

at org.jivesoftware.openfire.MessageRouter.route(MessageRouter.java:97)

at org.jivesoftware.openfire.spi.PacketRouterImpl.route(PacketRouterImpl.java:72)

at org.jivesoftware.openfire.net.StanzaHandler.processMessage(StanzaHandler.java:3 35)

at org.jivesoftware.openfire.net.ClientStanzaHandler.processMessage(ClientStanzaHa ndler.java:91)

at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:196)

at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:163)

at org.jivesoftware.openfire.nio.ConnectionHandler.messageReceived(ConnectionHandl er.java:133)

at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.messageReceived (AbstractIoFilterChain.java:570)

at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)

at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)

at org.apache.mina.common.IoFilterAdapter.messageReceived(IoFilterAdapter.java:80)

at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)

at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)

at org.apache.mina.filter.codec.support.SimpleProtocolDecoderOutput.flush(SimplePr otocolDecoderOutput.java:58)

at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:180)

at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)

at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)

at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java :239)

at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(Execut orFilter.java:283)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

Don’t know if this significant.

Martyn_Baker · May 7, 2008, 7:12am

Following on from Daniel’s previous comment about a memory leak, I wonder if that’s the case.

I was out of the office yesterday and didn’t use the admin console at all and looked today on the server screen and the memory used is only creeping up now, currently around 60Mb. The server’s been up for a couple of days now whereas previously, it died after a day and half.

I usually look at the sessions screen, with auto refresh set at 90 seconds, does anyone have any thoughts ?

Martyn_Baker · May 12, 2008, 9:03am

Further to the above, I’ve been off for a couple of days and noticed the server has stayed up with memory usage creeping to perhaps 60Mb then going back down to 17 odd Mb. I haven’t been using the admin console at all.

However, I’ve also noticed that everytime it goes back down, I get the following in the error log - other logs don’t report anything. Does anyone have any thoughts on what’s causing this ? I’ve deleted the ContentFilter Plugin since the previous post.