Good morning:
We have been suffering some memory problems with the connection manager, up to the point that
we cannot use it for more than some hours.
For some hours, the heap memory gets gc, but it keeps growing.
At some point we get error messages of type java.lang.OutOfMemoryError: Java heap space.
Then it’s not possible to connect to the CM anymore.
I have been taking jmap heap dumps and analysing them.
Maybe if you see the analysis, somebody can give a clue about what to
look for in the code and try to get a fix.
I am somewhat surprised than nobody has found this problem. Maybe it’s
because of the clients we use, we use some some custom made flash
clients that are starting sessions and probably not ending them
properly. But i think the server should end this sessions sometime and
free resources.
These are the options for the java VM.
JVM_SETTINGS="-Xmx512m -Xms512m -Xss80k
-XX:+HeapDumpOnOutOfMemoryError -XX:GCTimeRatio=19 -XX:+PrintGCDetails
-XX:+PrintGCTimeStamps
-Xloggc:/opt/connection_manager/logs/garbage-collector-`date'+%F-%H-%M-%S'`.log"
The java version is:
java version “1.6.0_03”
Java™ SE Runtime Environment (build 1.6.0_03-b05)
running in fedora linux.
This is the histogram at the end of the test:
Class Instance Count Total Size
class [B 114705 458730375
class java.nio.HeapByteBuffer 111653 3907855
class org.apache.mina.common.support.DefaultWriteFuture 111648 2344608
class org.apache.mina.common.SimpleByteBufferAllocator$SimpleByteBuffer 111649 1451437
class org.apache.mina.common.IoFilter$WriteRequest 111648 1339776
class [C 9127 1058136
class java.util.concurrent.ConcurrentLinkedQueue$Node 112071 896568
class java.util.concurrent.atomic.AtomicInteger 112328 449312
class [I 2441 229152
class [S 2555 148932
class java.lang.Class 1913 145388
class java.lang.String 7318 117088
It looks that the HeapByteBuffer are not freed properly, and WriteRequests get stuck, or something like that.
I have attached a file showing how the instance count evolves with
time, and the log of the garbage collector.
I hope that somebody can give a clue about which part of the code should i look into,
because i haven’t found where to start.
¡¡Thanks a lot for openfire, it has worked perfectly up to now!!
– Gaizka