Openfire 3.7.1-service stays running but connections are dropped

Hi,

We’re running Openfire 3.7.1 on Windows server 2003 Standard with SP2. The service stays running, but all connections are dropped. We have to restart the server in order for it to work. This has been an on going problem. Any ideas would help.

Thanks

I have experienced this as well. Haven’t been able to track down the exact cause. However I can share that if you kill the service/process (With pskill or similar) and restart it, the issue will resolve. Lot quicker than a full reboot of the server.

Thanks

I think we have found the solution to the problem. From reading other posts people has pointed out that it’s a Java memory issue. We tried xmp.pep.enable, cache.username2roster.maxLifetime, decrease the Flush Interval and Batch Size in Group Chat Settings but none help. We found the post where they said in order to fix the java memory leak issue you need to increase the size of Java memory, see the link below on how to do it.

I hope that this will resolve our issue and help other as well.

The service is stop working again even we have increased the javamemory. This is really sad that no one seems to have a solution to the memory issue. It’s look like we might have to give up on Openfire.

Feel free to use other XMPP platforms if you wish, nobody is forcing you to stay with openfire. The development/nightly build of openfire has a number of stability and memory fixes, it may be worth trying it first before leaving.

Daryl,

Unfortunately all the solution we found in this forum didn’t resolve the issue. We’re willing to try any solution that work. If you or someone else there have any other solution please let us know. As you can see what we tried almost everything, look at my previous post.

Do you have any logs to indicate what the root cause is? Sounds like we’re mostly guessing so far? Does Windows log the JVM output somewhere? That is separate from the log4j stuff Openfire itself logs.

We didn’t have any log for the JVM output, not sure where. We only have the error log, debug, and info in the openfire directory C:\Program Files\Openfire\logs

2012.12.07 09:52:35 org.jivesoftware.openfire.container.PluginManager - Error loading plugin: C:\Program Files\Openfire\plugins\webchat

java.lang.NoClassDefFoundError: org/mortbay/jetty/servlet/Context

at java.lang.Class.getDeclaredConstructors0(Native Method)

at java.lang.Class.privateGetDeclaredConstructors(Unknown Source)

at java.lang.Class.getConstructor0(Unknown Source)

at java.lang.Class.newInstance0(Unknown Source)

at java.lang.Class.newInstance(Unknown Source)

at org.jivesoftware.openfire.container.PluginManager.loadPlugin(PluginManager.java :420)

at org.jivesoftware.openfire.container.PluginManager.access$300(PluginManager.java :80)

at org.jivesoftware.openfire.container.PluginManager$PluginMonitor.run(PluginManag er.java:1067)

at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)

at java.util.concurrent.FutureTask.runAndReset(Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101 (Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodi c(Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknow n Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

Caused by: java.lang.ClassNotFoundException: org.mortbay.jetty.servlet.Context

at java.net.URLClassLoader$1.run(Unknown Source)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(Unknown Source)

at java.lang.ClassLoader.loadClass(Unknown Source)

at java.lang.ClassLoader.loadClass(Unknown Source)

… 17 more

2012.12.07 09:52:55 org.jivesoftware.openfire.container.PluginManager - Error loading plugin: C:\Program Files\Openfire\plugins\webchat

java.lang.NoClassDefFoundError: org/mortbay/jetty/servlet/Context

at java.lang.Class.getDeclaredConstructors0(Native Method)

at java.lang.Class.privateGetDeclaredConstructors(Unknown Source)

at java.lang.Class.getConstructor0(Unknown Source)

at java.lang.Class.newInstance0(Unknown Source)

at java.lang.Class.newInstance(Unknown Source)

at org.jivesoftware.openfire.container.PluginManager.loadPlugin(PluginManager.java :420)

at org.jivesoftware.openfire.container.PluginManager.access$300(PluginManager.java :80)

at org.jivesoftware.openfire.container.PluginManager$PluginMonitor.run(PluginManag er.java:1067)

at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)

at java.util.concurrent.FutureTask.runAndReset(Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101 (Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodi c(Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknow n Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

Caused by: java.lang.ClassNotFoundException: org.mortbay.jetty.servlet.Context

at java.net.URLClassLoader$1.run(Unknown Source)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(Unknown Source)

at java.lang.ClassLoader.loadClass(Unknown Source)

at java.lang.ClassLoader.loadClass(Unknown Source)

… 17 more

2012.12.07 09:53:15 org.jivesoftware.openfire.container.PluginManager - Error loading plugin: C:\Program Files\Openfire\plugins\webchat

java.lang.NoClassDefFoundError: org/mortbay/jetty/servlet/Context

at java.lang.Class.getDeclaredConstructors0(Native Method)

at java.lang.Class.privateGetDeclaredConstructors(Unknown Source)

at java.lang.Class.getConstructor0(Unknown Source)

at java.lang.Class.newInstance0(Unknown Source)

at java.lang.Class.newInstance(Unknown Source)

at org.jivesoftware.openfire.container.PluginManager.loadPlugin(PluginManager.java :420)

at org.jivesoftware.openfire.container.PluginManager.access$300(PluginManager.java :80)

at org.jivesoftware.openfire.container.PluginManager$PluginMonitor.run(PluginManag er.java:1067)

at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)

at java.util.concurrent.FutureTask.runAndReset(Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101 (Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodi c(Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknow n Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

Caused by: java.lang.ClassNotFoundException: org.mortbay.jetty.servlet.Context

at java.net.URLClassLoader$1.run(Unknown Source)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(Unknown Source)

at java.lang.ClassLoader.loadClass(Unknown Source)

at java.lang.ClassLoader.loadClass(Unknown Source)

… 17 more

2012.12.07 09:53:35 org.jivesoftware.openfire.container.PluginManager - Error loading plugin: C:\Program Files\Openfire\plugins\webchat

java.lang.NoClassDefFoundError: org/mortbay/jetty/servlet/Context

at java.lang.Class.getDeclaredConstructors0(Native Method)

at java.lang.Class.privateGetDeclaredConstructors(Unknown Source)

at java.lang.Class.getConstructor0(Unknown Source)

at java.lang.Class.newInstance0(Unknown Source)

at java.lang.Class.newInstance(Unknown Source)

at org.jivesoftware.openfire.container.PluginManager.loadPlugin(PluginManager.java :420)

at org.jivesoftware.openfire.container.PluginManager.access$300(PluginManager.java :80)

at org.jivesoftware.openfire.container.PluginManager$PluginMonitor.run(PluginManag er.java:1067)

at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)

at java.util.concurrent.FutureTask.runAndReset(Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101 (Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodi c(Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknow n Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

Caused by: java.lang.ClassNotFoundException: org.mortbay.jetty.servlet.Context

at java.net.URLClassLoader$1.run(Unknown Source)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(Unknown Source)

at java.lang.ClassLoader.loadClass(Unknown Source)

at java.lang.ClassLoader.loadClass(Unknown Source)

… 17 more

2012.12.07 09:53:55 org.jivesoftware.openfire.container.PluginManager - Error loading plugin: C:\Program Files\Openfire\plugins\webchat

java.lang.NoClassDefFoundError: org/mortbay/jetty/servlet/Context

at java.lang.Class.getDeclaredConstructors0(Native Method)

at java.lang.Class.privateGetDeclaredConstructors(Unknown Source)

at java.lang.Class.getConstructor0(Unknown Source)

at java.lang.Class.newInstance0(Unknown Source)

at java.lang.Class.newInstance(Unknown Source)

at org.jivesoftware.openfire.container.PluginManager.loadPlugin(PluginManager.java :420)

at org.jivesoftware.openfire.container.PluginManager.access$300(PluginManager.java :80)

at org.jivesoftware.openfire.container.PluginManager$PluginMonitor.run(PluginManag er.java:1067)

at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)

at java.util.concurrent.FutureTask.runAndReset(Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101 (Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodi c(Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknow n Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

Caused by: java.lang.ClassNotFoundException: org.mortbay.jetty.servlet.Context

at java.net.URLClassLoader$1.run(Unknown Source)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(Unknown Source)

at java.lang.ClassLoader.loadClass(Unknown Source)

at java.lang.ClassLoader.loadClass(Unknown Source)

… 17 more

2012.12.07 09:54:15 org.jivesoftware.openfire.container.PluginManager - Error loading plugin: C:\Program Files\Openfire\plugins\webchat

java.lang.NoClassDefFoundError: org/mortbay/jetty/servlet/Context

at java.lang.Class.getDeclaredConstructors0(Native Method)

at java.lang.Class.privateGetDeclaredConstructors(Unknown Source)

at java.lang.Class.getConstructor0(Unknown Source)

at java.lang.Class.newInstance0(Unknown Source)

at java.lang.Class.newInstance(Unknown Source)

at org.jivesoftware.openfire.container.PluginManager.loadPlugin(PluginManager.java :420)

at org.jivesoftware.openfire.container.PluginManager.access$300(PluginManager.java :80)

at org.jivesoftware.openfire.container.PluginManager$PluginMonitor.run(PluginManag er.java:1067)

at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)

at java.util.concurrent.FutureTask.runAndReset(Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101 (Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodi c(Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknow n Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

Caused by: java.lang.ClassNotFoundException: org.mortbay.jetty.servlet.Context

at java.net.URLClassLoader$1.run(Unknown Source)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(Unknown Source)

at java.lang.ClassLoader.loadClass(Unknown Source)

at java.lang.ClassLoader.loadClass(Unknown Source)

… 17 more

I’m not sure if this is JVA log or not.

java.lang.NullPointerException

at org.hsqldb.scriptio.ScriptWriterText.writeDeleteStatement(Unknown Source)

at org.hsqldb.persist.Log.writeDeleteStatement(Unknown Source)

at org.hsqldb.persist.Logger.writeDeleteStatement(Unknown Source)

at org.hsqldb.Table.deleteNoCheck(Unknown Source)

at org.hsqldb.Table.updateRowSet(Unknown Source)

at org.hsqldb.Table.update(Unknown Source)

at org.hsqldb.CompiledStatementExecutor.executeUpdateStatement(Unknown Source)

at org.hsqldb.CompiledStatementExecutor.executeImpl(Unknown Source)

at org.hsqldb.CompiledStatementExecutor.execute(Unknown Source)

at org.hsqldb.Session.sqlExecuteCompiledNoPreChecks(Unknown Source)

at org.hsqldb.Session.sqlExecute(Unknown Source)

at org.hsqldb.Session.sqlExecute(Unknown Source)

at org.hsqldb.Session.execute(Unknown Source)

at org.hsqldb.jdbc.jdbcPreparedStatement.executeUpdate(Unknown Source)

at sun.reflect.GeneratedMethodAccessor49.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

at java.lang.reflect.Method.invoke(Unknown Source)

at org.logicalcobwebs.proxool.ProxyStatement.invoke(ProxyStatement.java:100)

at org.logicalcobwebs.proxool.ProxyStatement.intercept(ProxyStatement.java:57)

at $java.sql.Wrapper$$EnhancerByProxool$$3bb9b809.executeUpdate()

at org.jivesoftware.openfire.reporting.stats.RrdSqlBackend.sync(RrdSqlBackend.java :194)

at org.jivesoftware.openfire.reporting.stats.RrdSqlBackend.close(RrdSqlBackend.jav a:178)

at org.jrobin.core.RrdDb.close(Unknown Source)

at org.jivesoftware.openfire.reporting.stats.StatsEngine$SampleTask.run(StatsEngin e.java:405)

at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

at java.util.concurrent.FutureTask.run(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

java.lang.NullPointerException

at org.hsqldb.scriptio.ScriptWriterText.writeDeleteStatement(Unknown Source)

at org.hsqldb.persist.Log.writeDeleteStatement(Unknown Source)

at org.hsqldb.persist.Logger.writeDeleteStatement(Unknown Source)

at org.hsqldb.Table.deleteNoCheck(Unknown Source)

at org.hsqldb.Table.updateRowSet(Unknown Source)

at org.hsqldb.Table.update(Unknown Source)

at org.hsqldb.CompiledStatementExecutor.executeUpdateStatement(Unknown Source)

at org.hsqldb.CompiledStatementExecutor.executeImpl(Unknown Source)

at org.hsqldb.CompiledStatementExecutor.execute(Unknown Source)

at org.hsqldb.Session.sqlExecuteCompiledNoPreChecks(Unknown Source)

at org.hsqldb.Session.sqlExecute(Unknown Source)

at org.hsqldb.Session.sqlExecute(Unknown Source)

at org.hsqldb.Session.execute(Unknown Source)

at org.hsqldb.jdbc.jdbcPreparedStatement.executeUpdate(Unknown Source)

at sun.reflect.GeneratedMethodAccessor49.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

at java.lang.reflect.Method.invoke(Unknown Source)

at org.logicalcobwebs.proxool.ProxyStatement.invoke(ProxyStatement.java:100)

at org.logicalcobwebs.proxool.ProxyStatement.intercept(ProxyStatement.java:57)

at $java.sql.Wrapper$$EnhancerByProxool$$3bb9b809.executeUpdate()

at org.jivesoftware.openfire.reporting.stats.RrdSqlBackend.sync(RrdSqlBackend.java :194)

at org.jivesoftware.openfire.reporting.stats.RrdSqlBackend.close(RrdSqlBackend.jav a:178)

at org.jrobin.core.RrdDb.close(Unknown Source)

at org.jivesoftware.openfire.reporting.stats.StatsEngine$SampleTask.run(StatsEngin e.java:405)

at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

at java.util.concurrent.FutureTask.run(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

Can you start by removing the webchat plugin? Are you actually using it?

No, we do not use webchat but yes we can try to remove it.

One possible culprit in our situation (Again just trying to isolate the issue) Was we were doing live reporting off of fastpath data from the openfire database. Once we made a replicated database with nightly dumps and change the reporting over to that, the issue let up.

So maybe see if your database is tweaking somehow, again I never found a solid answer I can just post what we did to isolate the issue.