Urgent help needed, Openfire not responding

Hi,

We are using Openfire 3.7.1 and has pretty much stopped working since a couple of days ago. It seems to be impossible to connect to the server via our apps and scripts.

Looking at the error log I see different errors.

2013.03.09 03:32:33 org.jivesoftware.openfire.nio.ConnectionHandler - Closing session due to exception: (SOCKET, R: /168.187.215.97:34131, L: /10.1.0.6:5222, S: 0.0.0.0/0.0.0.0:5222)

org.apache.mina.filter.codec.ProtocolDecoderException: java.lang.Exception: Disallowed character (Hexdump: 17 03 01 00 D6 A1 54 DD 9C E7 70 6F A7 97 9F FE F7 61 5E BD E0 6C 9A 92 C7 F3 80 ED 1E 25 8E F2 12 0D 57 7E EC 45 34 B0 72 57 B8 $

at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:170)

at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)

at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)

at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java :239)

at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(Execut orFilter.java:283)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java: 886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)

at java.lang.Thread.run(Thread.java:662)

Caused by: java.lang.Exception: Disallowed character

at org.jivesoftware.openfire.nio.XMLLightweightParser.read(XMLLightweightParser.ja va:206)

at org.jivesoftware.openfire.nio.XMPPDecoder.doDecode(XMPPDecoder.java:41)

at org.apache.mina.filter.codec.CumulativeProtocolDecoder.decode(CumulativeProtoco lDecoder.java:133)

at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:163)


2013.03.09 03:36:05 org.jivesoftware.util.cache.DefaultCache - Cache Roster was full, shrinked to 90% in 0ms.


java.net.SocketException: Socket closed

at java.net.SocketInputStream.socketRead0(Native Method)

at java.net.SocketInputStream.read(SocketInputStream.java:129)

at com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:113)

at com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(Re adAheadInputStream.java:160)

at com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:188)

at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1994)

at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2411)

at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2916)

at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1631)

at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1723)

at com.mysql.jdbc.Connection.execSQL(Connection.java:3283)

at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1332)

at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:882)

at sun.reflect.GeneratedMethodAccessor44.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.ja va:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at org.logicalcobwebs.proxool.ProxyStatement.invoke(ProxyStatement.java:100)

at org.logicalcobwebs.proxool.ProxyStatement.intercept(ProxyStatement.java:57)

at $java.sql.Statement$$EnhancerByProxool$$44888c70.execute()

at org.jivesoftware.openfire.spi.PresenceManagerImpl.deleteOfflinePresenceFromDB(P resenceManagerImpl.java:229)

at org.jivesoftware.openfire.spi.PresenceManagerImpl.userAvailable(PresenceManager Impl.java:214)

at org.jivesoftware.openfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:157)

at org.jivesoftware.openfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:135)

at org.jivesoftware.openfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:199)

at org.jivesoftware.openfire.PresenceRouter.handle(PresenceRouter.java:149)

at org.jivesoftware.openfire.PresenceRouter.route(PresenceRouter.java:85)

at org.jivesoftware.openfire.spi.PacketRouterImpl.route(PacketRouterImpl.java:84)

at org.jivesoftware.openfire.net.StanzaHandler.processPresence(StanzaHandler.java: 355)

at org.jivesoftware.openfire.net.ClientStanzaHandler.processPresence(ClientStanzaH andler.java:100)

at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:272)

at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:194)

at org.jivesoftware.openfire.nio.ConnectionHandler.messageReceived(ConnectionHandl er.java:169)

at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.messageReceived (AbstractIoFilterChain.java:570)

at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)

at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)

at org.apache.mina.common.IoFilterAdapter.messageReceived(IoFilterAdapter.java:80)

at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)

at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)

at org.apache.mina.filter.codec.support.SimpleProtocolDecoderOutput.flush(SimplePr otocolDecoderOutput.java:58)

at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:185)

at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)

at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)

at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java :239)

at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(Execut orFilter.java:283)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java: 886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)

at java.lang.Thread.run(Thread.java:662)

Anyone knows what the issue can be?

Thanks

Some information regarding your setup would be helpful. Are you using a clustered setup with specialized client (“It seems to be impossible to connect to the server via our apps and scripts.”) ?

Hi Walter,

I realized that I have left out some info. I have replied to my own post here with our latest issue as detailed as possible: http://community.igniterealtime.org/message/228864

After we upgraded resources on our Galera Cluster Replication and increased roster cache to 1024MB it seemed to have solved these issues.

Our main issue now is now as described in my latest post here http://community.igniterealtime.org/message/228864

Thank you