More Memory/IQHandler Issues

Openfire Openfire Support
1

Today our Openfire server stopped responding. I restarted and looked at the error logs, it was riddled with:

2008.06.13 10:19:07 [org.jivesoftware.openfire.handler.IQHandler.process(IQHandler.java:69)
] Internal server error
java.lang.IllegalArgumentException: IQ must be of type 'set' or 'get'. Original IQ: <iq type="result" id="494-277" to="im.domain.com" from="jsmith@im.domain.com/Adium"><query xmlns="http://jabber.org/protocol/disco#info"><identity category="client" type="pc" name="pidgin"></identity><feature var="jabber:iq:last"></feature><feature var="jabber:iq:oob"></feature><feature var="jabber:iq:time"></feature><feature var="xmpp:urn:time"></feature><feature var="jabber:iq:version"></feature><feature var="jabber:x:conference"></feature><feature var="http://jabber.org/protocol/bytestreams"></feature><feature var="http://jabber.org/protocol/disco#info"></feature><feature var="http://jabber.org/protocol/disco#items"></feature><feature var="http://jabber.org/protocol/muc"></feature><feature var="http://jabber.org/protocol/muc#user"></feature><feature var="http://jabber.org/protocol/si"></feature><feature var="http://jabber.org/protocol/si/profile/file-transfer"></feature><feature var="http://jabber.org/protocol/xhtml-im"></feature><feature var="urn:xmpp:ping"></feature><feature var="http://www.xmpp.org/extensions/xep-0199.html#ns"></feature><feature var="http://jabber.org/protocol/mood"></feature><feature var="http://jabber.org/protocol/mood+notify"></feature><feature var="http://jabber.org/protocol/nick"></feature><feature var="http://jabber.org/protocol/nick+notify"></feature><feature var="http://jabber.org/protocol/tune"></feature><feature var="http://jabber.org/protocol/tune+notify"></feature><feature var="http://www.xmpp.org/extensions/xep-0084.html#ns-metadata"></feature><feature var="http://www.xmpp.org/extensions/xep-0084.html#ns-data"></feature><feature var="http://www.xmpp.org/extensions/xep-0224.html#ns"></feature><feature var="http://www.xmpp.org/extensions/xep-0084.html#ns-metadata+notify"></feature><feature var="http://jabber.org/protocol/commands"></feature></query></iq>
     at org.xmpp.packet.IQ.createResultIQ(IQ.java:355)
     at org.jivesoftware.openfire.disco.IQDiscoInfoHandler.handleIQ(IQDiscoInfoHandler.java:99)
     at org.jivesoftware.openfire.handler.IQHandler.process(IQHandler.java:49)
     at org.jivesoftware.openfire.IQRouter.handle(IQRouter.java:349)
     at org.jivesoftware.openfire.IQRouter.route(IQRouter.java:101)
     at org.jivesoftware.openfire.spi.PacketRouterImpl.route(PacketRouterImpl.java:68)
     at org.jivesoftware.openfire.net.StanzaHandler.processIQ(StanzaHandler.java:299)
     at org.jivesoftware.openfire.net.ClientStanzaHandler.processIQ(ClientStanzaHandler.java:79)
     at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:264)
     at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:163)
     at org.jivesoftware.openfire.nio.ConnectionHandler.messageReceived(ConnectionHandler.java:133)
     at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.messageReceived(AbstractIoFilterChain.java:570)
     at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(AbstractIoFilterChain.java:299)
     at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilterChain.java:53)
     at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived(AbstractIoFilterChain.java:648)
     at org.apache.mina.common.IoFilterAdapter.messageReceived(IoFilterAdapter.java:80)
     at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(AbstractIoFilterChain.java:299)
     at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilterChain.java:53)
     at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived(AbstractIoFilterChain.java:648)
     at org.apache.mina.filter.codec.support.SimpleProtocolDecoderOutput.flush(SimpleProtocolDecoderOutput.java:58)
     at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:180)
     at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(AbstractIoFilterChain.java:299)
     at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilterChain.java:53)
     at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived(AbstractIoFilterChain.java:648)
     at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java:239)
     at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(ExecutorFilter.java:283)
     at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
     at java.lang.Thread.run(Unknown Source)

These messages are still piling up even though I’ve logged the user out. What seems to be happening here? I did find this JIRA issue:

http://www.igniterealtime.org/issues/browse/JM-890

But I’m not sure it’s related. Any clues?

Thanks!

Environment

Java Version:

1.6.0_03 Sun Microsystems Inc. – Java HotSpot™ Server VM

Appserver:

jetty-6.1.x

Host Name:

webreports

OS / Hardware:

Windows 2003 / x86

Locale / Timezone:

en / Eastern Standard Time

(-5 GMT)

Java Memory

|

165.88 MB of 493.06 MB (33.6%) used

|

|

2

Hey Tony,

As you said, I don’t think that error is related to your problem. Next time your server stops responding make sure to obtain a few thread dumps of the JVM. If the problem is memory related (e.g. running out of memory) you can then add this parameter -XX:+HeapDumpOnOutOfMemoryError to the startup script so a heap dump will be generated.

To get thread dumps (from Mac or Linux/Unix) you can execute kill -3 [process id].

Regards,

– Gato

3

I’ll turn on “the dumper” this weekend so I can catch the problem when it happens again.

Also, it turned out to be just one user causing the issue. On Adium. This is the second time this has happened to me with this particular chat client. Unfortunately, I have about as much luck getting rid of it as I do growing wings and flying to the Galapagos.

I don’t know if this helps or not, but I ended up solving the issue by deleting the user’s roster, roster groups and user record from jiveRoster, jiveRosterGroups, and jiveUser.

This seemed to make it happy again, although I have no idea why that would be the case.

Thanks for the quick replay Gato!

4

tmarkel wrote:

Unfortunately, I have about as much luck getting rid of it as I do growing wings and flying to the Galapagos.

I dont know what exactly did you mean, but you can try Client Control Plugin to do a minimal client restriction.

5

I tried that in order to ban Exodus, and had to un-ban that client as users were showing up at my door bearing pitchforks and torches…

6

Why banning Exodus? It’s a nice client

7

It appears to have been abandoned shortly after it’s initial release. It has great potential. It is a very nearly feature-complete XMPP client. I can see that a few developers are still working on it fairly actively, but with 37 Defects reported, and only 1 accepted, and 36 still “new”, little in the way of community involvement, it just sounds like abandonware to me…

8

Yeah. It was abandoned for a long time. I was surprised too see it has been moved to Google Code and even a new version was released. But that was last year. Btw, 2 last reported issue are mine But, with some limitations this client still fits our needs nearly for a 3 years now.

9

I thought that was you…

I really like Exodus, and I hope the project continues. Making it cross-platform would be really cool.