Possible LDAP deadlock

OK…I’‘ve mentioned that I’‘m still having LDAP issues (seemingly) in previous threads. I’‘ve finally gotten a thread dump when the server wedges (ain’'t intermittent problems a pain to diagnose?)

Some setup info…

SuSE 10.0, rpm install, LDAP auth and user providers (not using group or vcard providers currently, though I would like to get back to using them sooner or later, but anything that exacerbates these LDAP wedges is bad for me right now).

I have a shared roster group manually created (since I’‘m not currently using the LDAP group provider) with all users added to it and the shared roster group being present in everyone’‘s roster (basically, everyone in the company shows up in everyone else’‘s roster). When whatever it is happens, any operation on the server that retrieves all users (such as logging in and pulling the roster with that shared roster group with all users, or going to the edit group page for the shared group, which displays all the users in the group, will hang and never return. I can click on other pages in the admin console and they will return, but not anything that deals with the full list of users. I suspect, but have not be able to confirm, that if I went to through the pages of users in the user’‘s summary, I would hit a page that would hang waiting for a specific user to return…I haven’‘t confirmed this, yet, however. I will try to do that next time this wedges…perhaps there’'s a specific user object that is triggering this and I can narrow it down to 15 users, at least, doing that.

wildfire.xml (with large comments eliminated to save space):

<adminDN/>
<adminPassword/>
<sslEnabled>true</sslEnabled>
<groupSearchFilter><![CDATA[(&(cn=)(cn=Users))]]></FN>
        <NICKNAME attrs="uid"></NICKNAME>













uid: home: shell:

and the thread dump:

Full thread dump Java HotSpot™ Server VM (1.5.0_06-b05 mixed mode):

“Client SR - 32872251” daemon prio=1 tid=0x082f87e0 nid=0x6458 waiting for monitor entry

at org.jivesoftware.wildfire.user.UserManager.getUser(UserManager.java:169)

  • waiting to lock (a java.lang.String)

at org.jivesoftware.wildfire.user.User.getRoster(User.java:280)

at org.jivesoftware.wildfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:187)

at org.jivesoftware.wildfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :104)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“SunJsseListener1-2” prio=1 tid=0x080dee50 nid=0x6405 in Object.wait()

at java.lang.Object.wait(Native Method)

  • waiting on <0x7b3b16f0> (a org.mortbay.util.ThreadPool$PoolThread)
    at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:522)
  • locked <0x7b3b16f0> (a org.mortbay.util.ThreadPool$PoolThread)

“Client SR - 7141756” daemon prio=1 tid=0x080dcf58 nid=0x63d4 waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bcc0868> (a java.lang.String)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.broadcastUpdate(Presenc eUpdateHandler.java:257)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:109)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:153)
    at org.jivesoftware.wildfire.SessionManager$ClientSessionListener.onConnectionClos e(SessionManager.java:1288)
    at org.jivesoftware.wildfire.net.SocketConnection.notifyCloseListeners(SocketConne ction.java:552)
  • locked <0x62305468> (a java.util.HashMap)

at org.jivesoftware.wildfire.net.SocketConnection.close(SocketConnection.java:374)

at org.jivesoftware.wildfire.handler.IQAuthHandler.login(IQAuthHandler.java:199)

at org.jivesoftware.wildfire.handler.IQAuthHandler.handleIQ(IQAuthHandler.java:140 )

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 11830622” daemon prio=1 tid=0x080e04d8 nid=0x63ce waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bdef650> (a java.lang.String)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.broadcastUpdate(Presenc eUpdateHandler.java:257)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:109)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:153)
    at org.jivesoftware.wildfire.SessionManager$ClientSessionListener.onConnectionClos e(SessionManager.java:1288)
    at org.jivesoftware.wildfire.net.SocketConnection.notifyCloseListeners(SocketConne ction.java:552)
  • locked <0x7b557fe8> (a java.util.HashMap)

at org.jivesoftware.wildfire.net.SocketConnection.close(SocketConnection.java:374)

at org.jivesoftware.wildfire.handler.IQAuthHandler.login(IQAuthHandler.java:199)

at org.jivesoftware.wildfire.handler.IQAuthHandler.handleIQ(IQAuthHandler.java:140 )

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 12001189” daemon prio=1 tid=0x085ee530 nid=0x63bc waiting for monitor entry

at org.jivesoftware.wildfire.user.UserManager.getUser(UserManager.java:169)

  • waiting to lock (a java.lang.String)

at org.jivesoftware.wildfire.user.User.getRoster(User.java:280)

at org.jivesoftware.wildfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:187)

at org.jivesoftware.wildfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :104)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 27321041” daemon prio=1 tid=0x08376eb8 nid=0x63aa waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bd79d98> (a java.lang.String)

at org.jivesoftware.wildfire.user.User.getRoster(User.java:280)

at org.jivesoftware.wildfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:187)

at org.jivesoftware.wildfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :104)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“pool-215-thread-1” prio=1 tid=0x083139d0 nid=0x6364 waiting on condition

at sun.misc.Unsafe.park(Native Method)

at java.util.concurrent.locks.LockSupport.park(Unknown Source)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unk nown Source)

at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

“Server SR - 29397542” daemon prio=1 tid=0x08312a48 nid=0x6363 runnable

at java.net.SocketInputStream.socketRead0(Native Method)

at java.net.SocketInputStream.read(Unknown Source)

at sun.nio.cs.StreamDecoder$CharsetSD.readBytes(Unknown Source)

at sun.nio.cs.StreamDecoder$CharsetSD.implRead(Unknown Source)

at sun.nio.cs.StreamDecoder.read(Unknown Source)

  • locked <0x7b64dab0> (a java.io.InputStreamReader)

at java.io.InputStreamReader.read(Unknown Source)

at org.xmlpull.mxp1.MXParser.fillBuf(MXParser.java:2971)

at org.xmlpull.mxp1.MXParser.more(MXParser.java:3025)

at org.jivesoftware.wildfire.net.MXParser.nextImpl(MXParser.java:75)

at org.xmlpull.mxp1.MXParser.nextToken(MXParser.java:1100)

at org.dom4j.io.XMPPPacketReader.parseDocument(XMPPPacketReader.java:290)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:174)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 16619879” daemon prio=1 tid=0x08376c60 nid=0x6347 waiting for monitor entry

at org.jivesoftware.wildfire.user.UserManager.getUser(UserManager.java:169)

  • waiting to lock (a java.lang.String)

at org.jivesoftware.wildfire.user.User.getRoster(User.java:280)

at org.jivesoftware.wildfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:187)

at org.jivesoftware.wildfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :104)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 16408949” daemon prio=1 tid=0x08375cf8 nid=0x6326 waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bddd4d0> (a java.lang.String)

at org.jivesoftware.wildfire.user.User.getRoster(User.java:280)

at org.jivesoftware.wildfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:187)

at org.jivesoftware.wildfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :104)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“pool-214-thread-1” prio=1 tid=0x081a41d0 nid=0x62f6 waiting on condition

at sun.misc.Unsafe.park(Native Method)

at java.util.concurrent.locks.LockSupport.park(Unknown Source)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unk nown Source)

at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

“Server SR - 13540318” daemon prio=1 tid=0x085da000 nid=0x62f5 runnable

at java.net.SocketInputStream.socketRead0(Native Method)

at java.net.SocketInputStream.read(Unknown Source)

at sun.nio.cs.StreamDecoder$CharsetSD.readBytes(Unknown Source)

at sun.nio.cs.StreamDecoder$CharsetSD.implRead(Unknown Source)

at sun.nio.cs.StreamDecoder.read(Unknown Source)

  • locked <0x7b584750> (a java.io.InputStreamReader)

at java.io.InputStreamReader.read(Unknown Source)

at org.xmlpull.mxp1.MXParser.fillBuf(MXParser.java:2971)

at org.xmlpull.mxp1.MXParser.more(MXParser.java:3025)

at org.jivesoftware.wildfire.net.MXParser.nextImpl(MXParser.java:320)

at org.xmlpull.mxp1.MXParser.nextToken(MXParser.java:1100)

at org.dom4j.io.XMPPPacketReader.parseDocument(XMPPPacketReader.java:290)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:174)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 25595509” daemon prio=1 tid=0x081a31f8 nid=0x62eb waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bddd4d0> (a java.lang.String)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.broadcastUpdate(Presenc eUpdateHandler.java:257)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:109)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:153)
    at org.jivesoftware.wildfire.SessionManager$ClientSessionListener.onConnectionClos e(SessionManager.java:1288)
    at org.jivesoftware.wildfire.net.SocketConnection.notifyCloseListeners(SocketConne ction.java:552)
  • locked <0x630f99d0> (a java.util.HashMap)

at org.jivesoftware.wildfire.net.SocketConnection.close(SocketConnection.java:374)

at org.jivesoftware.wildfire.handler.IQAuthHandler.login(IQAuthHandler.java:199)

at org.jivesoftware.wildfire.handler.IQAuthHandler.handleIQ(IQAuthHandler.java:140 )

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 32349505” daemon prio=1 tid=0x08315af0 nid=0x62d7 waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bdef0d0> (a java.lang.String)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.broadcastUpdate(Presenc eUpdateHandler.java:257)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:109)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:153)
    at org.jivesoftware.wildfire.SessionManager$ClientSessionListener.onConnectionClos e(SessionManager.java:1288)
    at org.jivesoftware.wildfire.net.SocketConnection.notifyCloseListeners(SocketConne ction.java:552)
  • locked <0x7b50f788> (a java.util.HashMap)

at org.jivesoftware.wildfire.net.SocketConnection.close(SocketConnection.java:374)

at org.jivesoftware.wildfire.handler.IQBindHandler.handleIQ(IQBindHandler.java:111 )

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“pool-213-thread-1” prio=1 tid=0x08314c10 nid=0x62cd waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bdd48b0> (a java.lang.String)

at org.jivesoftware.wildfire.spi.PresenceManagerImpl.handleProbe(PresenceManagerIm pl.java:187)

at org.jivesoftware.wildfire.PresenceRouter.handle(PresenceRouter.java:120)

at org.jivesoftware.wildfire.PresenceRouter.route(PresenceRouter.java:61)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:73)

at org.jivesoftware.wildfire.net.SocketReader.processPresence(SocketReader.java:44 5)

at org.jivesoftware.wildfire.net.ServerSocketReader.access$101(ServerSocketReader. java:45)

at org.jivesoftware.wildfire.net.ServerSocketReader$2.run(ServerSocketReader.java: 101)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

“pool-212-thread-1” prio=1 tid=0x08314118 nid=0x62b1 waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bcc0868> (a java.lang.String)

at org.jivesoftware.wildfire.spi.PresenceManagerImpl.handleProbe(PresenceManagerIm pl.java:187)

at org.jivesoftware.wildfire.PresenceRouter.handle(PresenceRouter.java:120)

at org.jivesoftware.wildfire.PresenceRouter.route(PresenceRouter.java:61)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:73)

at org.jivesoftware.wildfire.net.SocketReader.processPresence(SocketReader.java:44 5)

at org.jivesoftware.wildfire.net.ServerSocketReader.access$101(ServerSocketReader. java:45)

at org.jivesoftware.wildfire.net.ServerSocketReader$2.run(ServerSocketReader.java: 101)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

“Client SR - 7790866” daemon prio=1 tid=0x0831a018 nid=0x6262 waiting for monitor entry

at org.jivesoftware.wildfire.user.UserManager.getUser(UserManager.java:169)

  • waiting to lock (a java.lang.String)

at org.jivesoftware.wildfire.user.User.getRoster(User.java:280)

at org.jivesoftware.wildfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:187)

at org.jivesoftware.wildfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :104)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 28245371” daemon prio=1 tid=0x083190b0 nid=0x624a waiting for monitor entry

at org.jivesoftware.wildfire.user.UserManager.getUser(UserManager.java:169)

  • waiting to lock (a java.lang.String)

at org.jivesoftware.wildfire.user.User.getRoster(User.java:280)

at org.jivesoftware.wildfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:187)

at org.jivesoftware.wildfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :104)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 19417131” daemon prio=1 tid=0x08318128 nid=0x622c waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bd79d98> (a java.lang.String)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.broadcastUpdate(Presenc eUpdateHandler.java:257)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:109)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:153)
    at org.jivesoftware.wildfire.SessionManager$ClientSessionListener.onConnectionClos e(SessionManager.java:1288)
    at org.jivesoftware.wildfire.net.SocketConnection.notifyCloseListeners(SocketConne ction.java:552)
  • locked <0x7b5afaa8> (a java.util.HashMap)

at org.jivesoftware.wildfire.net.SocketConnection.close(SocketConnection.java:374)

at org.jivesoftware.wildfire.handler.IQAuthHandler.login(IQAuthHandler.java:199)

at org.jivesoftware.wildfire.handler.IQAuthHandler.handleIQ(IQAuthHandler.java:140 )

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 26708076” daemon prio=1 tid=0x08317088 nid=0x6228 waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bd79d98> (a java.lang.String)

at org.jivesoftware.wildfire.user.User.getRoster(User.java:280)

at org.jivesoftware.wildfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:187)

at org.jivesoftware.wildfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :104)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 6534752” daemon prio=1 tid=0x085dd8b8 nid=0x61f3 waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bdae870> (a java.lang.String)

at org.jivesoftware.wildfire.user.User.getRoster(User.java:280)

at org.jivesoftware.wildfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:187)

at org.jivesoftware.wildfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :104)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“pool-210-thread-1” prio=1 tid=0x085daf60 nid=0x61e6 waiting on condition

at sun.misc.Unsafe.park(Native Method)

at java.util.concurrent.locks.LockSupport.park(Unknown Source)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unk nown Source)

at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

“Server SR - 30925696” daemon prio=1 tid=0x081cf870 nid=0x61e5 runnable

at java.net.SocketInputStream.socketRead0(Native Method)

at java.net.SocketInputStream.read(Unknown Source)

at sun.nio.cs.StreamDecoder$CharsetSD.readBytes(Unknown Source)

at sun.nio.cs.StreamDecoder$CharsetSD.implRead(Unknown Source)

at sun.nio.cs.StreamDecoder.read(Unknown Source)

  • locked <0x7b3dbff0> (a java.io.InputStreamReader)

at java.io.InputStreamReader.read(Unknown Source)

at org.xmlpull.mxp1.MXParser.fillBuf(MXParser.java:2971)

at org.xmlpull.mxp1.MXParser.more(MXParser.java:3025)

at org.jivesoftware.wildfire.net.MXParser.nextImpl(MXParser.java:75)

at org.xmlpull.mxp1.MXParser.nextToken(MXParser.java:1100)

at org.dom4j.io.XMPPPacketReader.parseDocument(XMPPPacketReader.java:290)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:174)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Outgoing Server Reader” daemon prio=1 tid=0x081cced8 nid=0x61e1 runnable

at java.net.SocketInputStream.socketRead0(Native Method)

at java.net.SocketInputStream.read(Unknown Source)

at sun.nio.cs.StreamDecoder$CharsetSD.readBytes(Unknown Source)

at sun.nio.cs.StreamDecoder$CharsetSD.implRead(Unknown Source)

at sun.nio.cs.StreamDecoder.read(Unknown Source)

  • locked <0x7b4ef528> (a java.io.InputStreamReader)

at java.io.InputStreamReader.read(Unknown Source)

at org.xmlpull.mxp1.MXParser.fillBuf(MXParser.java:2971)

at org.xmlpull.mxp1.MXParser.more(MXParser.java:3025)

at org.jivesoftware.wildfire.net.MXParser.nextImpl(MXParser.java:320)

at org.xmlpull.mxp1.MXParser.nextToken(MXParser.java:1100)

at org.dom4j.io.XMPPPacketReader.parseDocument(XMPPPacketReader.java:290)

at org.jivesoftware.wildfire.server.OutgoingServerSocketReader$1.run(OutgoingServe rSocketReader.java:91)

“Client SR - 32437185” daemon prio=1 tid=0x08075098 nid=0x6180 waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bd79d98> (a java.lang.String)

at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.broadcastUpdate(Presenc eUpdateHandler.java:257)

at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:109)

at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:153)

at org.jivesoftware.wildfire.PresenceRouter.handle(PresenceRouter.java:92)

at org.jivesoftware.wildfire.PresenceRouter.route(PresenceRouter.java:61)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:73)

at org.jivesoftware.wildfire.SessionManager.removeSession(SessionManager.java:1238 )

at org.jivesoftware.wildfire.SessionManager.getBestRoute(SessionManager.java:745)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:277)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 6526027” daemon prio=1 tid=0x085eebe8 nid=0x6166 waiting for monitor entry

at org.jivesoftware.wildfire.user.UserManager.getUser(UserManager.java:169)

  • waiting to lock (a java.lang.String)

at org.jivesoftware.wildfire.user.User.getRoster(User.java:280)

at org.jivesoftware.wildfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:187)

at org.jivesoftware.wildfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :104)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“pool-209-thread-1” prio=1 tid=0x08378a60 nid=0x614e waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bdd24c8> (a java.lang.String)

at org.jivesoftware.wildfire.spi.PresenceManagerImpl.handleProbe(PresenceManagerIm pl.java:187)

at org.jivesoftware.wildfire.PresenceRouter.handle(PresenceRouter.java:120)

at org.jivesoftware.wildfire.PresenceRouter.route(PresenceRouter.java:61)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:73)

at org.jivesoftware.wildfire.net.SocketReader.processPresence(SocketReader.java:44 5)

at org.jivesoftware.wildfire.net.ServerSocketReader.access$101(ServerSocketReader. java:45)

at org.jivesoftware.wildfire.net.ServerSocketReader$2.run(ServerSocketReader.java: 101)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

“Client SR - 11379670” daemon prio=1 tid=0x08385d98 nid=0x611b waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bddca28> (a java.lang.String)

at org.jivesoftware.wildfire.user.User.getRoster(User.java:280)

at org.jivesoftware.wildfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:187)

at org.jivesoftware.wildfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :104)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 19816691” daemon prio=1 tid=0x08384bb0 nid=0x610f waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bddca28> (a java.lang.String)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.broadcastUpdate(Presenc eUpdateHandler.java:257)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:109)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:153)
    at org.jivesoftware.wildfire.SessionManager$ClientSessionListener.onConnectionClos e(SessionManager.java:1288)
    at org.jivesoftware.wildfire.net.SocketConnection.notifyCloseListeners(SocketConne ction.java:552)
  • locked <0x6309ded0> (a java.util.HashMap)

at org.jivesoftware.wildfire.net.SocketConnection.close(SocketConnection.java:374)

at org.jivesoftware.wildfire.handler.IQBindHandler.handleIQ(IQBindHandler.java:111 )

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 30656253” daemon prio=1 tid=0x081d0798 nid=0x610a waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bddca28> (a java.lang.String)

at org.jivesoftware.wildfire.user.User.getRoster(User.java:280)

at org.jivesoftware.wildfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:187)

at org.jivesoftware.wildfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :104)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 32111476” daemon prio=1 tid=0x081cfbc0 nid=0x6103 waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bddca28> (a java.lang.String)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.broadcastUpdate(Presenc eUpdateHandler.java:257)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:109)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:153)
    at org.jivesoftware.wildfire.SessionManager$ClientSessionListener.onConnectionClos e(SessionManager.java:1288)
    at org.jivesoftware.wildfire.net.SocketConnection.notifyCloseListeners(SocketConne ction.java:552)
  • locked <0x630d32e8> (a java.util.HashMap)

at org.jivesoftware.wildfire.net.SocketConnection.close(SocketConnection.java:374)

at org.jivesoftware.wildfire.handler.IQBindHandler.handleIQ(IQBindHandler.java:111 )

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 1660570” daemon prio=1 tid=0x08379900 nid=0x60f9 waiting for monitor entry

at org.jivesoftware.wildfire.user.UserManager.getUser(UserManager.java:169)

  • waiting to lock (a java.lang.String)

at org.jivesoftware.wildfire.user.User.getRoster(User.java:280)

at org.jivesoftware.wildfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:187)

at org.jivesoftware.wildfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :104)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 22101968” daemon prio=1 tid=0x080d2758 nid=0x60f5 waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bd79d98> (a java.lang.String)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.broadcastUpdate(Presenc eUpdateHandler.java:257)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:109)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:153)
    at org.jivesoftware.wildfire.SessionManager$ClientSessionListener.onConnectionClos e(SessionManager.java:1288)
    at org.jivesoftware.wildfire.net.SocketConnection.notifyCloseListeners(SocketConne ction.java:552)
  • locked <0x6261d790> (a java.util.HashMap)

at org.jivesoftware.wildfire.net.SocketConnection.close(SocketConnection.java:374)

at org.jivesoftware.wildfire.handler.IQAuthHandler.login(IQAuthHandler.java:199)

at org.jivesoftware.wildfire.handler.IQAuthHandler.handleIQ(IQAuthHandler.java:140 )

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“pool-205-thread-1” prio=1 tid=0x0844d078 nid=0x60c1 waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bdd48b0> (a java.lang.String)

at org.jivesoftware.wildfire.spi.PresenceManagerImpl.handleProbe(PresenceManagerIm pl.java:187)

at org.jivesoftware.wildfire.PresenceRouter.handle(PresenceRouter.java:120)

at org.jivesoftware.wildfire.PresenceRouter.route(PresenceRouter.java:61)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:73)

at org.jivesoftware.wildfire.net.SocketReader.processPresence(SocketReader.java:44 5)

at org.jivesoftware.wildfire.net.ServerSocketReader.access$101(ServerSocketReader. java:45)

at org.jivesoftware.wildfire.net.ServerSocketReader$2.run(ServerSocketReader.java: 101)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

“Client SR - 13549340” daemon prio=1 tid=0x0844d828 nid=0x60a9 waiting for monitor entry

at org.jivesoftware.wildfire.user.UserManager.getUser(UserManager.java:169)

  • waiting to lock (a java.lang.String)

at org.jivesoftware.wildfire.user.User.getRoster(User.java:280)

at org.jivesoftware.wildfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:187)

at org.jivesoftware.wildfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :104)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 13148199” daemon prio=1 tid=0x081f1978 nid=0x60a3 waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bdd48b0> (a java.lang.String)

at org.jivesoftware.wildfire.user.User.getRoster(User.java:280)

at org.jivesoftware.wildfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:187)

at org.jivesoftware.wildfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :104)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 16447937” daemon prio=1 tid=0x0839b428 nid=0x607c waiting for monitor entry

at org.jivesoftware.wildfire.user.UserManager.getUser(UserManager.java:169)

  • waiting to lock (a java.lang.String)

at org.jivesoftware.wildfire.user.User.getRoster(User.java:280)

at org.jivesoftware.wildfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:187)

at org.jivesoftware.wildfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :104)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 4816459” daemon prio=1 tid=0x0819f118 nid=0x6075 waiting for monitor entry

at org.jivesoftware.wildfire.user.UserManager.getUser(UserManager.java:169)

  • waiting to lock (a java.lang.String)

at org.jivesoftware.wildfire.user.User.getRoster(User.java:280)

at org.jivesoftware.wildfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:187)

at org.jivesoftware.wildfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :104)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 18163498” daemon prio=1 tid=0x08405e58 nid=0x606e waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bddc208> (a java.lang.String)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.broadcastUpdate(Presenc eUpdateHandler.java:257)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:109)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:153)
    at org.jivesoftware.wildfire.SessionManager$ClientSessionListener.onConnectionClos e(SessionManager.java:1288)
    at org.jivesoftware.wildfire.net.SocketConnection.notifyCloseListeners(SocketConne ction.java:552)
  • locked <0x62e953f0> (a java.util.HashMap)
    at org.jivesoftware.wildfire.net.SocketConnection.forceClose(SocketConnection.java :444)
    at org.jivesoftware.wildfire.net.SocketConnection.checkHealth(SocketConnection.jav a:404)
    at org.jivesoftware.wildfire.net.SocketConnection.requestWriting(SocketConnection. java:569)
    at org.jivesoftware.wildfire.net.SocketConnection.close(SocketConnection.java:346)
  • locked <0x62e95300> (a org.jivesoftware.wildfire.net.SocketConnection)

at org.jivesoftware.wildfire.net.SocketConnection.validate(SocketConnection.java:1 97)

at org.jivesoftware.wildfire.SessionManager.isActiveRoute(SessionManager.java:785)

at org.jivesoftware.wildfire.handler.IQBindHandler.handleIQ(IQBindHandler.java:97)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“pool-203-thread-1” prio=1 tid=0x081a8f58 nid=0x602c waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bdd48b0> (a java.lang.String)

at org.jivesoftware.wildfire.spi.PresenceManagerImpl.handleProbe(PresenceManagerIm pl.java:187)

at org.jivesoftware.wildfire.PresenceRouter.handle(PresenceRouter.java:120)

at org.jivesoftware.wildfire.PresenceRouter.route(PresenceRouter.java:61)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:73)

at org.jivesoftware.wildfire.net.SocketReader.processPresence(SocketReader.java:44 5)

at org.jivesoftware.wildfire.net.ServerSocketReader.access$101(ServerSocketReader. java:45)

at org.jivesoftware.wildfire.net.ServerSocketReader$2.run(ServerSocketReader.java: 101)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

“Client SR - 27383243” daemon prio=1 tid=0x0819df78 nid=0x5fb7 waiting for monitor entry

at org.jivesoftware.wildfire.user.UserManager.getUser(UserManager.java:169)

  • waiting to lock (a java.lang.String)

at org.jivesoftware.wildfire.user.User.getRoster(User.java:280)

at org.jivesoftware.wildfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:187)

at org.jivesoftware.wildfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :104)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 31765821” daemon prio=1 tid=0x0830d870 nid=0x5f9e waiting for monitor entry

at org.jivesoftware.wildfire.user.UserManager.getUser(UserManager.java:169)

  • waiting to lock (a java.lang.String)

at org.jivesoftware.wildfire.user.User.getRoster(User.java:280)

at org.jivesoftware.wildfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:187)

at org.jivesoftware.wildfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :104)

at org.jivesoftware.wildfire.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.wildfire.IQRouter.handle(IQRouter.java:265)

at org.jivesoftware.wildfire.IQRouter.route(IQRouter.java:96)

at org.jivesoftware.wildfire.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.wildfire.net.SocketReader.processIQ(SocketReader.java:396)

at org.jivesoftware.wildfire.net.ClientSocketReader.processIQ(ClientSocketReader.j ava:50)

at org.jivesoftware.wildfire.net.SocketReader.readStream(SocketReader.java:264)

at org.jivesoftware.wildfire.net.SocketReader.run(SocketReader.java:119)

at java.lang.Thread.run(Unknown Source)

“Client SR - 14396291” daemon prio=1 tid=0x08399740 nid=0x5f62 waiting for monitor entry

at org.jivesoftware.wildfire.roster.RosterManager.getRoster(RosterManager.java:88)

  • waiting to lock <0x5bdae870> (a java.lang.String)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.broadcastUpdate(Presenc eUpdateHandler.java:257)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:109)
    at org.jivesoftware.wildfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:153)
    at org.jivesoftware.wildfire.SessionManager$ClientSessionListener.onConnectionClos e(SessionManager.java:1288)
    at org.jivesoftware.wildfire.net.SocketConnection.notifyCloseListeners(SocketConne ction.java:552)
  • locked <0x629ab908> (a java.util.HashMap)
    at org.jivesoftware.wildfire.net.SocketConnection.forceClose(SocketConnection.java :444)
    at org.jivesoftware.wildfire.net.SocketConnection.checkHealth(SocketConnection.jav a:417)
    at org.jivesoftware.wildfire.net.SocketConnection.requestWriting(SocketConnection. java:569)
    at org.jivesoftware.wildfire.net.SocketConnection.close(SocketConnection.java:346)
  • locked <0x629a97d0> (a org.jivesoftware.wildfire.net.SocketConnection)

at org.jivesoftware.wildfire.net.SocketConnection.validate(SocketConnection.java:1 97)

at org.jivesoftware

jmcadams,

Thanks for the detailed information. Here’'s what I see:

  • A bunch of threads locked on monitor 0x5bcab758. That’'s the synchronized line in the getUser method:

public User getUser(String username) throws UserNotFoundException {

// Make sure that the username is valid.

username = username.trim().toLowerCase();

User user = userCache.get(username);

if (user == null) {

synchronized (username.intern()) {

user = userCache.get(username);

if (user == null) {

user = provider.loadUser(username);

userCache.put(username, user);

}

}

}

return user;

}[/code]

  • One thread has locked the monitor. It appears to be waiting in some Sun SSL code.

So, this isn’'t a deadlock, but it is some sort of bottleneck. A few things to try:

  1. Try taking two thread dumps in a row about 10 seconds apart. That will tell us if the threads are moving or not.

  2. Enable LDAP tracing. It would be good to see if there is any more detailed info about what the LDAP server is doing.

  3. See if you can enable logging in your LDAP server. Is Wildfire attempting some very slow operation?? Maybe you need to add indexes to your directory to aid in query performance?

A quick Google search didn’'t turn up anything, so those are my best ideas at the moment.

Thanks,

Matt

removing FUD

Conor

Message was edited by: Conor

Hey Conor,

The point of calling the intern is to make sure you have that internal reference so if there is more than one call for that particular user’‘s roster there won’'t be a synchronization issue as the monitor will lock out the second requestor until the first request has been completed.

Hope that helps,

Alex

  • One thread has locked the monitor. It appears to be

waiting in some Sun SSL code.

Hrmm…ok…we’‘re using eDir, and we have to do binding over TLS or SSL (I think I’'m configured to do ldaps, rather than using start_tls. Perhaps something with the SSL connection to the LDAP server is hanging. :confused:

So, this isn’'t a deadlock, but it is some sort of

bottleneck. A few things to try:

  1. Try taking two thread dumps in a row about 10

seconds apart. That will tell us if the threads are

moving or not.

I’‘m pretty sure they’‘re not, but I’'ll try this next time around.

  1. Enable LDAP tracing. It would be good to see if

there is any more detailed info about what the LDAP

server is doing.

OK, this is gonna be gargantuan, but I’'ll enable it so its on for the next startup.

  1. See if you can enable logging in your LDAP server.

Is Wildfire attempting some very slow operation??

Maybe you need to add indexes to your directory to

aid in query performance?

Nothing in the LDAP server has ever responded that slowly…we’‘re not pushing eDir’'s capabilities at all…not even close.

A quick Google search didn’'t turn up anything, so

those are my best ideas at the moment.

Well…certainly some things to look at anyway.

Good starting points.

I’'ll try to look into the LDAPS stuff a bit more as well.

Jeff