Openfire locks up every 24 hours

My openfire setup started to lock up every 24 hours or so after the 3.7 upgrade.

Issue is the logs for some reason are empty so i am unable to tell what the cause is. Every time I enabled debugging and restart openfire debugging goes back to desabled.

Also wehn i click on remove plug in it states that plug in removed, if i refrwesh the page the plug in is back in the list and running.

Anyone have any idea what would cause this?

Hello,

Is it exhausting memory?

daryl

Memory does not seem to be an issue, i have nagios monitoring the server and memory usage seems to be fine.

Today i also monitored MySQl connections to make sure I am not hitting the max 25 connections but it stays at solid 13 connections while running.

I think it has something to do with me not able to delte plug insa dn logs not running.

This might be helpful. But I noticed after reboot there are still multiple established connections on port 5222.


COMMAND  PID USER   FD   TYPE DEVICE SIZE NODE NAME

java    2567 root   90u  IPv6  13064       TCP chat.prnpt.com:xmpp-client->ts5.prnpt.com:2163 (ESTABLISHED)

java    2567 root   95u  IPv6   6084       TCP *:xmpp-client (LISTEN)

java    2567 root  108u  IPv6  14448       TCP chat.prnpt.com:xmpp-client->ts1.prnpt.com:4833 (ESTABLISHED)

java    2567 root  112u  IPv6  13083       TCP chat.prnpt.com:xmpp-client->ts2.prnpt.com:2941 (ESTABLISHED)

java    2567 root  117u  IPv6  13313       TCP chat.prnpt.com:xmpp-client->ts1.prnpt.com:4792 (ESTABLISHED)

java    2567 root  119u  IPv6  13430       TCP chat.prnpt.com:xmpp-client->m604.milpitas.prnpt.com:60128 (ESTABLISHED)

java    2567 root  120u  IPv6  13302       TCP chat.prnpt.com:xmpp-client->ts2.prnpt.com:2951 (ESTABLISHED)

java    2567 root  121u  IPv6  13327       TCP chat.prnpt.com:xmpp-client->128.0.16.244:3373 (ESTABLISHED)

java    2567 root  122u  IPv6  15669       TCP chat.prnpt.com:xmpp-client->ts5.prnpt.com:2274 (ESTABLISHED)

java    2567 root  123u  IPv6  13136       TCP chat.prnpt.com:xmpp-client->ts2.prnpt.com:2946 (ESTABLISHED)

java    2567 root  124u  IPv6  13453       TCP chat.prnpt.com:xmpp-client->ts1.prnpt.com:4798 (ESTABLISHED)

java    2567 root  125u  IPv6   6129       TCP chat.prnpt.com:xmpp-client->it1876.prnpt.com:53262 (ESTABLISHED)

java    2567 root  126u  IPv6  14976       TCP chat.prnpt.com:xmpp-client->ts2.prnpt.com:3107 (ESTABLISHED)

java    2567 root  127u  IPv6  14453       TCP chat.prnpt.com:xmpp-client->it1824.prnpt.com:1784 (ESTABLISHED)

java    2567 root  128u  IPv6  13518       TCP chat.prnpt.com:xmpp-client->ts1.prnpt.com:4803 (ESTABLISHED)

java    2567 root  129u  IPv6   8606       TCP chat.prnpt.com:xmpp-client->ts4.prnpt.com:1988 (ESTABLISHED)

java    2567 root  130u  IPv6  13531       TCP chat.prnpt.com:xmpp-client->it1883.prnpt.com:4776 (ESTABLISHED)

java    2567 root  131u  IPv6  14715       TCP chat.prnpt.com:xmpp-client->it1879.prnpt.com:52745 (ESTABLISHED)

java    2567 root  132u  IPv6  13546       TCP chat.prnpt.com:xmpp-client->ts3.prnpt.com:1658 (ESTABLISHED)

java    2567 root  133r  IPv6  14485       TCP chat.prnpt.com:xmpp-client->ts3.prnpt.com:1699 (ESTABLISHED)

java    2567 root  134w  IPv6  11200       TCP chat.prnpt.com:xmpp-client->m606.prnpt.com:3056 (ESTABLISHED)

java    2567 root  135u  IPv6  14494       TCP chat.prnpt.com:xmpp-client->ts4.prnpt.com:2091 (ESTABLISHED)

java    2567 root  136u  IPv6  14639       TCP chat.prnpt.com:xmpp-client->ts3.prnpt.com:1704 (ESTABLISHED)

java    2567 root  137u  IPv6  15769       TCP chat.prnpt.com:xmpp-client->ts1.prnpt.com:4871 (ESTABLISHED)

java    2567 root  138u  IPv6  14561       TCP chat.prnpt.com:xmpp-client->ts4.prnpt.com:2096 (ESTABLISHED)

java    2567 root  139u  IPv6  14799       TCP chat.prnpt.com:xmpp-client->ts4.prnpt.com:2110 (ESTABLISHED)

java    2567 root  140u  IPv6  14860       TCP chat.prnpt.com:xmpp-client->ts2.prnpt.com:3097 (ESTABLISHED)

java    2567 root  141u  IPv6   6887       TCP chat.prnpt.com:xmpp-client->ts3.prnpt.com:1552 (ESTABLISHED)

java    2567 root  142u  IPv6   6890       TCP chat.prnpt.com:xmpp-client->ts1.prnpt.com:4712 (ESTABLISHED)

java    2567 root  143u  IPv6   6955       TCP chat.prnpt.com:xmpp-client->it1817.prnpt.com:3785 (ESTABLISHED)

java    2567 root  144u  IPv6   6931       TCP chat.prnpt.com:xmpp-client->ts5.prnpt.com:rtcm-sc104 (ESTABLISHED)

java    2567 root  145u  IPv6   6926       TCP chat.prnpt.com:xmpp-client->ts4.prnpt.com:1968 (ESTABLISHED)

java    2567 root  146u  IPv6  15334       TCP chat.prnpt.com:xmpp-client->ts5.prnpt.com:2244 (ESTABLISHED)

java    2567 root  147u  IPv6  15336       TCP chat.prnpt.com:xmpp-client->ts4.prnpt.com:2120 (ESTABLISHED)

java    2567 root  148u  IPv6   6978       TCP chat.prnpt.com:xmpp-client->it0473.prnpt.com:1662 (ESTABLISHED)

java    2567 root  150u  IPv6  15674       TCP chat.prnpt.com:xmpp-client->ts1.prnpt.com:4845 (ESTABLISHED)

java    2567 root  151u  IPv6  15366       TCP chat.prnpt.com:xmpp-client->ts4.prnpt.com:2125 (ESTABLISHED)

java    2567 root  152u  IPv6  15369       TCP chat.prnpt.com:xmpp-client->ts3.prnpt.com:1733 (ESTABLISHED)

java    2567 root  155u  IPv6   8578       TCP chat.prnpt.com:xmpp-client->ts2.prnpt.com:2874 (ESTABLISHED)

java    2567 root  157u  IPv6   7324       TCP chat.prnpt.com:xmpp-client->ts2.prnpt.com:2869 (ESTABLISHED)

java    2567 root  161u  IPv6   8630       TCP chat.prnpt.com:xmpp-client->ts1.prnpt.com:4739 (ESTABLISHED)

java    2567 root  162u  IPv6   8632       TCP chat.prnpt.com:xmpp-client->ts5.prnpt.com:2112 (ESTABLISHED)

java    2567 root  163u  IPv6   8634       TCP chat.prnpt.com:xmpp-client->it1822.prnpt.com:1968 (ESTABLISHED)

java    2567 root  164u  IPv6   8674       TCP chat.prnpt.com:xmpp-client->ts5.prnpt.com:2117 (ESTABLISHED)

java    2567 root  166u  IPv6   8870       TCP chat.prnpt.com:xmpp-client->ts3.prnpt.com:1564 (ESTABLISHED)

java    2567 root  178u  IPv6  15835       TCP chat.prnpt.com:xmpp-client->ts1.prnpt.com:4885 (ESTABLISHED)

java    2567 root  179r  IPv6  15845       TCP chat.prnpt.com:xmpp-client->m305.milpitas.prnpt.com:50952 (ESTABLISHED)

I might have met the same issue as you.

I have an Openfire running on a Linux with the latest JRE. It seems to work just fine at daytime, but the next morning I realize no more could login with any XMPP clients.

From the packets captured, it seems the port 5222 is still available but wouldn’t repond anything.

Do you have any ideas?

Oh, in addition, as soon as I restart it, everything comes back to be fine. But I don’t want to restart it everyday.

On the Openfire admin console, what does it show for memory usage?

daryl

Hmm good point currently it is at 60 %

I will keep an eye on it throught the day and see if it raises once everybody signs on.

It is odd that it is at 60 % with 40 users, I have 500mb alocated to the java memory.

Could there be a memory leak?

Looks like memory might be the issue it just jumped up to 94% still runing, but only 20 more users signed on from when it was at 60% this makes no sense.

Any way to adjust the java machine memory?

Just found the mem leakage notice, going to try disabpling PEP and see if it helps after a reboot tongiht.

http://community.igniterealtime.org/index.jspa#

Apperently the 3.6.4 memory leak has not been fixed in 3.7.0

So you might want to give this a try also.

http://community.igniterealtime.org/thread/40018

Dunno if it works as i will not have a chance to test it till after office hours tonight.

I think i found another issue that would cause it to take a long time for users to connect after a server reboot.

I use LDAp for my user and group listings.

For some reason if you delete the group from AD and than reboot the server it sits there and keeps spitting out the error over and over again that group is not found.

This needs to be fixed after not finding a group a few times openfire should remove this group form AD.

Anybody else run into this?

2011.03.31 12:45:52 Groups ([OF SoCal Purchasing]) include non-existent username (rrodriguez)

2011.03.31 12:45:52 Groups ([OF La Jolla]) include non-existent username (lajolla)

2011.03.31 12:45:52 Groups ([OF Santa Clara]) include non-existent username (acasem)

2011.03.31 12:45:52 Groupname OF 1 not found

org.jivesoftware.openfire.group.GroupNotFoundException: Groupname OF 1 not found

at org.jivesoftware.openfire.ldap.LdapManager.findGroupDN(LdapManager.java:1109)

at org.jivesoftware.openfire.ldap.LdapManager.findGroupDN(LdapManager.java:1036)

at org.jivesoftware.openfire.ldap.LdapGroupProvider.getGroup(LdapGroupProvider.jav a:102)

at org.jivesoftware.openfire.group.GroupManager.getGroup(GroupManager.java:294)

at org.jivesoftware.openfire.group.GroupManager.getGroup(GroupManager.java:273)

at org.jivesoftware.openfire.group.GroupCollection$UserIterator.getNextElement(Gro upCollection.java:113)

at org.jivesoftware.openfire.group.GroupCollection$UserIterator.hasNext(GroupColle ction.java:76)

at org.jivesoftware.openfire.roster.RosterManager.getSharedGroups(RosterManager.ja va:170)

at org.jivesoftware.openfire.roster.Roster.(Roster.java:127)

at org.jivesoftware.openfire.roster.RosterManager.getRoster(RosterManager.java:94)

at org.jivesoftware.openfire.handler.PresenceUpdateHandler.broadcastUpdate(Presenc eUpdateHandler.java:305)

at org.jivesoftware.openfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:147)

at org.jivesoftware.openfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:135)

at org.jivesoftware.openfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:199)

at org.jivesoftware.openfire.PresenceRouter.handle(PresenceRouter.java:149)

at org.jivesoftware.openfire.PresenceRouter.route(PresenceRouter.java:85)

at org.jivesoftware.openfire.spi.PacketRouterImpl.route(PacketRouterImpl.java:84)

at org.jivesoftware.openfire.net.StanzaHandler.processPresence(StanzaHandler.java: 355)

at org.jivesoftware.openfire.net.ClientStanzaHandler.processPresence(ClientStanzaH andler.java:100)

at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:272)

at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:194)

at org.jivesoftware.openfire.nio.ConnectionHandler.messageReceived(ConnectionHandl er.java:169)

at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.messageReceived (AbstractIoFilterChain.java:570)

at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)

at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)

at org.apache.mina.common.IoFilterAdapter.messageReceived(IoFilterAdapter.java:80)

at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)

at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)

at org.apache.mina.filter.codec.support.SimpleProtocolDecoderOutput.flush(SimplePr otocolDecoderOutput.java:58)

at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:185)

at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)

at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)

at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java :239)

at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(Execut orFilter.java:283)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java: 886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)

at java.lang.Thread.run(Thread.java:662)