Let me start off by saying that I am new to OpenFire, so this may be something really easy that I am overlooking - but I just can’t seem to figure out this problem so any suggestions would be GREATLY appreciated!
I just setup OpenFire 3.7.0 on a Windows Server a couple of weeks ago. It’s a VMware virtual machine running Windows 2008 Server Standard x64 with 4GB of memory allocated and unlimited CPU resources from dual Intel Xeon E5430 2.66Ghz Quad-Core processors. The VMware host also has 2 Broadcom NetXtreme II BCM5708 1Gb network cards and there are no bandwidth limitations or restrictions configured. The performance monitors in the vSphere client are not showing any major network, memory, or processor spikes - everything seems to be operating smoothly.
We went through about a week of testing with minimal users (about 10) all using Spark 2.6.0 as the client except for myself - I’m using Pidgin Portable 2.8.0. During the testing period with the minimal number of users everything was working fine - everyone could connect, the Rosters automatically populated as expected (although it would usually take a few minutes), statuses updated properly, and everyone could chat with each other as expected.
Now I have deployed the Spark client to all of our users (about 20 more) and we are now having some issues. It seems as if the first 17 users or so who connect are having no problems, but anyone who connects after that is having issues. When using the Spark client, the users can connect - but their Roster never appears and when i look at the ‘Client Sessions’ tab on the OpenFire server, they show as ‘Status = Closed’. Sometimes the presence says ‘Online’ or their custom status, and sometimes it says ‘Offline’. When I try to connect wth my Pidgin client, it says ‘Busy - Connecting’ for a LONG time, then finally comes back and says ‘Disconnected - Ping Timed Out’.
At this point I am open to any suggestions or advice on how to resolve or troubleshoot the issue. I’m not finding anything helpful in the Pidgin or Spark debug logs, at least not anything that stands out to me.
I cleared the Error log, tried to connect, then all this appeared:
2011.06.15 08:41:11 VCard provider is read-only.
java.lang.UnsupportedOperationException: VCard provider is read-only.
at org.jivesoftware.openfire.vcard.VCardManager.setVCard(VCardManager.java:158)
at org.jivesoftware.openfire.handler.IQvCardHandler.handleIQ(IQvCardHandler.java:9 5)
at org.jivesoftware.openfire.handler.IQHandler.process(IQHandler.java:65)
at org.jivesoftware.openfire.IQRouter.handle(IQRouter.java:372)
at org.jivesoftware.openfire.IQRouter.route(IQRouter.java:121)
at org.jivesoftware.openfire.spi.PacketRouterImpl.route(PacketRouterImpl.java:76)
at org.jivesoftware.openfire.net.StanzaHandler.processIQ(StanzaHandler.java:337)
at org.jivesoftware.openfire.net.ClientStanzaHandler.processIQ(ClientStanzaHandler .java:93)
at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:302)
at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:194)
at org.jivesoftware.openfire.nio.ConnectionHandler.messageReceived(ConnectionHandl er.java:169)
at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.messageReceived (AbstractIoFilterChain.java:570)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)
at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)
at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)
at org.apache.mina.common.IoFilterAdapter.messageReceived(IoFilterAdapter.java:80)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)
at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)
at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)
at org.apache.mina.filter.codec.support.SimpleProtocolDecoderOutput.flush(SimplePr otocolDecoderOutput.java:58)
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:185)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)
at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)
at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)
at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java :239)
at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(Execut orFilter.java:283)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)
at java.lang.Thread.run(Unknown Source)
2011.06.15 08:41:35 Groups ([FFAUsers]) include non-existent username (ffaexecs)
2011.06.15 08:41:36 Groups ([FFAUsers]) include non-existent username (ffaexecs)
2011.06.15 08:41:36 Groups ([FFAUsers]) include non-existent username (ffaexecs)
2011.06.15 08:41:36 Groups ([3FMUsers, 3FMManagers, FFFMUsers]) include non-existent username (fffmmanagers)
2011.06.15 08:41:36 Groups ([3FMUsers, 3FMManagers, FFFMUsers]) include non-existent username (fffmmanagers)
2011.06.15 08:41:36 Groups ([3FMUsers, 3FMManagers, FFFMUsers]) include non-existent username (fffmmanagers)
2011.06.15 08:41:37 Groups ([FFAUsers]) include non-existent username (ffanewaccounts)
2011.06.15 08:41:37 Groups ([FFAUsers]) include non-existent username (ffanewaccounts)
2011.06.15 08:41:37 Groups ([FFAUsers]) include non-existent username (ffanewaccounts)
2011.06.15 08:41:37 Groups ([3FMUsers]) include non-existent username (fffmusers)
2011.06.15 08:41:38 Groups ([3FMUsers]) include non-existent username (fffmusers)
2011.06.15 08:41:38 Groups ([3FMUsers]) include non-existent username (fffmusers)
2011.06.15 08:41:40 Groups ([FFFMManagers]) include non-existent username (3fmmanagers)
2011.06.15 08:41:40 Groups ([FFFMManagers]) include non-existent username (3fmmanagers)
2011.06.15 08:41:40 Groups ([FFFMManagers]) include non-existent username (3fmmanagers)
2011.06.15 08:41:40 Groups ([FFAUsers]) include non-existent username (ffaaccounting)
2011.06.15 08:41:40 Groups ([FFAUsers]) include non-existent username (dpausers)
2011.06.15 08:41:40 Groups ([FFAUsers]) include non-existent username (ffaaccounting)
2011.06.15 08:41:41 Groups ([FFAUsers]) include non-existent username (ffaaccounting)
2011.06.15 08:41:41 Groups ([FFAUsers]) include non-existent username (dpausers)
2011.06.15 08:41:41 Groups ([FFAUsers]) include non-existent username (dpausers)
2011.06.15 08:41:44 Groups ([FFAUsers]) include non-existent username (ffareception)
2011.06.15 08:41:44 Groups ([FFAUsers]) include non-existent username (ffareception)
2011.06.15 08:41:44 Groups ([FFAUsers]) include non-existent username (ffareception)
2011.06.15 08:41:44 Groups ([FFAUsers]) include non-existent username (ffamanagers)
2011.06.15 08:41:45 Groups ([FFAClientRelations]) include non-existent username (dpaclientrelations)
2011.06.15 08:41:45 Groups ([FFAUsers]) include non-existent username (ffamanagers)
2011.06.15 08:41:45 Groups ([FFAUsers]) include non-existent username (ffamanagers)
2011.06.15 08:41:45 Groups ([FFAClientRelations]) include non-existent username (dpaclientrelations)
2011.06.15 08:41:45 Groups ([FFAClientRelations]) include non-existent username (dpaclientrelations)
2011.06.15 08:41:46 Groups ([FFAUsers]) include non-existent username (it)
2011.06.15 08:41:46 Groups ([FFAUsers]) include non-existent username (it)
2011.06.15 08:41:46 Groups ([FFAUsers]) include non-existent username (it)
2011.06.15 08:41:47 Groups ([3FMUsers, FFFMProcessors]) include non-existent username (3fmprocessors)
2011.06.15 08:41:47 Groups ([3FMUsers, FFFMProcessors]) include non-existent username (3fmprocessors)
2011.06.15 08:41:48 Groups ([3FMUsers, FFFMProcessors]) include non-existent username (3fmprocessors)
2011.06.15 08:41:49 Groups ([FFAUsers]) include non-existent username (ithelpers)
2011.06.15 08:41:49 Groups ([FFAUsers]) include non-existent username (ithelpers)
2011.06.15 08:41:49 Groups ([FFAUsers]) include non-existent username (ithelpers)
2011.06.15 08:41:50 Groups ([FFAUsers]) include non-existent username (ffaexecs)
2011.06.15 08:41:50 Groups ([3FMUsers, 3FMManagers, FFFMUsers]) include non-existent username (fffmmanagers)
2011.06.15 08:41:50 Groups ([FFAUsers]) include non-existent username (ffanewaccounts)
2011.06.15 08:41:50 Groups ([3FMUsers]) include non-existent username (fffmusers)
2011.06.15 08:41:51 Groups ([FFFMManagers]) include non-existent username (3fmmanagers)
2011.06.15 08:41:51 Groups ([FFAUsers]) include non-existent username (ffaaccounting)
2011.06.15 08:41:51 Groups ([3FMProcessors, FFFMUsers]) include non-existent username (fffmprocessors)
2011.06.15 08:41:51 Groups ([FFAUsers]) include non-existent username (dpausers)
2011.06.15 08:41:51 Groups ([3FMProcessors, FFFMUsers]) include non-existent username (fffmprocessors)
2011.06.15 08:41:51 Groups ([FFAUsers]) include non-existent username (ffasupervisors)
2011.06.15 08:41:51 Groups ([FFAUsers]) include non-existent username (ffareception)
2011.06.15 08:41:51 Groups ([FFAUsers]) include non-existent username (ffasupervisors)
2011.06.15 08:41:51 Groups ([3FMProcessors, FFFMUsers]) include non-existent username (fffmprocessors)
2011.06.15 08:41:51 Groups ([FFAUsers]) include non-existent username (ffamanagers)
2011.06.15 08:41:52 Groups ([FFAUsers]) include non-existent username (ffasupervisors)
2011.06.15 08:41:52 Groups ([FFAClientRelations]) include non-existent username (dpaclientrelations)
2011.06.15 08:41:52 Groups ([FFAUsers]) include non-existent username (it)
2011.06.15 08:41:52 Groups ([3FMUsers, FFFMProcessors]) include non-existent username (3fmprocessors)
2011.06.15 08:41:52 Groups ([FFAManagers, FFAUsers]) include non-existent username (ffadirectors)
2011.06.15 08:41:52 Groups ([FFAUsers]) include non-existent username (ithelpers)
2011.06.15 08:41:52 Groups ([FFAManagers, FFAUsers]) include non-existent username (ffadirectors)
2011.06.15 08:41:52 Groups ([3FMProcessors, FFFMUsers]) include non-existent username (fffmprocessors)
2011.06.15 08:41:53 Groups ([FFAManagers, FFAUsers]) include non-existent username (ffadirectors)
2011.06.15 08:41:53 Groups ([FFAUsers]) include non-existent username (ffasupervisors)
2011.06.15 08:41:53 Groups ([FFAManagers, FFAUsers]) include non-existent username (ffadirectors)
2011.06.15 08:41:55 Groups ([FFFMUsers]) include non-existent username (3fmloanofficers)
2011.06.15 08:41:55 Groups ([FFAUsers]) include non-existent username (ffaunderwriting)
2011.06.15 08:41:55 Groups ([FFFMUsers]) include non-existent username (3fmloanofficers)
2011.06.15 08:41:55 Groups ([FFAUsers]) include non-existent username (ffaexecutiveassistants)
2011.06.15 08:41:55 Groups ([FFFMUsers]) include non-existent username (3fmloanofficers)
2011.06.15 08:41:55 Groups ([FFAUsers]) include non-existent username (ffaunderwriting)
2011.06.15 08:41:56 Groups ([FFFMUsers]) include non-existent username (3fmloanofficers)
2011.06.15 08:41:56 Groups ([FFAUsers]) include non-existent username (ffaexecutiveassistants)
2011.06.15 08:41:56 Groups ([FFAUsers]) include non-existent username (ffaunderwriting)
2011.06.15 08:41:56 Groups ([FFAUsers]) include non-existent username (ffaexecutiveassistants)
2011.06.15 08:41:56 Groups ([FFAUsers]) include non-existent username (ffaunderwriting)
2011.06.15 08:41:56 Groups ([FFAUsers]) include non-existent username (ffaclientrelations)
2011.06.15 08:41:56 Groups ([FFAUsers]) include non-existent username (ffaexecutiveassistants)
2011.06.15 08:41:56 Groups ([FFAUsers]) include non-existent username (ffaclientrelations)
2011.06.15 08:41:56 Groups ([FFAUsers]) include non-existent username (ffaclientrelations)
2011.06.15 08:41:57 Groups ([FFAUsers]) include non-existent username (ffaclientrelations)
2011.06.15 08:41:58 Groups ([FFAUsers]) include non-existent username (quickbooks)
2011.06.15 08:41:58 Groups ([FFAUsers]) include non-existent username (quickbooks)
2011.06.15 08:41:59 Groups ([FFAUsers]) include non-existent username (quickbooks)
2011.06.15 08:41:59 Groups ([FFAUsers]) include non-existent username (quickbooks)
2011.06.15 08:42:00 Groups ([FFAUsers]) include non-existent username (ffaexecs)
2011.06.15 08:42:00 Groups ([3FMUsers, 3FMManagers, FFFMUsers]) include non-existent username (fffmmanagers)
2011.06.15 08:42:00 Groups ([FFAUsers]) include non-existent username (ffanewaccounts)
2011.06.15 08:42:00 Groups ([3FMUsers]) include non-existent username (fffmusers)
2011.06.15 08:42:01 Groups ([FFFMManagers]) include non-existent username (3fmmanagers)
2011.06.15 08:42:01 Groups ([FFAUsers]) include non-existent username (ffaaccounting)
2011.06.15 08:42:01 Groups ([FFAUsers]) include non-existent username (dpausers)
2011.06.15 08:42:01 Groups ([FFAUsers]) include non-existent username (ffareception)
2011.06.15 08:42:01 Groups ([FFAUsers]) include non-existent username (ffamanagers)
2011.06.15 08:42:01 Groups ([FFAClientRelations]) include non-existent username (dpaclientrelations)
2011.06.15 08:42:01 Groups ([FFAUsers]) include non-existent username (it)
2011.06.15 08:42:01 Groups ([3FMUsers, FFFMProcessors]) include non-existent username (3fmprocessors)
2011.06.15 08:42:01 Groups ([FFAUsers]) include non-existent username (ithelpers)
2011.06.15 08:42:01 Groups ([3FMProcessors, FFFMUsers]) include non-existent username (fffmprocessors)
2011.06.15 08:42:02 Groups ([FFAUsers]) include non-existent username (ffasupervisors)
2011.06.15 08:42:02 Groups ([FFAManagers, FFAUsers]) include non-existent username (ffadirectors)
2011.06.15 08:42:02 Groups ([FFAUsers]) include non-existent username (ffasettlement)
2011.06.15 08:42:02 Groups ([FFFMUsers]) include non-existent username (3fmloanofficers)
2011.06.15 08:42:02 Groups ([FFAUsers]) include non-existent username (ffasettlement)
2011.06.15 08:42:02 Groups ([3FMUsers]) include non-existent username (3fmexecs)
2011.06.15 08:42:02 Groups ([FFAUsers]) include non-existent username (ffaunderwriting)
2011.06.15 08:42:02 Groups ([3FMUsers]) include non-existent username (3fmexecs)
2011.06.15 08:42:02 Groups ([FFAUsers]) include non-existent username (ffasettlement)
2011.06.15 08:42:02 Groups ([FFAUsers]) include non-existent username (ffaexecutiveassistants)
2011.06.15 08:42:02 Groups ([FFAUsers]) include non-existent username (ffasettlement)
2011.06.15 08:42:02 Groups ([3FMUsers]) include non-existent username (3fmexecs)
2011.06.15 08:42:02 Groups ([FFAUsers]) include non-existent username (ffaclientrelations)
2011.06.15 08:42:02 Groups ([FFFMUsers]) include non-existent username (3fmusers)
2011.06.15 08:42:02 Groups ([FFAUsers]) include non-existent username (quickbooks)
2011.06.15 08:42:02 Groups ([3FMUsers]) include non-existent username (3fmexecs)
2011.06.15 08:42:03 Groups ([FFFMUsers]) include non-existent username (3fmusers)
2011.06.15 08:42:03 Groups ([FFAUsers]) include non-existent username (ffasettlement)
2011.06.15 08:42:03 Groups ([3FMUsers]) include non-existent username (3fmexecs)
2011.06.15 08:42:03 Groups ([FFFMUsers]) include non-existent username (3fmusers)
2011.06.15 08:42:03 Groups ([FFFMUsers]) include non-existent username (3fmusers)
2011.06.15 08:42:03 Groups ([FFFMUsers]) include non-existent username (3fmusers)
2011.06.15 08:42:10 Internal server error
java.io.FileNotFoundException: \nas\shares\IT\Logs\OpenFire\Messages\jive.audit-20110615-000.log (Access is denied)
at java.io.FileOutputStream.openAppend(Native Method)
at java.io.FileOutputStream.(Unknown Source)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.createAuditFile(AuditorImpl.jav a:352)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.prepareAuditFile(AuditorImpl.ja va:217)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.saveQueuedPackets(AuditorImpl.j ava:380)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.access$200(AuditorImpl.java:59)
at org.jivesoftware.openfire.audit.spi.AuditorImpl$SaveQueuedPacketsTask.run(Audit orImpl.java:367)
at java.util.TimerThread.mainLoop(Unknown Source)
at java.util.TimerThread.run(Unknown Source)
2011.06.15 08:42:10 Internal server error
java.io.FileNotFoundException: \nas\shares\IT\Logs\OpenFire\Messages\jive.audit-20110615-000.log (Access is denied)
at java.io.FileOutputStream.openAppend(Native Method)
at java.io.FileOutputStream.(Unknown Source)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.createAuditFile(AuditorImpl.jav a:352)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.prepareAuditFile(AuditorImpl.ja va:217)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.saveQueuedPackets(AuditorImpl.j ava:380)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.access$200(AuditorImpl.java:59)
at org.jivesoftware.openfire.audit.spi.AuditorImpl$SaveQueuedPacketsTask.run(Audit orImpl.java:367)
at java.util.TimerThread.mainLoop(Unknown Source)
at java.util.TimerThread.run(Unknown Source)
2011.06.15 08:42:10 Internal server error
java.io.FileNotFoundException: \nas\shares\IT\Logs\OpenFire\Messages\jive.audit-20110615-000.log (Access is denied)
at java.io.FileOutputStream.openAppend(Native Method)
at java.io.FileOutputStream.(Unknown Source)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.createAuditFile(AuditorImpl.jav a:352)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.prepareAuditFile(AuditorImpl.ja va:217)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.saveQueuedPackets(AuditorImpl.j ava:380)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.access$200(AuditorImpl.java:59)
at org.jivesoftware.openfire.audit.spi.AuditorImpl$SaveQueuedPacketsTask.run(Audit orImpl.java:367)
at java.util.TimerThread.mainLoop(Unknown Source)
at java.util.TimerThread.run(Unknown Source)
2011.06.15 08:42:10 Internal server error
java.io.FileNotFoundException: \nas\shares\IT\Logs\OpenFire\Messages\jive.audit-20110615-000.log (Access is denied)
at java.io.FileOutputStream.openAppend(Native Method)
at java.io.FileOutputStream.(Unknown Source)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.createAuditFile(AuditorImpl.jav a:352)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.prepareAuditFile(AuditorImpl.ja va:217)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.saveQueuedPackets(AuditorImpl.j ava:380)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.access$200(AuditorImpl.java:59)
at org.jivesoftware.openfire.audit.spi.AuditorImpl$SaveQueuedPacketsTask.run(Audit orImpl.java:367)
at java.util.TimerThread.mainLoop(Unknown Source)
at java.util.TimerThread.run(Unknown Source)
2011.06.15 08:42:10 Internal server error
java.io.FileNotFoundException: \nas\shares\IT\Logs\OpenFire\Messages\jive.audit-20110615-000.log (Access is denied)
at java.io.FileOutputStream.openAppend(Native Method)
at java.io.FileOutputStream.(Unknown Source)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.createAuditFile(AuditorImpl.jav a:352)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.prepareAuditFile(AuditorImpl.ja va:217)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.saveQueuedPackets(AuditorImpl.j ava:380)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.access$200(AuditorImpl.java:59)
at org.jivesoftware.openfire.audit.spi.AuditorImpl$SaveQueuedPacketsTask.run(Audit orImpl.java:367)
at java.util.TimerThread.mainLoop(Unknown Source)
at java.util.TimerThread.run(Unknown Source)
2011.06.15 08:42:10 Internal server error
java.io.FileNotFoundException: \nas\shares\IT\Logs\OpenFire\Messages\jive.audit-20110615-000.log (Access is denied)
at java.io.FileOutputStream.openAppend(Native Method)
at java.io.FileOutputStream.(Unknown Source)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.createAuditFile(AuditorImpl.jav a:352)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.prepareAuditFile(AuditorImpl.ja va:217)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.saveQueuedPackets(AuditorImpl.j ava:380)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.access$200(AuditorImpl.java:59)
at org.jivesoftware.openfire.audit.spi.AuditorImpl$SaveQueuedPacketsTask.run(Audit orImpl.java:367)
at java.util.TimerThread.mainLoop(Unknown Source)
at java.util.TimerThread.run(Unknown Source)
2011.06.15 08:42:10 Internal server error
java.io.FileNotFoundException: \nas\shares\IT\Logs\OpenFire\Messages\jive.audit-20110615-000.log (Access is denied)
at java.io.FileOutputStream.openAppend(Native Method)
at java.io.FileOutputStream.(Unknown Source)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.createAuditFile(AuditorImpl.jav a:352)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.prepareAuditFile(AuditorImpl.ja va:217)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.saveQueuedPackets(AuditorImpl.j ava:380)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.access$200(AuditorImpl.java:59)
at org.jivesoftware.openfire.audit.spi.AuditorImpl$SaveQueuedPacketsTask.run(Audit orImpl.java:367)
at java.util.TimerThread.mainLoop(Unknown Source)
at java.util.TimerThread.run(Unknown Source)
2011.06.15 08:42:10 Internal server error
java.io.FileNotFoundException: \nas\shares\IT\Logs\OpenFire\Messages\jive.audit-20110615-000.log (Access is denied)
at java.io.FileOutputStream.openAppend(Native Method)
at java.io.FileOutputStream.(Unknown Source)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.createAuditFile(AuditorImpl.jav a:352)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.prepareAuditFile(AuditorImpl.ja va:217)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.saveQueuedPackets(AuditorImpl.j ava:380)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.access$200(AuditorImpl.java:59)
at org.jivesoftware.openfire.audit.spi.AuditorImpl$SaveQueuedPacketsTask.run(Audit orImpl.java:367)
at java.util.TimerThread.mainLoop(Unknown Source)
at java.util.TimerThread.run(Unknown Source)
2011.06.15 08:42:10 Internal server error
java.io.FileNotFoundException: \nas\shares\IT\Logs\OpenFire\Messages\jive.audit-20110615-000.log (Access is denied)
at java.io.FileOutputStream.openAppend(Native Method)
at java.io.FileOutputStream.(Unknown Source)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.createAuditFile(AuditorImpl.jav a:352)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.prepareAuditFile(AuditorImpl.ja va:217)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.saveQueuedPackets(AuditorImpl.j ava:380)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.access$200(AuditorImpl.java:59)
at org.jivesoftware.openfire.audit.spi.AuditorImpl$SaveQueuedPacketsTask.run(Audit orImpl.java:367)
at java.util.TimerThread.mainLoop(Unknown Source)
at java.util.TimerThread.run(Unknown Source)
2011.06.15 08:42:10 Internal server error
java.io.FileNotFoundException: \nas\shares\IT\Logs\OpenFire\Messages\jive.audit-20110615-000.log (Access is denied)
at java.io.FileOutputStream.openAppend(Native Method)
at java.io.FileOutputStream.(Unknown Source)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.createAuditFile(AuditorImpl.jav a:352)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.prepareAuditFile(AuditorImpl.ja va:217)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.saveQueuedPackets(AuditorImpl.j ava:380)
at org.jivesoftware.openfire.audit.spi.AuditorImpl.access$200(AuditorImpl.java:59)
at org.jivesoftware.openfire.audit.spi.AuditorImpl$SaveQueuedPacketsTask.run(Audit orImpl.java:367)
at java.util.TimerThread.mainLoop(Unknown Source)
at java.util.TimerThread.run(Unknown Source)
I also found this over and over again in the Warn Log:
2011.06.15 08:42:04 Cache Roster was full, shrinked to 90% in 0ms.
2011.06.15 08:42:04 Cache Roster was full, shrinked to 90% in 0ms.
2011.06.15 08:42:04 Cache Roster was full, shrinked to 90% in 0ms.
2011.06.15 08:42:04 Cache Roster was full, shrinked to 90% in 0ms.
2011.06.15 08:42:04 Cache Roster was full, shrinked to 90% in 0ms.
2011.06.15 08:42:29 Cache Roster was full, shrinked to 90% in 0ms.
2011.06.15 08:42:30 Cache Roster was full, shrinked to 90% in 0ms.
2011.06.15 08:42:42 Cache Roster was full, shrinked to 90% in 0ms.
2011.06.15 08:42:43 Cache Roster was full, shrinked to 90% in 0ms.
2011.06.15 08:42:48 Cache Roster was full, shrinked to 90% in 0ms.
2011.06.15 08:43:08 Cache Roster was full, shrinked to 90% in 0ms.
2011.06.15 08:43:09 Cache Roster was full, shrinked to 90% in 0ms.
2011.06.15 08:43:09 Cache Roster was full, shrinked to 90% in 0ms.
Though i’m not seeing any OutOfMemory errors, but it looks like Openfire is not getting enough memory. What does it show about Java Memory on the start page in Admin Console. Watch it for some time and hit refresh every 30 seconds. Usually it stays on ~80% ocassionally jumping to nearly 100. At least in my environment. Maybe it is a problem with Openfire’s internal java on x64 system. You can try deleting program files/openfire/jre folder and then installing the latest x64 java jre from Oracle http://www.oracle.com/technetwork/java/javase/downloads/jre-6u26-download-400751 .html. Openfire will switch to external java if it cant find the built-in one.
Actually, in my environment the Java Memory indicator hovers between 10-15% (30-40MB of 247.5MB) - I’ve never seen it go any higher, even when we are experiencing the issue.
I’ll try your suggestion of manually installing the latest JRE and let you know how that goes…
I also found these articles about manually configuring caches for LDAP, but neither includes any default or recommended values, so I just tried 10MB for each except the username2roster which I configured at 20MB. Unfortunately, this did-not seem to help. I’m going to go back and try some of the filtering mentioned in the first link as well since I am using the root of the domain as the Base DN. Unfortunately I don’t have any choice without restructuring my entire AD, which I would prefer not to do. I’ll let you know how that turns out as-well.
I tried renaming the program files (x86)/openfire/jre folder to ‘jre_bak’ and installing the latest JRE from the Oracle site (jre-6u26-windows-x64.exe), and now when I try to start the OpenFire service I get ‘Windows could not start the Openfire service on Local Computer. Error 1: Incorrect function.’. If I try to launch openfire.exe from the bin folder, i get ‘Error: No JVM could be found on your system. Please define EXE4J_JAVA_HOME to point to an installed JDK or JRE or download a JRE from www.java.com’. I’m not sure where to definte that value, so I tried just copying the jre6 folder from program files/java to program files (x86)/openfire and renaming to ‘jre’, but I still get the same error. How do I define the EXE4J_JAVA_HOME value, or do I just need to reinstall OpenFire after installing the latest x64 JRE?
OK, instead of messing with environmental variables I just uninstalled the x64 version and installed the latest 32-bit version (jre-6u26-windows-i586.exe), then I was able to start the OpenFire service without error. However, I’m still having the same results…
Next I’m going to try to implement some of the filtering suggested in the ‘OpenFire AD LDAP Integration’ article referenced above, although I will have to do it manually through the ‘System Properties’ page now by editing the ldap.searchFilter and ldap.groupSearchFilter options…wish me luck!
OK, I THINK I have finally resolved this issue - and I’m honestly not exactly sure what the actual issue was. It may have been a combination of issues, so I am going to list everything I did so that it may help someone else in the future.
Ultimately I think the issue was that I have to use the ROOT of my domain as the Base DN because of the current AD structure (I could rearrange AD so that all users are within a specific OU, but I’d rather not). Because of this, OpenFire was having to sort through literally thousands of objects including tons of disabled users. I may still go back and do some additional filtering to exclude computer objects since these are also not necessary, but it’s working for now as-is. I tried doing some group filtering, but I couldn’t manage to get it to do what I want because I use the AD groups to populate the roster groups. I’m sure with a bit more tweaking I could get it to filter out the unnecessary groups, but again for now it’s working so I’m going with what I’ve got.
I’m going to start with the most recent changes I made, because these are probably the ones that actually solved the problem:
The last thing I changed before it started working was I applied a filter to the ldap.searchFilter to exclude disabled users (since we have hundreds of them). To do this, I went to the System Properties screen, clicked Edit next to ldap.searchFilter and entered a value of:*** (&(objectClass=organizationalPerson)(!(userAccountControl:1.2.840.113556.1.4.80 3:=2)))***
Another thing I changed after applying the disabled users filter and before restarting the server was I disabled all unnecssary fields from being mapped from AD to OpenFire. There may be another way to do this, but I did it by setting the ‘setup’ value to ‘false’ in openfire.xml, then deleting the unnecessary values in {brackets} from the fields in the mapping section when configuring LDAP. For example, I did-not need physical addresses or phone numbers - only username, full name, and email address.
Before making the other two changes, I added the following cache settings to the system properties manually - but this did-not resolve the issue on it’s own after a restart. I came up with these values on my own, so you might want to tweak them a bit and find what works best for you. Also see the articles referenced above for more info about these settings:
cache.ldap.maxLifetime
300000
cache.ldap.size
3932160
cache.userCache.maxLifetime
300000
cache.userCache.size
3932160
cache.userGroup.maxLifetime
300000
cache.userGroup.size
3932160
cache.username2roster.maxLifetime
600000
cache.username2roster.size
3932160
cache.vcardCache.maxLifetime
300000
cache.vcardCache.size
3932160
Before that, I followed wroot’s suggestion of renaming/deleting the ‘jre’ folder within the OpenFire program folder, and installing the latest 32bit version from the Java website. Note it WILL NOT work with the x64 version, even on a 64bit machine - at least I could not figure out how.
So there you have it, I’ll update this thread if I run into any more issues - but as of now it seems to be working properly for all of my users and most importantly ME!
I’m glad you have found a solution. I don’t have much experience with LDAP integration, so i never hit this problem myself. So it wasn’t a memory issue, but the caches exhausted by the number of AD objects. I think now it should probably work with the built-in java also.