Openfire suddenly dies

Openfire Openfire Support
1

Can’t seem to find any previous posts about this,

the issue is that the servern unexpectadly dies about approximately every 22nd-28th hour…

No errors in logs, and when we restart the server it only logs “server started”-msgs as usal, nothin about dump, crash or similar from the breakage.

Anyone have any clues to what might cause this and some kind of troubleshooting or what to test?

Server is openfire 3.7.0 (from today 3.7.1) running on a windows 2008 r2 with 8GB ram, running as a service.

vmoptions are:

-Xms1024m

-Xmx1550m

which i can’t change to -Xmx2048m, then the openfire service refuses to start.

emdedded-db, openfire-script is abou ~180megs.

No errors when crashing in warn, error, info nor debug log.

~java mem according to site is fluxing from ~400mb to 800mb (1440 avail).

Anything else needed to give me a hint or direction?

2

Do you have xmpp.pep.enabled set to false in the system properties?

If you want to use a large JVM heap, you need to run a 64bit JRE.

3

xmpp.pep is disabled yes.

And im only in need of 2gb for java, so no need for 64bit just yet :slight_smile:

Server died today too, with only 1/6 userload… :confused:

I’ve also raised the rostercache to 20mb since it had a lot of “90percent full”, shrinked" messages.

Still no change in behavior. Dies every 20-28th hour or so… :confused:

4

So what’s the error in the log when it fails?

To be clear, you have the system property ‘xmpp.pep.enabled’ set to ‘false’? Anything else, and it will not do what you expect.

5

Yes, thats the variable i ha e in system properties, set to value false.

And no errors nor warnings in any log. Not even in the debug log.

Anything i could try to reproduce this? :slight_smile:

6

Just found some hs_err logs and this was the latest:

I can’t set

-Xms1024m -Xmx2048m due to failure at startup, but that should be the correct way to solve this out of mem error?

Seems like a swaperror to, but i guess that’s the java-heap-mem, not the actual swap mem on the windowsdisk since that’s about 8gig free…

A fatal error has been detected by the Java Runtime Environment:

java.lang.OutOfMemoryError: requested 2048000 bytes for GrET in C:\BUILD_AREA\jdk6_18\hotspot\src\share\vm\utilities\growableArray.cpp. Out of swap space?

Internal Error (allocation.inline.hpp:39), pid=4208, tid=3080

Error: GrET in C:\BUILD_AREA\jdk6_18\hotspot\src\share\vm\utilities\growableArray.cpp

JRE version: 6.0_18-b07

Java VM: Java HotSpot™ Client VM (16.0-b13 mixed mode windows-x86 )

If you would like to submit a bug report, please visit:

http://java.sun.com/webapps/bugreport/crash.jsp

--------------- T H R E A D ---------------

Current thread (0x01ed6800): VMThread [stack: 0x01f90000,0x01fe0000] [id=3080]

Stack: [0x01f90000,0x01fe0000], sp=0x01fdfa70, free space=13e01fdfa80k

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)

V [jvm.dll+0x1f4000]

V [jvm.dll+0xa1e0c]

V [jvm.dll+0xcfe77]

V [jvm.dll+0x8d67]

V [jvm.dll+0xde408]

V [jvm.dll+0x15df8a]

V [jvm.dll+0x1e1e14]

V [jvm.dll+0x1a1aad]

V [jvm.dll+0xc834f]

V [jvm.dll+0xca01c]

V [jvm.dll+0xca370]

V [jvm.dll+0xce42a]

V [jvm.dll+0x1d8592]

V [jvm.dll+0xc9398]

V [jvm.dll+0x7e7cb]

V [jvm.dll+0x1f442b]

V [jvm.dll+0x1f6ed3]

V [jvm.dll+0x1f615e]

V [jvm.dll+0x1f64ac]

V [jvm.dll+0x1f68d2]

V [jvm.dll+0x17f9cc]

C [MSVCR71.dll+0xb381]

C [kernel32.dll+0x1339a]

C [ntdll.dll+0x39ef2]

C [ntdll.dll+0x39ec5]

VM_Operation (0x6ec7e9ec): GenCollectForAllocation, mode: safepoint, requested by thread 0x6c097c00

--------------- P R O C E S S ---------------

Java Threads: ( => current thread )

0x6bfe1000 JavaThread “pool-monitoring152” daemon [_thread_blocked, id=4712, stack(0x6cad0000,0x6cb20000)]

0x6bfdfc00 JavaThread “pool-monitoring150” daemon [_thread_blocked, id=2664, stack(0x70850000,0x708a0000)]

0x6bfe1c00 JavaThread “pool-openfire14” daemon [_thread_blocked, id=2796, stack(0x73a40000,0x73a90000)]

0x6bfe1400 JavaThread “Thread-401” daemon [_thread_in_native, id=2244, stack(0x6ca40000,0x6ca90000)]

0x6bfdf800 JavaThread “pool-10-thread-5” [_thread_blocked, id=908, stack(0x71320000,0x71370000)]

0x6bfdf000 JavaThread “pool-10-thread-4” [_thread_blocked, id=1384, stack(0x701a0000,0x701f0000)]

0x6c0af400 JavaThread “pool-10-thread-3” [_thread_blocked, id=5012, stack(0x6d2a0000,0x6d2f0000)]

0x6c0af000 JavaThread “pool-10-thread-2” [_thread_blocked, id=4132, stack(0x71290000,0x712e0000)]

0x6bfde800 JavaThread “pool-10-thread-1” [_thread_blocked, id=3540, stack(0x702c0000,0x70310000)]

0x6bfde400 JavaThread “Queued Packets Processor” daemon [_thread_blocked, id=3492, stack(0x70230000,0x70280000)]

0x6c0ad800 JavaThread “client-17” daemon [_thread_blocked, id=3996, stack(0x6ef00000,0x6ef50000)]

0x6c0ad000 JavaThread “client-16” daemon [_thread_blocked, id=4892, stack(0x6ee70000,0x6eec0000)]

0x6c0acc00 JavaThread “client-15” daemon [_thread_blocked, id=1784, stack(0x6ede0000,0x6ee30000)]

0x6c0ac400 JavaThread “client-14” daemon [_thread_blocked, id=1624, stack(0x6ed50000,0x6eda0000)]

0x6c0ac000 JavaThread “client-13” daemon [_thread_blocked, id=668, stack(0x6ecc0000,0x6ed10000)]

0x6c097c00 JavaThread “client-12” daemon [_thread_blocked, id=416, stack(0x6ec30000,0x6ec80000)]

0x6c097800 JavaThread “client-11” daemon [_thread_blocked, id=4488, stack(0x6eba0000,0x6ebf0000)]

0x6c097000 JavaThread “client-10” daemon [_thread_blocked, id=3848, stack(0x6eb10000,0x6eb60000)]

0x6c096c00 JavaThread “HashSessionScavenger-1” daemon [_thread_blocked, id=2992, stack(0x6ea80000,0x6ead0000)]

0x6c096400 JavaThread “client-9” daemon [_thread_blocked, id=3776, stack(0x6e9f0000,0x6ea40000)]

0x6c095c00 JavaThread “client-8” daemon [_thread_blocked, id=3860, stack(0x6e960000,0x6e9b0000)]

0x6c095800 JavaThread “client-7” daemon [_thread_blocked, id=3428, stack(0x6e8d0000,0x6e920000)]

0x6c095000 JavaThread “client-6” daemon [_thread_blocked, id=3152, stack(0x6e840000,0x6e890000)]

0x6c094c00 JavaThread “client-5” daemon [_thread_blocked, id=4564, stack(0x6e7b0000,0x6e800000)]

0x6c094400 JavaThread “client-4” daemon [_thread_blocked, id=3388, stack(0x6e720000,0x6e770000)]

0x6c25ec00 JavaThread “client-3” daemon [_thread_blocked, id=1128, stack(0x6e690000,0x6e6e0000)]

0x6c25e400 JavaThread “client-2” daemon [_thread_blocked, id=3232, stack(0x6e600000,0x6e650000)]

0x6c25e000 JavaThread “client-1” daemon [_thread_blocked, id=4808, stack(0x6e570000,0x6e5c0000)]

0x6c25d800 JavaThread “SocketAcceptorIoProcessor-0.1” [_thread_blocked, id=4924, stack(0x6e4e0000,0x6e530000)]

0x6c25d400 JavaThread “SocketAcceptorIoProcessor-0.0” [_thread_in_native, id=3988, stack(0x6e450000,0x6e4a0000)]

0x6c25cc00 JavaThread “Jetty-QTP-BOSH-47” [_thread_blocked, id=2256, stack(0x6e3c0000,0x6e410000)]

0x6c25c800 JavaThread “Jetty-QTP-BOSH-46” [_thread_blocked, id=3304, stack(0x6e330000,0x6e380000)]

0x6c25c000 JavaThread “Jetty-QTP-BOSH-45” [_thread_blocked, id=3968, stack(0x6e2a0000,0x6e2f0000)]

0x6c25bc00 JavaThread “Jetty-QTP-BOSH-44” [_thread_blocked, id=900, stack(0x6d6d0000,0x6d720000)]

0x6c25b400 JavaThread “Jetty-QTP-BOSH-43” [_thread_blocked, id=2160, stack(0x6d640000,0x6d690000)]

0x6c11e400 JavaThread “Jetty-QTP-BOSH-42” [_thread_blocked, id=280, stack(0x6d550000,0x6d5a0000)]

0x6c11e000 JavaThread “Jetty-QTP-BOSH-41 - Acceptor0 HttpBindManager$JiveSslConnector@0.0.0.0:7443” [_thread_blocked, id=2388, stack(0x6d4c0000,0x6d510000)]

0x6c11d800 JavaThread “Jetty-QTP-BOSH-40 - Acceptor0 SelectChannelConnector@0.0.0.0:7070” [_thread_blocked, id=4968, stack(0x6d430000,0x6d480000)]

0x6c11d400 JavaThread “SocketAcceptor-1” [_thread_in_native, id=4352, stack(0x6d3a0000,0x6d3f0000)]

0x6c11cc00 JavaThread “SocketAcceptor-0” [_thread_in_native, id=4504, stack(0x6d350000,0x6d3a0000)]

0x6c11c000 JavaThread “timer-monitoring” daemon [_thread_blocked, id=4388, stack(0x6d1c0000,0x6d210000)]

0x6c11bc00 JavaThread “HashSessionScavenger-0” daemon [_thread_blocked, id=4632, stack(0x6d070000,0x6d0c0000)]

0x6c11b400 JavaThread “Jetty-QTP-AdminConsole-34” [_thread_blocked, id=3168, stack(0x6cfe0000,0x6d030000)]

0x6c11b000 JavaThread “Jetty-QTP-AdminConsole-33” [_thread_blocked, id=852, stack(0x6cf50000,0x6cfa0000)]

0x6c14a400 JavaThread “Jetty-QTP-AdminConsole-32” [_thread_blocked, id=4728, stack(0x6cec0000,0x6cf10000)]

0x6c14a000 JavaThread “Jetty-QTP-AdminConsole-31” [_thread_blocked, id=1372, stack(0x6ce30000,0x6ce80000)]

0x6c149800 JavaThread “Jetty-QTP-AdminConsole-30” [_thread_blocked, id=3640, stack(0x6cda0000,0x6cdf0000)]

0x6c149400 JavaThread “Jetty-QTP-AdminConsole-29 - Acceptor0 AdminConsolePlugin$JiveSslConnector@0.0.0.0:9091” [_thread_blocked, id=740, stack(0x6cd10000,0x6cd60000)]

0x6c148c00 JavaThread “Jetty-QTP-AdminConsole-28” [_thread_blocked, id=3928, stack(0x6cc80000,0x6ccd0000)]

0x6c148800 JavaThread “Jetty-QTP-AdminConsole-27 - Acceptor0 SelectChannelConnector@0.0.0.0:9090” [_thread_blocked, id=4840, stack(0x6bc20000,0x6bc70000)]

0x6c148000 JavaThread “pool-8-thread-1” [_thread_blocked, id=4252, stack(0x6cbf0000,0x6cc40000)]

0x6c147c00 JavaThread “SocketSendingTracker” daemon [_thread_blocked, id=468, stack(0x6cb60000,0x6cbb0000)]

0x6c146c00 JavaThread “MUC cleanup” [_thread_blocked, id=4400, stack(0x6c960000,0x6c9b0000)]

0x6c14c000 JavaThread “Flash Cross Domain” [_thread_in_native, id=340, stack(0x6c8d0000,0x6c920000)]

0x6bead800 JavaThread “Update Manager” daemon [_thread_blocked, id=3116, stack(0x6c840000,0x6c890000)]

0x6c172c00 JavaThread “Auditor” [_thread_blocked, id=1104, stack(0x6c740000,0x6c790000)]

0x6c172800 JavaThread “ClusterManager events dispatcher” daemon [_thread_blocked, id=3044, stack(0x6c6b0000,0x6c700000)]

0x6c01d800 JavaThread “timer-openfire” daemon [_thread_blocked, id=3136, stack(0x6c620000,0x6c670000)]

0x6c14d400 JavaThread “PubSub maintenance” [_thread_blocked, id=5052, stack(0x6c550000,0x6c5a0000)]

0x6bebdc00 JavaThread “HSQLDB Timer @14e3f41” daemon [_thread_blocked, id=3216, stack(0x6c430000,0x6c480000)]

0x6bebe800 JavaThread “Prototyper” daemon [_thread_blocked, id=2588, stack(0x6c340000,0x6c390000)]

0x6bebd800 JavaThread “HouseKeeper” daemon [_thread_blocked, id=4776, stack(0x6c2b0000,0x6c300000)]

0x01f4f400 JavaThread “Exe4JStartupThread” daemon [_thread_blocked, id=672, stack(0x6bb90000,0x6bbe0000)]

0x01f15400 JavaThread “Low Memory Detector” daemon [_thread_blocked, id=844, stack(0x6ba70000,0x6bac0000)]

0x01f0c800 JavaThread “CompilerThread0” daemon [_thread_blocked, id=4736, stack(0x6b9e0000,0x6ba30000)]

0x01f09800 JavaThread “Attach Listener” daemon [_thread_blocked, id=3084, stack(0x6b950000,0x6b9a0000)]

0x01ede800 JavaThread “Finalizer” daemon [_thread_blocked, id=3784, stack(0x6b8c0000,0x6b910000)]

0x01ed9c00 JavaThread “Reference Handler” daemon [_thread_blocked, id=1092, stack(0x6b830000,0x6b880000)]

0x02049800 JavaThread “main” [_thread_in_native, id=2620, stack(0x00370000,0x003c0000)]

Other Threads:

=>0x01ed6800 VMThread [stack: 0x01f90000,0x01fe0000] [id=3080]

0x01f20c00 WatcherThread [stack: 0x6bb00000,0x6bb50000] [id=1148]

VM state:at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event])

[0x02047bb8] Threads_lock - owner thread: 0x01ed6800

[0x02047fc8] Heap_lock - owner thread: 0x6c097c00

Heap

def new generation total 363648K, used 1288K [0x046e0000, 0x1d170000, 0x24b80000)

eden space 323264K, 0% used [0x046e0000, 0x046e0000, 0x18290000)

from space 40384K, 3% used [0x1aa00000, 0x1ab422c8, 0x1d170000)

to space 40384K, 0% used [0x18290000, 0x18290000, 0x1aa00000)

tenured generation total 812316K, used 812264K [0x24b80000, 0x564c7000, 0x654e0000)

the space 812316K, 99% used [0x24b80000, 0x564ba148, 0x564ba200, 0x564c7000)

compacting perm gen total 26368K, used 26310K [0x654e0000, 0x66ea0000, 0x694e0000)

the space 26368K, 99% used [0x654e0000, 0x66e91928, 0x66e91a00, 0x66ea0000)

No shared spaces configured.

Dynamic libraries:

0x00400000 - 0x0043b000 C:\Program Files (x86)\Openfire\bin\openfire-service.exe

0x77030000 - 0x771b0000 C:\Windows\SysWOW64\ntdll.dll

0x76660000 - 0x76770000 C:\Windows\syswow64\kernel32.dll

0x750c0000 - 0x75106000 C:\Windows\syswow64\KERNELBASE.dll

0x761d0000 - 0x76270000 C:\Windows\syswow64\ADVAPI32.DLL

0x76270000 - 0x7631c000 C:\Windows\syswow64\msvcrt.dll

0x74770000 - 0x74789000 C:\Windows\SysWOW64\sechost.dll

0x76560000 - 0x76650000 C:\Windows\syswow64\RPCRT4.dll

0x74710000 - 0x74770000 C:\Windows\syswow64\SspiCli.dll

0x74700000 - 0x7470c000 C:\Windows\syswow64\CRYPTBASE.dll

0x75240000 - 0x752d0000 C:\Windows\syswow64\GDI32.dll

0x74fb0000 - 0x750b0000 C:\Windows\syswow64\USER32.dll

0x76550000 - 0x7655a000 C:\Windows\syswow64\LPK.dll

0x75140000 - 0x751dd000 C:\Windows\syswow64\USP10.dll

0x753f0000 - 0x7603a000 C:\Windows\syswow64\SHELL32.DLL

0x74c30000 - 0x74c87000 C:\Windows\syswow64\SHLWAPI.dll

0x76160000 - 0x761c0000 C:\Windows\system32\IMM32.DLL

0x76480000 - 0x7654c000 C:\Windows\syswow64\MSCTF.dll

0x6d800000 - 0x6da97000 c:\program files (x86)\openfire\jre\bin\client\jvm.dll

0x74260000 - 0x74292000 C:\Windows\system32\WINMM.dll

0x7c360000 - 0x7c3b6000 C:\Windows\system32\MSVCR71.dll

0x74370000 - 0x743bc000 C:\Windows\system32\apphelp.dll

0x6d7b0000 - 0x6d7bc000 c:\program files (x86)\openfire\jre\bin\verify.dll

0x6d330000 - 0x6d34f000 c:\program files (x86)\openfire\jre\bin\java.dll

0x6d290000 - 0x6d298000 c:\program files (x86)\openfire\jre\bin\hpi.dll

0x761c0000 - 0x761c5000 C:\Windows\syswow64\PSAPI.DLL

0x6d7f0000 - 0x6d7ff000 c:\program files (x86)\openfire\jre\bin\zip.dll

0x74c90000 - 0x74dec000 C:\Windows\syswow64\ole32.dll

0x6d610000 - 0x6d623000 C:\Program Files (x86)\Openfire\jre\bin\net.dll

0x74790000 - 0x747c5000 C:\Windows\syswow64\WS2_32.dll

0x77000000 - 0x77006000 C:\Windows\syswow64\NSI.dll

0x73f80000 - 0x73fbc000 C:\Windows\system32\mswsock.dll

0x73f60000 - 0x73f66000 C:\Windows\System32\wship6.dll

0x6d630000 - 0x6d639000 C:\Program Files (x86)\Openfire\jre\bin\nio.dll

0x74490000 - 0x744a0000 C:\Windows\system32\NLAapi.dll

0x74480000 - 0x74490000 C:\Windows\system32\napinsp.dll

0x74150000 - 0x74194000 C:\Windows\system32\DNSAPI.dll

0x73b60000 - 0x73b68000 C:\Windows\System32\winrnr.dll

0x73f50000 - 0x73f55000 C:\Windows\System32\wshtcpip.dll

0x73b40000 - 0x73b5c000 C:\Windows\system32\IPHLPAPI.DLL

0x73b30000 - 0x73b37000 C:\Windows\system32\WINNSI.DLL

0x73b20000 - 0x73b26000 C:\Windows\system32\rasadhlp.dll

0x733a0000 - 0x733d8000 C:\Windows\System32\fwpuclnt.dll

0x74120000 - 0x74136000 C:\Windows\system32\CRYPTSP.dll

0x73df0000 - 0x73e2b000 C:\Windows\system32\rsaenh.dll

0x74250000 - 0x7425b000 C:\Windows\system32\profapi.dll

0x6d790000 - 0x6d798000 C:\Program Files (x86)\Openfire\jre\bin\sunmscapi.dll

0x74ae0000 - 0x74bfd000 C:\Windows\syswow64\CRYPT32.dll

0x750b0000 - 0x750bc000 C:\Windows\syswow64\MSASN1.dll

VM Arguments:

jvm_args: -Dexe4j.isInstall4j=true -Dexe4j.isService=true -Dexe4j.moduleName=C:\Program Files (x86)\Openfire\bin\openfire-service.exe -Dexe4j.processCommFile=C:\Windows\TEMP\e4j_p4208.tmp -Dexe4j.tempDir= -Dexe4j.unextractedPosition=0 -Dexe4j.consoleCodepage=cp0 -DopenfireHome=C:\Program Files (x86)\Openfire\bin…/ -Xrs -Xms1024m -Xmx1550m

java_command:

Launcher Type: generic

Environment Variables:

PATH=C:\Program Files (x86)\Pervasive Software\PSQL\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\W indows\System32\WindowsPowerShell\v1.0;C:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn\VSShell\Common7\IDE;C:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn;C:\Program Files\Microsoft SQL Server\100\Tools\Binn;C:\Program Files (x86)\Microsoft SQL Server\100\DTS\Binn;c:\Program Files\Microsoft SQL Server\100\DTS\Binn;;c:\program files (x86)\openfire\jre\bin

USERNAME=IAGHDB02$

OS=Windows_NT

PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 44 Stepping 2, GenuineIntel

--------------- S Y S T E M ---------------

OS: Windows Server 2008 R2 Build 7601 Service Pack 1

CPU:total 2 (1 cores per cpu, 1 threads per core) family 6 model 44 stepping 2, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt

Memory: 4k page, physical 8253044k(5516128k free), swap 16504236k(13709872k free)

vm_info: Java HotSpot™ Client VM (16.0-b13) for windows-x86 JRE (1.6.0_18-b07), built on Dec 17 2009 13:35:55 by “java_re” with MS VC++ 7.1 (VS2003)

time: Sat May 05 14:30:12 2012

elapsed time: 103623 seconds