Crashes with Wildfire 2.6.2 + NativeAuth on RHEL4

Openfire Openfire Support
1

Hello everyone,

I’‘m hoping someone can help me out with a recurring problem I’'ve been experiencing with Wildfire randomly crashing. Sometimes, the server will stay up for a day, and sometimes, it will stay up for 2 days. Any insight you can provide is greatly appreciated.

Here is the Java dump:

  1. An unexpected error has been detected by HotSpot Virtual Machine:

  1. SIGSEGV (0xb) at pc=0x0036ecca, pid=24753, tid=2280893360

  1. Java VM: Java HotSpot™ Server VM (1.5.0_06-b05 mixed mode)

  2. Problematic frame:

  3. C getenv+0xaa

T H R E A D -

Current thread (0x082b2980): JavaThread “Client SR - 9424699” daemon

siginfo:si_signo=11, si_errno=0, si_code=1, si_addr=0x895f86f8

Registers:

EAX=0xbffffe21, EBX=0x0046aff4, ECX=0x0000903b, EDX=0x00000054

ESP=0x87f39538, EBP=0x87f3955c, ESI=0x895f86f8, EDI=0x8c0b9e48

EIP=0x0036ecca, CR2=0x895f86f8, EFLAGS=0x00010282

Top of Stack: (sp=0x87f39538)

0x87f39538: 00460392 87f39578 00000001 00460394

0x87f39548: 00005a54 00000000 0046aff4 00000001

0x87f39558: 00000001 87f395cc 003c0d29 00460392

0x87f39568: 082b2980 af299e88 00000013 b2b817f8

0x87f39578: af299ddc af299e5c 00000013 8eaac35d

0x87f39588: 87f395fc 00000013 af299e38 af299dd0

0x87f39598: af299e50 b2c43124 af299dd0 b2b57b91

0x87f395a8: af299e50 00000013 87f395fc 87f39604

Instructions: (pc=0x0036ecca)

0x0036ecba: c0 74 9d 89 c6 eb 09 83 c7 04 8b 37 85 f6 74 90

0x0036ecca: 0f b7 0e 66 39 4d ec 75 ee 8b 4d f0 8d 46 02 8b

Stack: [0x87eba000,0x87f3b000), sp=0x87f39538, free space=509k

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)

C getenv+0xaa

C

C tzset+0x3e

C __strftime_l+0x3e

C vsyslog+0xfb

C syslog+0x1f

C RSA_log+0x1a

C pam_sm_authenticate+0x152

C pamdispatch+0x37a

C pam_authenticate+0x47

C

C shaj_chkpasswd_pam+0x8c

C Java_com_cenqua_shaj_PAMAuthenticator_checkPasswordImpl+0x87

j com.cenqua.shaj.PAMAuthenticator.checkPasswordImpl(Ljava/lang/String;Ljava/lang /String;Ljava/lang/String;Lcom/cenqua/shaj/log/Log;)Z+0

j com.cenqua.shaj.PAMAuthenticator.checkPAMPassword(Ljava/lang/String;Ljava/lang/ String;Ljava/lang/String;Lcom/cenqua/shaj/log/Log;)Z+55

j com.cenqua.shaj.PAMAuthenticator.checkPassword(Ljava/lang/String;Ljava/lang/Str ing;Ljava/lang/String;Lcom/cenqua/shaj/log/Log;)Z+5

j com.cenqua.shaj.Shaj.checkPassword(Ljava/lang/String;Ljava/lang/String;Ljava/la ng/String;)Z+9

j org.jivesoftware.wildfire.auth.NativeAuthProvider.authenticate(Ljava/lang/Strin g;Ljava/lang/String;)V+10

j org.jivesoftware.wildfire.auth.AuthFactory.authenticate(Ljava/lang/String;Ljava /lang/String;)Lorg/jivesoftware/wildfire/auth/AuthToken;+5

j org.jivesoftware.wildfire.net.SASLAuthentication.doPlainAuthentication(Lorg/dom 4j/Element;)Z+143

j org.jivesoftware.wildfire.net.SASLAuthentication.doHandshake(Lorg/dom4j/Element ;)Z+94

j org.jivesoftware.wildfire.net.SocketReader.authenticateClient(Lorg/dom4j/Elemen t;)Z+47

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)

j com.cenqua.shaj.PAMAuthenticator.checkPasswordImpl(Ljava/lang/String;Ljava/lang /String;Ljava/lang/String;Lcom/cenqua/shaj/log/Log;)Z+0

j com.cenqua.shaj.PAMAuthenticator.checkPAMPassword(Ljava/lang/String;Ljava/lang/ String;Ljava/lang/String;Lcom/cenqua/shaj/log/Log;)Z+55

j com.cenqua.shaj.PAMAuthenticator.checkPassword(Ljava/lang/String;Ljava/lang/Str ing;Ljava/lang/String;Lcom/cenqua/shaj/log/Log;)Z+5

j com.cenqua.shaj.Shaj.checkPassword(Ljava/lang/String;Ljava/lang/String;Ljava/la ng/String;)Z+9

j org.jivesoftware.wildfire.auth.NativeAuthProvider.authenticate(Ljava/lang/Strin g;Ljava/lang/String;)V+10

j org.jivesoftware.wildfire.auth.AuthFactory.authenticate(Ljava/lang/String;Ljava /lang/String;)Lorg/jivesoftware/wildfire/auth/AuthToken;+5

j org.jivesoftware.wildfire.net.SASLAuthentication.doPlainAuthentication(Lorg/dom 4j/Element;)Z+143

j org.jivesoftware.wildfire.net.SASLAuthentication.doHandshake(Lorg/dom4j/Element ;)Z+94

j org.jivesoftware.wildfire.net.SocketReader.authenticateClient(Lorg/dom4j/Elemen t;)Z+47

v ~C2IAdapter

J org.jivesoftware.wildfire.net.SocketReader.readStream()V

J org.jivesoftware.wildfire.net.SocketReader.run()V

v ~I2CAdapter

j java.lang.Thread.run()V+11

v ~StubRoutines::call_stub

P R O C E S S -

Java Threads: ( => current thread )

=>0x082b2980 JavaThread “Client SR - 9424699” daemon

0x08091a68 JavaThread “Client SR - 12655631” daemon

0x083d3ef8 JavaThread “Client SR - 11110649” daemon

0x080914b0 JavaThread “Client SR - 11991339” daemon

0x081f7d88 JavaThread “Client SR - 25538699” daemon

0x085dbbe0 JavaThread “Client SR - 7345254” daemon

0x08092ef8 JavaThread “Client SR - 18084235” daemon

0x082b6310 JavaThread “Client SR - 28579615” daemon

0x083e5c28 JavaThread “Client SR - 25697662” daemon

0x083ce450 JavaThread “Client SR - 30867975” daemon

0x083cf0b8 JavaThread “Client SR - 11529315” daemon

0x083cd3c8 JavaThread “Client SR - 29998362” daemon

0x083cc778 JavaThread “Client SR - 27860716” daemon

0x080ab530 JavaThread “Client SR - 12658274” daemon

0x080a6740 JavaThread “Client SR - 1030388” daemon

0x083fad70 JavaThread “Client SR - 7144790” daemon

0x081f9520 JavaThread “Client SR - 2479603” daemon

0x081aea80 JavaThread “Client SR - 24897496” daemon

0x0818ac70 JavaThread “Client SR - 29464963” daemon

0x084e5880 JavaThread “Client SR - 9748804” daemon

0x080d8060 JavaThread “Client SR - 25032385” daemon

0x082afd58 JavaThread “Client SR - 10776625” daemon

0x082ad350 JavaThread “Client SR - 30176354” daemon

0x082ac810 JavaThread “Client SR - 3799741” daemon

0x080947b8 JavaThread “Client SR - 25893557” daemon

0x08076638 JavaThread “Client SR - 281281” daemon

0x08071310 JavaThread “Client SR - 4458269” daemon

0x083c7830 JavaThread “Client SR - 13922953” daemon

0x083c2da0 JavaThread “Client SR - 21451030” daemon

0x08243c48 JavaThread “Client SR - 21251509” daemon

0x081aa9e0 JavaThread “Client SR - 25807119” daemon

0x08240638 JavaThread “Client SR - 28993767” daemon

0x082413c8 JavaThread “Client SR - 26414683” daemon

0x085056f0 JavaThread “Client SR - 31154292” daemon

0x08210d28 JavaThread “Client SR - 24320012” daemon

0x08211fc8 JavaThread “Client SR - 25316364” daemon

0x08213070 JavaThread “Client SR - 7992823” daemon

0x083ca560 JavaThread “Client SR - 20754339” daemon

0x8ca69058 JavaThread “Client SR - 22224012” daemon

0x083c9ae8 JavaThread “Client SR - 18710709” daemon

0x08189700 JavaThread “Client SR - 22167008” daemon

0x083ea410 JavaThread “Client SR - 23612281” daemon

0x083ea870 JavaThread “Client SR - 10668819” daemon

0x083b1f90 JavaThread “Client SR - 31105366” daemon

0x080a1bf8 JavaThread “Client SR - 14533924” daemon

0x080802f8 JavaThread “Client SR - 30671204” daemon

0x081f8468 JavaThread “pool-3-thread-5”

0x0850bb28 JavaThread “pool-3-thread-4”

0x08188f60 JavaThread “pool-3-thread-3”

0x0834a1c8 JavaThread “pool-3-thread-2”

0x08189948 JavaThread “Client SR - 3162309” daemon

0x08789170 JavaThread “pool-3-thread-1”

0x8ccc5a08 JavaThread “Queued Packets Processor” daemon

0x080db2d0 JavaThread “Client SR - 12133422” daemon

0x0850fd50 JavaThread “SunJsseListener1-1”

0x0840f480 JavaThread “SunJsseListener1-0”

0x08461780 JavaThread “Acceptor [SSL: ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=9091]]”

0x08450448 JavaThread “SocketListener0-1”

0x0844e610 JavaThread “SocketListener0-0”

0x082fd178 JavaThread “Acceptor ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=9090]”

0x08494e10 JavaThread “SessionScavenger” daemon

0x0805d2f0 JavaThread “Rollover” daemon

0x8d4579d0 JavaThread “DestroyJavaVM”

0x8dcd1728 JavaThread “pool-2-thread-1”

0x8d458048 JavaThread “SocketSendingTracker” daemon

0x8d457ba0 JavaThread “Secure Socket Listener” daemon

0x8cca9838 JavaThread “Socket Listener at port 5222” daemon

0x8cc5f330 JavaThread “Socket Listener at port 5269” daemon

0x8cc95960 JavaThread “pool-1-thread-1”

0x8cc95dc0 JavaThread “PubSub maintenance”

0x8ccb2518 JavaThread “Auditor”

0x8ccf6f60 JavaThread “MUC cleanup”

0x8cc96150 JavaThread “Sessions cleanup”

0x8cc38c80 JavaThread “Thread-2” daemon

0x8e5cb028 JavaThread “Thread-0” daemon

0x8e5a59d0 JavaThread “Low Memory Detector” daemon

0x8e5a45a0 JavaThread “CompilerThread1” daemon

0x8e5a35d0 JavaThread “CompilerThread0” daemon

0x8e5a25d0 JavaThread “AdapterThread” daemon

0x8e5a1700 JavaThread “Signal Dispatcher” daemon

0x8e598828 JavaThread “Finalizer” daemon

0x8e5982a0 JavaThread “Reference Handler” daemon

Other Threads:

0x8e594770 VMThread

0x8e5a6f60 WatcherThread

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap

PSYoungGen total 11008K, used 2086K [0xaf180000, 0xafde0000, 0xb2a60000)

eden space 9472K, 11% used [0xaf180000,0xaf29ab60,0xafac0000)

from space 1536K, 62% used [0xafac0000,0xafbaf090,0xafc40000)

to space 1600K, 0% used [0xafc50000,0xafc50000,0xafde0000)

PSOldGen total 170496K, used 102026K [0x92a60000, 0x9d0e0000, 0xaf180000)

object space 170496K, 59% used [0x92a60000,0x98e02868,0x9d0e0000)

PSPermGen total 32512K, used 16242K [0x8ea60000, 0x90a20000, 0x92a60000)

object space 32512K, 49% used [0x8ea60000,0x8fa3ca18,0x90a20000)

Dynamic libraries:

0032c000-00341000 r-xp 00000000 08:01 77631 /lib/ld-2.3.4.so

00341000-00342000 r-xp 00015000 08:01 77631 /lib/ld-2.3.4.so

00342000-00343000 rwxp 00016000 08:01 77631 /lib/ld-2.3.4.so

00345000-00469000 r-xp 00000000 08:01 77647 /lib/tls/libc-2.3.4.so

00469000-0046a000 r-xp 00124000 08:01 77647 /lib/tls/libc-2.3.4.so

0046a000-0046d000 rwxp 00125000 08:01 77647 /lib/tls/libc-2.3.4.so

0046d000-0046f000 rwxp 0046d000 00:00 0

00471000-00473000 r-xp 00000000 08:01 77676 /lib/libdl-2.3.4.so

00473000-00475000 rwxp 00001000 08:01 77676 /lib/libdl-2.3.4.so

00477000-00480000 r-xp 00000000 08:01 77689 /lib/libnss_files-2.3.4.so

00480000-00482000 rwxp 00008000 08:01 77689 /lib/libnss_files-2.3.4.so

004b9000-004da000 r-xp 00000000 08:01 77683 /lib/tls/libm-2.3.4.so

004da000-004dc000 rwxp 00020000 08:01 77683 /lib/tls/libm-2.3.4.so

004de000-004f0000 r-xp 00000000 08:01 77633 /lib/libnsl-2.3.4.so

004f0000-004f2000 rwxp 00011000 08:01 77633 /lib/libnsl-2.3.4.so

004f2000-004f4000 rwxp 004f2000 00:00 0

004f6000-00504000 r-xp 00000000 08:01 77612 /lib/tls/libpthread-2.3.4.so

00504000-00506000 rwxp 0000d000 08:01 77612 /lib/tls/libpthread-2.3.4.so

00506000-00508000 rwxp 00506000 00:00 0

0050a000-00519000 r-xp 00000000 08:01 77675 /lib/libresolv-2.3.4.so

00519000-0051b000 rwxp 0000f000 08:01 77675 /lib/libresolv-2.3.4.so

0051b000-0051d000 rwxp 0051b000 00:00 0

08048000-08057000 r-xp 00000000 08:01 86073 /opt/wildfire/jre/bin/java

08057000-08059000 rwxp 0000e000 08:01 86073 /opt/wildfire/jre/bin/java

08059000-087b0000 rwxp 08059000 00:00 0

87300000-87384000 rwxp 87300000 00:00 0

87384000-87400000 —p 87384000 00:00 0

875a3000-875a4000 rwxp 875a3000 00:00 0

875a4000-875a7000 rwxp 875a4000 00:00 0

875a7000-87626000 rwxp 875a7000 00:00 0

87626000-87629000 —p 87626000 00:00 0

87629000-876a7000 rwxp 87629000 00:00 0

876a7000-876aa000 —p 876a7000 00:00 0

876aa000-87728000 rwxp 876aa000 00:00 0

87728000-8772b000 —p 87728000 00:00 0

8772b000-877a9000 rwxp 8772b000 00:00 0

877a9000-877ac000 —p 877a9000 00:00 0

877ac000-8782b000 rwxp 877ac000 00:00 0

8782b000-8782e000 rwxp 8782b000 00:00 0

8782e000-878ac000 rwxp 8782e000 00:00 0

878ac000-878af000 rwxp 878ac000 00:00 0

878af000-8792d000 rwxp 878af000 00:00 0

8792d000-87930000 —p 8792d000 00:00 0

87930000-879ae000 rwxp 87930000 00:00 0

879ae000-879b1000 —p 879ae000 00:00 0

879b1000-87a2f000 rwxp 879b1000 00:00 0

87a2f000-87a32000 —p 87a2f000 00:00 0

87a32000-87ab0000 rwxp 87a32000 00:00 0

87ab0000-87ab3000 rwxp 87ab0000 00:00 0

87ab3000-87b31000 rwxp 87ab3000 00:00 0

87b31000-87b34000 rwxp 87b31000 00:00 0

87b34000-87bb3000 rwxp 87b34000 00:00 0

87bb3000-87bb6000 —p 87bb3000 00:00 0

87bb6000-87c34000 rwxp 87bb6000 00:00 0

87c34000-87c37000 rwxp 87c34000 00:00 0

87c37000-87cb5000 rwxp 87c37000 00:00 0

87cb5000-87cb8000 —p 87cb5000 00:00 0

87cb8000-87d36000 rwxp 87cb8000 00:00 0

87d36000-87d39000 —p 87d36000 00:00 0

87d39000-87db7000 rwxp 87d39000 00:00 0

87db7000-87dba000 rwxp 87db7000 00:00 0

87dba000-87e38000 rwxp 87dba000 00:00 0

87e38000-87e3b000 rwxp 87e38000 00:00 0

87e3b000-87eba000 rwxp 87e3b000 00:00 0

87eba000-87ebd000 —p 87eba000 00:00 0

87ebd000-87f3b000 rwxp 87ebd000 00:00 0

87f3b000-87f3e000 —p 87f3b000 00:00 0

87f3e000-87fbc000 rwxp 87f3e000 00:00 0

87fbc000-87fbf000 —p 87fbc000 00:00 0

87fbf000-8803d000 rwxp 87fbf000 00:00 0

8803d000-88040000 —p 8803d000 00:00 0

88040000-880be000 rwxp 88040000 00:00 0

880be000-880c1000 rwxp 880be000 00:00 0

880c1000-8813f000 rwxp 880c1000 00:00 0

8813f000-88142000 rwxp 8813f000 00:00 0

88142000-881c0000 rwxp 88142000 00:00 0

881c0000-881c3000 rwxp 881c0000 00:00 0

881c3000-88242000 rwxp 881c3000 00:00 0

88242000-88245000 rwxp 88242000 00:00 0

88245000-882c3000 rwxp 88245000 00:00 0

882c3000-882c6000 rwxp 882c3000 00:00 0

882c6000-88344000 rwxp 882c6000 00:00 0

88344000-88347000 —p 88344000 00:00 0

88347000-883c5000 rwxp 88347000 00:00 0

883c5000-883c8000 rwxp 883c5000 00:00 0

883c8000-88446000 rwxp 883c8000 00:00 0

88446000-88449000 rwxp 88446000 00:00 0

88449000-884c8000 rwxp 88449000 00:00 0

884c8000-884cb000 —p 884c8000 00:00 0

884cb000-88549000 rwxp 884cb000 00:00 0

88549000-8854c000 rwxp 88549000 00:00 0

8854c000-885ca000 rwxp 8854c000 00:00 0

885ca000-885cd000 —p 885ca000 00:00 0

885cd000-8864b000 rwxp 885cd000 00:00 0

8864b000-8864e000 —p 8864b000 00:00 0

8864e000-886cc000 rwxp 8864e000 00:00 0

886cc000-886cf000 —p 886cc000 00:00 0

886cf000-8874d000 rwxp 886cf000 00:00 0

8874d000-88750000 —p 8874d000 00:00 0

88750000-887ce000 rwxp 88750000 00:00 0

887ce000-887d1000 rwxp 887ce000 00:00 0

887d1000-88851000 rwxp 887d1000 00:00 0

88851000-88854000 rwxp 88851000 00:00 0

88854000-888d2000 rwxp 88854000 00:00 0

888d2000-888d5000 —p 888d2000 00:00 0

888d5000-88955000 rwxp 888d5000 00:00 0

88955000-88958000 —p 88955000 00:00 0

88958000-889d6000 rwxp 88958000 00:00 0

889d6000-889d9000 —p 889d6000 00:00 0

889d9000-88a59000 rwxp 889d9000 00:00 0

88a59000-88a5c000 rwxp 88a59000 00:00 0

88a5c000-88ada000 rwxp 88a5c000 00:00 0

88ada000-88add000 rwxp 88ada000 00:00 0

88add000-88b5b000 rwxp 88add000 00:00 0

88b5b000-88b5e000 —p 88b5b000 00:00 0

88b5e000-88bdc000 rwxp 88b5e000 00:00 0

88bdc000-88bdf000 —p 88bdc000 00:00 0

88bdf000-88c5e000 rwxp 88bdf000 00:00 0

88c5e000-88c61000 rwxp 88c5e000 00:00 0

88c61000-88cdf000 rwxp 88c61000 00:00 0

88cdf000-88ce2000 rwxp 88cdf000 00:00 0

88ce2000-88d61000 rwxp 88ce2000 00:00 0

88d61000-88d64000 —p 88d61000 00:00 0

88d64000-88de2000 rwxp 88d64000 00:00 0

88de2000-88de5000 —p 88de2000 00:00 0

88de5000-88e65000 rwxp 88de5000 00:00 0

88e65000-88e68000 —p 88e65000 00:00 0

88e68000-88ee7000 rwxp 88e68000 00:00 0

88ee7000-88eea000 —p 88ee7000 00:00 0

88eea000-88f68000 rwxp 88eea000 00:00 0

88f68000-88f6b000 rwxp 88f68000 00:00 0

88f6b000-88fe9000 rwxp 88f6b000 00:00 0

88fe9000-88fec000 rwxp 88fe9000 00:00 0

88fec000-8906b000 rwxp 88fec000 00:00 0

8906b000-8906e000 rwxp 8906b000 00:00 0

8906e000-890ec000 rwxp 8906e000 00:00 0

890ec000-890ef000 —p 890ec000 00:00 0

890ef000-8916e000 rwxp 890ef000 00:00 0

8916e000-89171000 rwxp 8916e000 00:00 0

89171000-891f1000 rwxp 89171000 00:00 0

891f1000-891f4000 —p 891f1000 00:00 0

891f4000-89272000 rwxp 891f4000 00:00 0

89272000-89275000 rwxp 89272000 00:00 0

89275000-892f3000 rwxp 89275000 00:00 0

892f3000-892f6000 —p 892f3000 00:00 0

892f6000-89374000 rwxp 892f6000 00:00 0

89374000-89377000 —p 89374000 00:00 0

89377000-893f5000 rwxp 89377000 00:00 0

893f5000-893f8000 —p 893f5000 00:00 0

893f8000-89476000 rwxp 893f8000 00:00 0

89476000-89479000 —p 89476000 00:00 0

89479000-894f7000 rwxp 89479000 00:00 0

894f7000-894fa000 —p 894f7000 00:00 0

894fa000-89579000 rwxp 894fa000 00:00 0

895fa000-895fd000 —p 895fa000 00:00 0

895fd000-8967b000 rwxp 895fd000 00:00 0

8967b000-8967e000 —p 8967b000 00:00 0

8967e000-896fc000 rwxp 8967e000 00:00 0

896fc000-896ff000 —p 896fc000 00:00 0

896ff000-8977d000 rwxp 896ff000 00:00 0

8977d000-89780000 rwxp 8977d000 00:00 0

89780000-897fe000 rwxp 89780000 00:00 0

897fe000-89801000 rwxp 897fe000 00:00 0

89801000-8987f000 rwxp 89801000 00:00 0

8987f000-89882000 —p 8987f000 00:00 0

89882000-89a00000 rwxp 89882000 00:00 0

89a78000-89a79000 rwxp 89a78000 00:00 0

89a79000-89a7c000 rwxp 89a79000 00:00 0

89a7c000-89afb000 rwxp 89a7c000 00:00 0

89afb000-89afe000 rwxp 89afb000 00:00 0

89afe000-89b7c000 rwxp 89afe000 00:00 0

89b7c000-89b7f000 —p 89b7c000 00:00 0

89b7f000-89bfe000 rwxp 89b7f000 00:00 0

89bfe000-89c01000 rwxp 89bfe000 00:00 0

89c01000-89c7f000 rwxp 89c01000 00:00 0

89c7f000-89c82000 —p 89c7f000 00:00 0

89c82000-89e00000 rwxp 89c82000 00:00 0

89e64000-89e67000 rwxp 89e64000 00:00 0

89e67000-89ee6000 rwxp 89e67000 00:00 0

89ee6000-89ee9000 rwxp 89ee6000 00:00 0

89ee9000-89f68000 rwxp 89ee9000 00:00 0

89f68000-89f6b000 rwxp 89f68000 00:00 0

89f6b000-89fe9000 rwxp 89f6b000 00:00 0

89fe9000-89fec000 rwxp 89fe9000 00:00 0

89fec000-8a06b000 rwxp 89fec000 00:00 0

8a06b000-8a06e000 —p 8a06b000 00:00 0

8a06e000-8a0ec000 rwxp 8a06e000 00:00 0

8a0ec000-8a0ef000 rwxp 8a0ec000 00:00 0

8a0ef000-8a16d000 rwxp 8a0ef000 00:00 0

8a16d000-8a170000 —p 8a16d000 00:00 0

8a170000-8a1ee000 rwxp 8a170000 00:00 0

8a1ee000-8a1f1000 —p 8a1ee000 00:00 0

8a1f1000-8a26f000 rwxp 8a1f1000 00:00 0

8a26f000-8a272000 —p 8a26f000 00:00 0

8a272000-8a2f0000 rwxp 8a272000 00:00 0

8a2f0000-8a2f3000 —p 8a2f0000 00:00 0

8a2f3000-8a371000 rwxp 8a2f3000 00:00 0

8a373000-8a376000 —p 8a373000 00:00 0

8a376000-8a3f4000 rwxp 8a376000 00:00 0

8a45f000-8a462000 rwxp 8a45f000 00:00 0

8a462000-8a465000 —p 8a462000 00:00 0

8a465000-8a4e4000 rwxp 8a465000 00:00 0

8a4e4000-8a4e7000 —p 8a4e4000 00:00 0

8a4e7000-8a565000 rwxp 8a4e7000 00:00 0

8a565000-8a568000 rwxp 8a565000 00:00 0

8a568000-8a5e6000 rwxp 8a568000 00:00 0

8a5e6000-8a5ed000 r-xp 00000000 08:01 77688 /lib/libgcc_s-3.4.5-20051201.so.1

8a5ed000-8a5ee000 rwxp 00006000 08:01 77688 /lib/libgcc_s-3.4.5-20051201.so.1

8a5f4000-8a5f5000 —p 8a5f4000 00:00 0

8a5f5000-8aff5000 rwxp 8a5f5000 00:00 0

8aff5000-8aff6000 —p 8aff5000 00:00 0

8aff6000-8b9f6000 rwxp 8aff6000 00:00 0

8b9f6000-8b9f9000 —p 8b9f6000 00:00 0

8b9f9000-8ba77000 rwxp 8b9f9000 00:00 0

8ba77000-8ba7a000 r-xs 00000000 08:01 100091 /opt/wildfire/plugins/search/lib/plugin-search.jar

8ba7a000-8ba7d000 r-xs 00000000 08:01 100090 /opt/wildfire/plugins/search/lib/plugin-search-jspc.jar

8ba7d000-8ba80000 —p 8ba7d000 00:00 0

8ba80000-8bafe000 rwxp 8ba80000 00:00 0

8bafe000-8bb01000 —p 8bafe000 00:00 0

8bb01000-8bb7f000 rwxp 8bb01000 00:00 0

8bb7f000-8bb82000 —p 8bb7f000 00:00 0

8bb82000-8bc00000 rwxp 8bb82000 00:00 0

8bc71000-8bc75000 r-xp 00000000 08:01 77566 /lib/libnss_dns-2.3.4.so

8bc75000-8bc77000 rwxp 00003000 08:01 77566 /lib/libnss_dns-2.3.4.so

8bc7d000-8bc80000 —p 8bc7d000 00:00 0

8bc80000-8bcfe000 rwxp 8bc80000 00:00 0

8bcfe000-8bd01000 —p 8bcfe000 00:00 0

8bd01000-8bd7f000 rwxp 8bd01000 00:00 0

8bd7f000-8bd82000 —p 8bd7f000 00:00 0

8bd82000-8befd000 rwxp 8bd82000 00:00 0

8befd000-8bf00000 —p 8befd000 00:00 0

8bf7f000-8bf82000 —p 8bf7f000 00:00 0

8bf82000-8c0ff000 rwxp 8bf82000 00:00 0

8c0ff000-8c100000 —p 8c0ff000 00:00 0

8c10c000-8c10f000 —p 8c10c000 00:00 0

8c10f000-8c18d000 rwxp 8c10f000 00:00 0

8c18d000-8c190000 —p 8c18d000 00:00 0

8c190000-8c20e000 rwxp 8c190000 00:00 0

8c20e000-8c27f000 r-xs 00000000 08:01 86658 /opt/wildfire/plugins/admin/webapp/WEB-INF/lib/admin-jsp.jar

8c27f000-8c282000 —p 8c27f000 00:00 0

8c282000-8c3fe000 rwxp 8c282000 00:00 0

8c3fe000-8c400000 —p 8c3fe000 00:00 0

8c447000-8c44c000 r-xp 00000000 08:01 77677 /lib/libcrypt-2.3.4.so

8c44c000-8c44e000 rwxp 00004000 08:01 77677 /lib/libcrypt-2.3.4.so

8c44e000-8c475000 rwxp 8c44e000 00:00 0

8c475000-8c478000 —p 8c475000 00:00 0

8c478000-8c4f6000 rwxp 8c478000 00:00 0

8c4f6000-8c4f9000 —p 8c4f6000 00:00 0

8c4f9000-8c577000 rwxp 8c4f9000 00:00 0

8c577000-8c57a000 —p 8c577000 00:00 0

8c57a000-8c5f8000 rwxp 8c57a000 00:00 0

8c5f8000-8c5fb000 —p 8c5f8000 00:00 0

8c5fb000-8c679000 rwxp 8c5fb000 00:00 0

8c679000-8c67c000 —p 8c679000 00:00 0

8c67c000-8c6fa000 rwxp 8c67c000 00:00 0

8c6fa000-8c6fd000 —p 8c6fa000 00:00 0

8c6fd000-8c77b000 rwxp 8c6fd000 00:00 0

8c77b000-8c77e000 —p 8c77b000 00:00 0

8c77e000-8c7fc000 rwxp 8c77e000 00:00 0

8c7fc000-8c7ff000 —p 8c7fc000 00:00 0

8c7ff000-8c87d000 rwxp 8c7ff000 00:00 0

8c87d000-8c880000 —p 8c87d000 00:00 0

8c880000-8c8fe000 rwxp 8c880000 00:00 0

8c8fe000-8c901000 —p 8c8fe000 00:00 0

8c901000-8c97f000 rwxp 8c901000 00:00 0

8c97f000-8c982000 —p 8c97f000 00:00 0

8c982000-8cafc000 rwxp 8c982000 00:00 0

8cafc000-8cb00000 —p 8cafc000 00:00 0

8cb01000-8cb02000 r-xp 00000000 08:01 81607 /lib/security/pam_deny.so

8cb02000-8cb03000 rwxp 00000000 08:01 81607 /lib/security/pam_deny.so

8cb03000-8cb07000 r-xp 00000000 08:01 81648 /lib/security/pam_winbind.so

8cb07000-8cb08000 rwxp 00003000 08:01 81648 /lib/security/pam_winbind.so

8cb08000-8cb30000 r-xp 00000000 08:01 118323 /lib/security/pam_securid.so

8cb30000-8cb38000 rwxp 00028000 08:01 118323 /lib/security/pam_securid.so

8cb38000-8cb3a000 rwxp 8cb38000 00:00 0

8cb3a000-8cb3d000 r-xp 00000000 08:01 81608 /lib/security/pam_env.so

8cb3d000-8cb3e000 rwxp 00002000 08:01 81608 /lib/security/pam_env.so

8cb3f000-8cb7f000 rwxp 8cb3f000 00:00 0

8cb7f000-8cb82000 —p 8cb7f000 00:00 0

8cb82000-8ccff000 rwxp 8cb82000 00:00 0

8ccff000-8cd00000 —p 8ccff000 00:00 0

8cd00000-8cd0f000 rwxp 8cd00000 00:00 0

8cd0f000-8cd1e000 r-xs 00000000 08:01 86651 /opt/wildfire/lib/startup.jar

8cd1e000-8cd3f000 r-xs 00000000 08:01 106286 /opt/wildfire/lib/servlet.jar

8cd3f000-8cd59000 r-xs 00000000 08:01 106288 /opt/wildfire/lib/jasper-runtime.jar

8cd59000-8cdae000 r-xs 00000000 08:01 86647 /opt/wildfire/lib/mail.jar

8cdae000-8cdbc000 r-xs 00000000 08:01 86638 /opt/wildfire/lib/activation.jar

8cdbc000-8cdea000 r-xs 00000000 08:01 106290 /opt/wildfire/lib/hsqldbutil.jar

8cdea000-8ce30000 r-xs 00000000 08:01 106295 /opt/wildfire/lib/jtds.jar

8ce30000-8cea9000 r-xs 00000000 08:01 106289 /opt/wildfire/lib/hsqldb.jar

8cea9000-8ceff000 r-xs 00000000 08:01 106291 /opt/wildfire/lib/jasper-compiler.jar

8ceff000-8cf1b000 r-xs 00000000 08:01 106294 /opt/wildfire/lib/commons-el.jar

8cf1b000-8d2e8000 r-xs 00000000 08:01 106292 /opt/wildfire/lib/wildfire.jar

8d2e8000-8d2fb000 r-xs 00000000 08:01 86645 /opt/wildfire/lib/jdic.jar

8d2fb000-8d400000 r-xs 00000000 08:01 86639 /opt/wildfire/lib/bouncycastle.jar

8d400000-8d501000 rwxp 8d400000 00:00 0

8d501000-8d504000 rwxp 8d501000 00:00 0

8d504000-8d570000 r-xs 00000000 08:01 106287 /opt/wildfire/lib/mysql.jar

8d570000-8d93d000 r-xs 00000000 08:01 106292 /opt/wildfire/lib/wildfire.jar

8d93d000-8d94c000 r-xs 00000000 08:01 86651 /opt/wildfire/lib/startup.jar

8d94c000-8d96d000 r-xs 00000000 08:01 106286 /opt/wildfire/lib/servlet.jar

8d96d000-8d9cf000 r-xs 00000000 08:01 106293 /opt/wildfire/lib/postgres.jar

8d9cf000-8da3b000 r-xs 00000000 08:01 106287 /opt/wildfire/lib/mysql.jar

8da3b000-8da90000 r-xs 00000000 08:01 86647 /opt/wildfire/lib/mail.jar

8da90000-8dad6000 r-xs 00000000 08:01 106295 /opt/wildfire/lib/jtds.jar

8dad6000-8dae9000 r-xs 00000000 08:01 86645 /opt/wildfire/lib/jdic.jar

8dae9000-8db03000 r-xs 00000000 08:01 106288 /opt/wildfire/lib/jasper-runtime.jar

8db03000-8db59000 r-xs 00000000 08:01 106291 /opt/wildfire/lib/jasper-compiler.jar

8db59000-8db87000 r-xs 00000000 08:01 106290 /opt/wildfire/lib/hsqldbutil.jar

8db87000-8dc00000 r-xs 00000000 08:01 106289 /opt/wildfire/lib/hsqldb.jar

8dc00000-8dcfd000 rwxp 8dc00000 00:00 0

8dcfd000-8dd00000 —p 8dcfd000 00:00 0

8dd00000-8dd62000 r-xs 00000000 08:01 106293 /opt/wildfire/lib/postgres.jar

8dd62000-8dd7e000 r-xs 00000000 08:01 106294 /opt/wildfire/lib/commons-el.jar

8dd7e000-8de83000 r-xs 00000000 08:01 86639 /opt/wildfire/lib/bouncycastle.jar

8de83000-8de91000 r-xs 00000000 08:01 86638 /opt/wildfire/lib/activation.jar

8de91000-8de94000 —p 8de91000 00:00 0

8de94000-8df12000 rwxp 8de94000 00:00 0

8df12000-8df61000 r-xs 00000000 08:01 94063 /opt/wildfire/.install4j/i4jruntime.jar

8df61000-8df8c000 r-xs 00000000 08:01 86099 /opt/wildfire/jre/lib/ext/sunpkcs11.jar

8df8c000-8e04f000 r-xs 00000000 08:01 86095 /opt/wildfire/jre/lib/ext/localedata.jar

8e04f000-8e075000 r-xs 00000000 08:01 86098 /opt/wildfire/jre/lib/ext/sunjce_provider.jar

8e075000-8e077000 r-xs 00000000 08:01 86096 /opt/wildfire/jre/lib/ext/dnsns.jar

8e077000-8e078000 —p 8e077000 00:00 0

8e078000-8e0f8000 rwxp 8e078000 00:00 0

8e0f8000-8e0fb000 —p 8e0f8000 00:00 0

8e0fb000-8e179000 rwxp 8e0fb000 00:00 0

8e179000-8e17c000 —p 8e179000 00:00 0

8e17c000-8e1fa000 rwxp 8e17c000 00:00 0

8e1fa000-8e1fd000 —p 8e1fa000 00:00 0

8e1fd000-8e27b000 rwxp 8e1fd000 00:00 0

8e27b000-8e27e000 —p 8e27b000 00:00 0

8e27e000-8e2fc000 rwxp 8e27e000 00:00 0

8e2fc000-8e2ff000 —p 8e2fc000 00:00 0

8e2ff000-8e37d000 rwxp 8e2ff000 00:00 0

8e37d000-8e380000 —p 8e37d000 00:00 0

8e380000-8e3fe000 rwxp 8e380000 00:00 0

8e3fe000-8e401000 —p 8e3fe000 00:00 0

8e401000-8e47f000 rwxp 8e401000 00:00 0

8e47f000-8e480000 —p 8e47f000 00:00 0

8e480000-8e5fa000 rwxp 8e480000 00:00 0

8e5fa000-8e600000 —p 8e5fa000 00:00 0

8e600000-8e603000 rwxp 8e600000 00:00 0

8e603000-8e60d000 r-xp 00000000 08:01 77687 /lib/libaudit.so.0.0.0

8e60d000-8e611000 rwxp 00009000 08:01 77687 /lib/libaudit.so.0.0.0

8e611000-8e618000 r-xp 00000000 08:01 77690 /lib/libpam.so.0.77

8e618000-8e619000 rwxp 00007000 08:01 77690 /lib/libpam.so.0.77

8e619000-8e61f000 rwxp 8e619000 00:00 0

8e61f000-8e621000 r-xp 00000000 08:01 106285 /opt/wildfire/lib/libshaj.so

8e621000-8e622000 rwxp 00001000 08:01 106285 /opt/wildfire/lib/libshaj.so

8e622000-8e636000 r-xp 00000000 08:01 86165 /opt/wildfire/jre/lib/i386/libnet.so

8e636000-8e637000 rwxp 00013000 08:01 86165 /opt/wildfire/jre/lib/i386/libnet.so

8e637000-8e638000 —p 8e637000 00:00 0

8e638000-8e6b8000 rwxp 8e638000 00:00 0

8e6b8000-8e6b9000 —p 8e6b8000 00:00 0

8e6b9000-8e739000 rwxp 8e6b9000 00:00 0

8e739000-8e73a000 —p 8e739000 00:00 0

8e73a000-8e7ba000 rwxp 8e73a000 00:00 0

8e7ba000-8e7bb000 —p 8e7ba000 00:00 0

8e7bb000-8e84b000 rwxp 8e7bb000 00:00 0

8e84b000-8e85b000 rwxp 8e84b000 00:00 0

8e85b000-8e8af000 rwxp 8e85b000 00:00 0

8e8af000-8e93f000 rwxp 8e8af000 00:00 0

8e93f000-8e94f000 rwxp 8e93f000 00:00 0

8e94f000-8e95f000 rwxp 8e94f000 00:00 0

8e95f000-8e9b3000 rwxp 8e95f000 00:00 0

8e9b3000-8ea42000 rwxp 8e9b3000 00:00 0

8ea42000-8ea49000 rwxp 8ea42000 00:00 0

8ea49000-8ea5f000 rwxp 8ea49000 00:00 0

8ea5f000-90a20000 rwxp 8ea5f000 00:00 0

90a20000-92a60000 rwxp 90a20000 00:00 0

92a60000-9d0e0000 rwxp 92a60000 00:00 0

9d0e0000-af180000 rwxp 9d0e0000 00:00 0

af180000-afde0000 rwxp af180000 00:00 0

afde0000-b2a60000 rwxp afde0000 00:00 0

b2a60000-b2a70000 rwxp b2a60000 00:00 0

b2a70000-b2ae1000 rwxp b2a70000 00:00 0

b2ae1000-b2e99000 rwxp b2ae1000 00:00 0

b2e99000-b4ae1000 rwxp b2e99000 00:00 0

b4ae1000-b52ff000 r-xs 00000000 08:01 86221 /opt/wildfire/jre/lib/charsets.jar

b52ff000-b5313000 r-xs 00000000 08:01 86191 /opt/wildfire/jre/lib/jce.jar

b5313000-b538a000 r-xs 00000000 08:01 114241 /opt/wildfire/jre/lib/jsse.jar

b538a000-b53f3000 rwxp b538a000 00:00 0

b53f3000-b75dd000 r-xs 00000000 08:01 87067 /opt/wildfire/jre/lib/rt.jar

b75dd000-b75f0000 r-xp 00000000 08:01 86170 /opt/wildfire/jre/lib/i386/libzip.so

b75f0000-b75f2000 rwxp 00012000 08:01 86170 /opt/wildfire/jre/lib/i386/libzip.so

b75f2000-b7613000 r-xp 00000000 08:01 86151 /opt/wildfire/jre/lib/i386/libjava.so

b7613000-b7615000 rwxp 00020000 08:01 86151 /opt/wildfire/jre/lib/i386/libjava.so

b7615000-b7620000 r-xp 00000000 08:01 86169 /opt/wildfire/jre/lib/i386/libverify.so

b7620000-b7621000 rwxp 0000b000 08:01 86169 /opt/wildfire/jre/lib/i386/libverify.so

b7621000-b7629000 rwxs 00000000 08:08 86786 /tmp/hsperfdata_root/24753

b7629000-b762f000 r-xp 00000000 08:01 86172 /opt/wildfire/jre/lib/i386/native_threads/libhpi.so

b762f000-b7630000 rwxp 00006000 08:01 86172 /opt/wildfire/jre/lib/i386/native_threads/libhpi.so

b7630000-b7631000 rwxp b7630000 00:00 0

b7631000-b7632000 r-xp b7631000 00:00 0

b7632000-b7b7f000 r-xp 00000000 08:01 86175 /opt/wildfire/jre/lib/i386/server/libjvm.so

b7b7f000-b7be1000 rwxp 0054d000 08:01 86175 /opt/wildfire/jre/lib/i386/server/libjvm.so

b7be1000-b7ffa000 rwxp b7be1000 00:00 0

bfe00000-bfe03000 —p bfe00000 00:00 0

bfe03000-c0000000 rwxp bfe03000 00:00 0

ffffe000-fffff000 —p 00000000 00:00 0

VM Arguments:

jvm_args: -Dinstall4j.jvmDir=/opt/wildfire/jre -Dinstall4j.appDir=/opt/wildfire -Dexe4j.moduleName=/opt/wildfire/bin/wildfire -Xms128m -Xmx512m

java_command: com.install4j.runtime.Launcher start org.jivesoftware.wildfire.starter.ServerStarter false false /opt/wildfire/bin/…/logs/stderror.log /opt/wildfire/bin/…/logs/stdoutt.log true true false true

true 0 0 20 20 Arial 0,0,0 8 500 version 2.6.2 20 40 Arial 0,0,0 8 500 -1 -DwildfireHome=/opt/wildfire -Dwildfire.lib.dir=/opt/wildfire/lib start

Launcher Type: SUN_STANDARD

Environment Variables:

Signal Handlers:

SIGSEGV: , sa_mask[0]=0x7ffbfeff, sa_flags=0x14000004

SIGBUS: , sa_mask[0]=0x7ffbfeff, sa_flags=0x14000004

SIGFPE: , sa_mask[0]=0x7ffbfeff, sa_flags=0x14000004

SIGPIPE: , sa_mask[0]=0x7ffbfeff, sa_flags=0x14000004

SIGILL: , sa_mask[0]=0x7ffbfeff, sa_flags=0x14000004

SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000

SIGUSR2: , sa_mask[0]=0x00000000, sa_flags=0x14000004

SIGHUP: SIG_IGN, sa_mask[0]=0x00000000, sa_flags=0x00000000

SIGINT: SIG_IGN, sa_mask[0]=0x00000000, sa_flags=0x00000000

SIGQUIT: , sa_mask[0]=0x7ffbfeff, sa_flags=0x14000004

SIGTERM: , sa_mask[0]=0x7ffbfeff, sa_flags=0x14000004

S Y S T E M -

OS:Red Hat Enterprise Linux ES release 4 (Nahant Update 3)

uname:Linux 2.6.9-22.ELsmp #1 SMP Mon Sep 19 18:32:14 EDT 2005 i686

libc:glibc 2.3.4 NPTL 2.3.4

rlimit: STACK 10240k, CORE 0k, NPROC 32764, NOFILE 1024, AS infinity

load average:0.00 0.00 0.00

CPU:total 4 family 15, cmov, cx8, fxsr, mmx, sse, sse2, ht

Memory: 4k page, physical 2074760k(1373816k free), swap 2040212k(2040212k free)

vm_info: Java HotSpot™ Server VM (1.5.0_06-b05) for linux-x86, built on Nov 10 2005 10:56:33 by java_re with gcc 3.2.1-7a (J2SE release)

2

Hi,

I first thought of http://www.jivesoftware.org/community/thread.jspa?threadID=17121 and I’‘m posting the link here just as a note so one can review this old thread, it’'s also about libshaj but the error did occur in libnss_winbind.

It crashes in the morning when a lot of users login, doesn’'t it? This should not be the case as Matt did fix this with JM-519

LG

3

LG,

Thanks for posting the link to that thread; it was definitely similar to the problem I’'ve been experiencing. The problem is my Wildfire installation actually crashes in the evening when load should be light.

After reading that thread and reviewing my particular configuration, I determined that I could safely remove winbind from my PAM configuration. Maybe, this change will correct the issue?

Any other information about the crash or pointers on correcting it are appreciated. Thanks!

4

It looks like it crashed in the SecurID pam module. Are you authenticating against SecurID? Perhaps there is something going weird when a user enters a bad number? Does your ACE server have any logging at that same time?

Message was edited by: slushpupie

5

Hi,

I wonder if http://sources.redhat.com/ml/bug-glibc/2002-11/msg00087.html describes your problem - it’‘s about localtime() which was in 2002 (and probably still is - I have no idea) not thread safe. I can’'t verify whether SecurID uses localtime() but it could be something to check or to ask a PAM/SecurID mailing list or forum

LG

6

The backtrace shows the error happing during the authentication, while trying to log something (RSA_log), in what appears to be looking up the envrionment. In fact, the backtrace ends at getenv, and later in the Java dump it was unable to output the environment as well.

The only reason I can think of that getenv would fail is not enough availible memory, but nothing else from the dump indicates that. Can you check what the memory usage is at the time of failure? Also, if you can afford the downtime, run a full memory check (use memtest86) to make sure you arnt just hitting bad memory.

7

Yeah, I am using pam_securid.so via PAM to authenticate users. Unfortunately, I don’‘t have access to the ACE server logs to know what’'s happening on that end…

Is Wildfire’'s RADIUS authentication more robust?