Database connection error handling issue

John_Nothingmore · July 19, 2005, 12:04pm

It’'s quite annoying bug (database is PostgreSQL 7.4.7)

Make some users connected and authenticated
Turn off database for few minutes (in my case for backup…)
Try to login another user (it will fail but that is OK, since database is down) - few exceptions will be thrown (one nested if I correctly recall)
Turn database on
Messenger will reconnect to database BUT trying to connect with another client will fail… Those already connected will have jabber running, those not will never connect. Moreover - admin console will be dead…

Not trying to connect during database inactivity will make server behave correctly…

Ryan_Graham · July 19, 2005, 3:19pm

Hi mnemonic,

Messenger will reconnect to database BUT trying to

connect with another client will fail… Those

already connected will have jabber running, those not

will never connect. Moreover - admin console will be

dead…

It sounds to me as if Messenger never reconnects to the database. How are you checking to see that Messenger has reconnected?

Thanks,

Ryan

John_Nothingmore · July 20, 2005, 5:35am

I’'m turning off database every night for few minutes. So if your assumption (Messenger never reconnects) were true, problem would be every morning… But problem arises sometimes only…
Running ps command in shell shows persistent connections to right database…

So, Messenger reconnects - but has big mess in internal structures and is working incorrectly…

Ryan_Graham · July 20, 2005, 1:35pm

Hi mnemonic,

Ah, you didn’'t originally mention that the problem was intermintent. Are you seeing any messages in the logs on the mornings when Messenger fails to reconnect?

Thanks,

Ryan

John_Nothingmore · July 22, 2005, 5:11am

Here we go:

ps afx

24478 ? S 0:02 /bin/java -server -Dinstall4j.jvmDir= -Dinstall4j.appDir=/opt/jive_messenger -Dexe4j.moduleName=/opt/jive_messenger/bin/messenge

24479 ? S 0:01 _ /bin/java -server -Dinstall4j.jvmDir= -Dinstall4j.appDir=/opt/jive_messenger -Dexe4j.moduleName=/opt/jive_messenger/bin/mess

24480 ? S 0:22 _ /bin/java -server -Dinstall4j.jvmDir= -Dinstall4j.appDir=/opt/jive_messenger -Dexe4j.moduleName=/opt/jive_messenger/bin/

…

24766 ? S 0:00 _ /bin/java -server -Dinstall4j.jvmDir= -Dinstall4j.appDir=/opt/jive_messenger -Dexe4j.moduleName=/opt/jive_messenger/bin/

…

24585 ? S 0:00 _ postgres: jabber jabber 127.0.0.1 idle

…

24612 ? S 0:00 _ postgres: jabber jabber 127.0.0.1 idle

so connection to database was established, messenger was running (morning state…) but no single person could connect to messenger

messenger logs:

2005.07.22 01:04:27 org.jivesoftware.messenger.auth.DefaultAuthProvider.authenticate(DefaultAuthProv ider.java:98) Exception in DbAuthProvider

org.postgresql.util.PSQLException: An I/O error has occured while flushing the output - Exception: java.net.SocketException: Broken pipe

Stack Trace:

java.net.SocketException: Broken pipe

at java.net.SocketOutputStream.socketWrite0(Native Method)

at java.net.SocketOutputStream.socketWrite(Unknown Source)

at java.net.SocketOutputStream.write(Unknown Source)

at java.io.BufferedOutputStream.flushBuffer(Unknown Source)

at java.io.BufferedOutputStream.flush(Unknown Source)

at org.postgresql.core.PGStream.flush(PGStream.java:411)

at org.postgresql.core.QueryExecutor.sendQueryV3(QueryExecutor.java:338)

at org.postgresql.core.QueryExecutor.executeV3(QueryExecutor.java:121)

at org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:100)

at org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:43)

at org.postgresql.jdbc1.AbstractJdbc1Statement.execute(AbstractJdbc1Statement.java :517)

at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java :50)

at org.postgresql.jdbc1.AbstractJdbc1Statement.executeQuery(AbstractJdbc1Statement .java:233)

at org.jivesoftware.messenger.auth.DefaultAuthProvider.authenticate(DefaultAuthPro vider.java:83)

at org.jivesoftware.messenger.auth.AuthFactory.authenticate(AuthFactory.java:118)

at org.jivesoftware.messenger.handler.IQAuthHandler.login(IQAuthHandler.java:224)

at org.jivesoftware.messenger.handler.IQAuthHandler.handleIQ(IQAuthHandler.java:14 1)

at org.jivesoftware.messenger.handler.IQHandler.process(IQHandler.java:48)

at org.jivesoftware.messenger.IQRouter.handle(IQRouter.java:202)

at org.jivesoftware.messenger.IQRouter.route(IQRouter.java:73)

at org.jivesoftware.messenger.PacketRouter.route(PacketRouter.java:65)

at org.jivesoftware.messenger.net.SocketReadThread.readStream(SocketReadThread.jav a:285)

at org.jivesoftware.messenger.net.SocketReadThread.run(SocketReadThread.java:105)

End of Stack Trace

at org.postgresql.core.PGStream.flush(PGStream.java:415)