Messages not arrive between online users

Edgan · February 11, 2010, 8:18pm

Sometimes the messages not arrive between online users, Why???.

Details:

“Server” ***********

SO: Windows XP

Physical memory: 2GB

Processor: Pentium 4 - 2.26Ghz

Firewall:

-Comodo with Attack Detection Settings like these:

-Intrusion detection: TCP Flood: 2047 packets/second. Duration: 20 seconds

-Intrusion detection:UDP Flood: 2047 packets/second. Duration: 20 seconds

-Intrusion detection:ICMP Flood: 2047 packets/second. Duration: 20 seconds

-Miscellaneous: [Checked] Block Fragmented IP datagrams. <<< I let to uncheck this option.

IM software *****************

Name: Openfire_3_6_4

External Database: MYSQL

Users: Were import using Import/Export plugin

Running mode: Windows service (openfire-service.exe)

Parameters openfire-service.vmoptions:

-Xms512m

-Xmx896m

Server Info: ***********************

-All the users are in xGroup by default.

-Cache settings customization:

cache.username2roster.size = 2097152

I customized this because the Roster cache always had 95% of percent used and frecuently many logged users appears offline. With this changued offline problem was fixed.

Environment
Java Version:
1.6.0_03 Sun Microsystems Inc. – Java HotSpot™ Server VM
Appserver:
jetty-6.1.x
Host Name:
estock2010
OS / Hardware:
Windows XP / x86
Locale / Timezone:
en / Central Standard Time (-6 GMT)
Java Memory

223,62 MB of 886,06 MB (25,2%) used

System Properties

Below is a list of the system properties. Values for password sensitive fields are hidden. Long property names and values are clipped. Hold your mouse over the property name to see the full value or to see both the full name and value, click the edit icon next to the property.

System Properties

**************** Log Viewer ******************

Errors:

line
1
2
2010.02.11 11:31:40 [org.jivesoftware.util.log.util.CommonsLogFactory$1.error(CommonsLogFactory.jav a:88)
] Line=19 The content of element type “dwr” must match “(init?,allow?,signatures?)”.

Warn:

line
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
2010.02.11 10:48:06 No ACK was received when sending stanza to: org.jivesoftware.openfire.nio.NIOConnection@284a01 MINA Session: (SOCKET, R: /189.177.238.130:49175, L: /999.999.999.999:5222, S: 0.0.0.0/0.0.0.0:5222)
2010.02.11 12:50:11 Invalid presence type
java.lang.IllegalArgumentException: No enum const class org.xmpp.packet.Presence$Type.invisible
at java.lang.Enum.valueOf(Unknown Source)
at org.xmpp.packet.Presence$Type.valueOf(Presence.java:322)
at org.xmpp.packet.Presence.getType(Presence.java:107)
at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:229)
at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:176)
at org.jivesoftware.openfire.nio.ConnectionHandler.messageReceived(ConnectionHandl er.java:133)
at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.messageReceived (AbstractIoFilterChain.java:570)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)
at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)
at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)
at org.apache.mina.common.IoFilterAdapter.messageReceived(IoFilterAdapter.java:80)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)
at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)
at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)
at org.apache.mina.filter.codec.support.SimpleProtocolDecoderOutput.flush(SimplePr otocolDecoderOutput.java:58)
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:185)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)
at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)
at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)
at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java :239)
at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(Execut orFilter.java:283)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)
at java.lang.Thread.run(Unknown Source)

Debug:

2010.02.11 12:54:03 149286 (01/05/00) - Connection #14 tested: OK
2010.02.11 12:54:03 149286 (01/05/00) - Connection #15 tested: OK
2010.02.11 12:54:03 149287 (01/05/00) - Connection #15 tested: OK
2010.02.11 12:54:10 ConnectionHandler: Closing connection that has been idle: org.jivesoftware.openfire.nio.NIOConnection@de3f2c MINA Session: (SOCKET, R: /201.137.64.241:57087, L: /201.144.207.221:5222, S: 0.0.0.0/0.0.0.0:5222)
2010.02.11 12:54:10 149287 (01/05/00) - Connection #16 tested: OK
2010.02.11 12:54:10 149288 (01/05/00) - Connection #16 tested: OK

2010.02.11 12:58:34 JettyLog: EXCEPTION
java.nio.channels.ClosedChannelException
at sun.nio.ch.SocketChannelImpl.ensureReadOpen(Unknown Source)
at sun.nio.ch.SocketChannelImpl.read(Unknown Source)
at org.mortbay.io.nio.ChannelEndPoint.fill(ChannelEndPoint.java:122)
at org.mortbay.jetty.security.SslHttpChannelEndPoint.fill(SslHttpChannelEndPoint.j ava:444)
at org.mortbay.jetty.security.SslHttpChannelEndPoint.fill(SslHttpChannelEndPoint.j ava:200)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:282)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:205)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:395)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:488)
2010.02.11 12:58:34 JettyLog: EOF
2010.02.11 12:58:35 JettyLog: EXCEPTION
java.nio.channels.ClosedChannelException
at sun.nio.ch.SocketChannelImpl.ensureReadOpen(Unknown Source)
at sun.nio.ch.SocketChannelImpl.read(Unknown Source)
at org.mortbay.io.nio.ChannelEndPoint.fill(ChannelEndPoint.java:122)
at org.mortbay.jetty.security.SslHttpChannelEndPoint.fill(SslHttpChannelEndPoint.j ava:444)
at org.mortbay.jetty.security.SslHttpChannelEndPoint.fill(SslHttpChannelEndPoint.j ava:200)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:282)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:205)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:395)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:488)
2010.02.11 12:58:35 JettyLog: EOF
2010.02.11 12:58:40 JettyLog: EXCEPTION
java.nio.channels.ClosedChannelException
at sun.nio.ch.SocketChannelImpl.ensureReadOpen(Unknown Source)
at sun.nio.ch.SocketChannelImpl.read(Unknown Source)
at org.mortbay.io.nio.ChannelEndPoint.fill(ChannelEndPoint.java:122)
at org.mortbay.jetty.security.SslHttpChannelEndPoint.fill(SslHttpChannelEndPoint.j ava:444)
at org.mortbay.jetty.security.SslHttpChannelEndPoint.fill(SslHttpChannelEndPoint.j ava:200)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:282)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:205)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:395)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:488)
2010.02.11 12:58:40 JettyLog: EOF
2010.02.11 12:58:40 JettyLog: EXCEPTION
java.nio.channels.ClosedChannelException
at sun.nio.ch.SocketChannelImpl.ensureReadOpen(Unknown Source)
at sun.nio.ch.SocketChannelImpl.read(Unknown Source)
at org.mortbay.io.nio.ChannelEndPoint.fill(ChannelEndPoint.java:122)
at org.mortbay.jetty.security.SslHttpChannelEndPoint.fill(SslHttpChannelEndPoint.j ava:444)
at org.mortbay.jetty.security.SslHttpChannelEndPoint.fill(SslHttpChannelEndPoint.j ava:200)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:282)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:205)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:395)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:488)
2010.02.11 12:58:40 JettyLog: EOF
2010.02.11 12:58:40 149523 (01/05/00) - Connection #16 tested: OK
2010.02.11 12:58:40 149524 (02/05/00) - Connection #17 tested: OK
2010.02.11 12:58:40 149525 (02/05/00) - Connection #17 tested: OK
2010.02.11 12:58:40 149525 (02/05/00) - Connection #18 tested: OK
2010.02.11 12:58:40 149526 (02/05/00) - Connection #18 tested: OK
2010.02.11 12:58:40 Stat: muc_traffic. Last sample: 1265914620. New sample: 1265914680
2010.02.11 12:58:40 149526 (02/05/00) - Connection #14 tested: OK
2010.02.11 12:58:41 149527 (02/05/00) - Connection #14 tested: OK

2010.02.11 13:05:22 PrivacyList: Packet was blocked:

f8dac1ba0eb7e25b1f1b0d406635ea98768ab4d0

NNNNNN
8

2010.02.11 13:05:22 PrivacyList: Packet was blocked:

8b426c843e9b2e4a4ed9f0a0188fe78d9ffbee6c

8

2010.02.11 13:05:22 PrivacyList: Packet was blocked:

8baf6aad6dc8d8dac4b47aa5531bab8e461299e9

NNNNNNNNNNNN
8

2010.02.11 13:05:22 150101 (01/05/00) - Connection #16 tested: OK
2010.02.11 13:05:22 150102 (01/05/00) - #16 registered a statement as closed which wasn’t known to be open. This could happen if you close a statement twice.
2010.02.11 13:05:22 150102 (01/05/00) - Connection #16 tested: OK
2010.02.11 13:05:22 150102 (01/05/00) - Connection #17 tested: OK
2010.02.11 13:05:22 150103 (01/05/00) - #17 registered a statement as closed which wasn’t known to be open. This could happen if you close a statement twice.
2010.02.11 13:05:22 150103 (01/05/00) - Connection #17 tested: OK
2010.02.11 13:05:22 150103 (01/05/00) - Connection #18 tested: OK

2010.02.11 13:50:41 154889 (02/06/00) - Connection #15 tested: OK
2010.02.11 13:50:41 Closing statement 1980305 (belonging to connection 19) automatically
2010.02.11 13:50:41 Closing statement 6ca010 (belonging to connection 19) automatically
2010.02.11 13:50:41 154889 (01/06/00) - Connection #19 tested: OK

Sessions >> Client sessions:

Name Resource

User1 Pandion

User2 8c6481dc <<<<< ¿Why?, if It uses Pandion too. I will check the version in each one.

User3 Pandion

*****Clients Stations.

SO: Win XP

Client Software: Pandion.

Thank you in advance.

Edgan · February 12, 2010, 5:43pm

I made some changues on my Openfire server.

Increase the cache.username2roster.size to 3145728 (MB). Now, it has this values:

Roster
3,00 MB
2,47 MB
82,5%
94,6%

When the Roster had 2MB, the percent used was 95%, now is 82.5%. If it’s necesary, I would increase it, one more time, to 4MB.

On security settings>>client conection settings, I selected Custom, check old SSL method available and disabled TSL method.

On compression settings >>client compression policy, check not available.

Firewall:

-Comodo with Attack Detection Settings like these:

-Intrusion detection: TCP Flood: 9999 packets/second. Duration: 60 seconds <<< Current configuration. Max value accepted

-Intrusion detection:UDP Flood: 9999 packets/second. Duration: 60 seconds <<< Current configuration. Max value accepted

-Intrusion detection:ICMP Flood: 9999 packets/second. Duration: 60 seconds <<< Current configuration. Max value accepted

-Intrusion detection: Miscellaneous: [Unchecked] Block Fragmented IP datagrams. <<< Current configuration

I found in Pandion’s forums that the client pandion had (or has, let me check to confirm) problems with the TSL method(confirmed). My clients used this method to conect with my server. Also, pandion and openfire have problems working together with compression traffic.

Pandion 2.5 not send a correct presence packet to openfire. I confirm it seeing the colum Resource values in the Sessions, the values are hexadecimal characters. To fix it you need to reinstall pandion. But first unistall, erase c:\Program Files\Pandion folder and after install again.

Pandion 2.5 have problems with networking that fixed in the latest version.

This morning, the numbers of concurrent conected users were about 110. They don’t reported problems, yet.

I am monitoring the server and instant messages users to inform you the result at the end of this day.

Edgan · April 9, 2010, 6:54pm

The solution was increase the cache.username2roster.size to mantain the percent used under the 90%.

My server is online since I posted the previous message until this moment, without problems.

Explanation:

In cache.username2roster.size is stored the users presence information. This information is used to show users online, show users status and send messages between users.

Resume:

Don’t permit that the cache variables are up to 90%. Fix it increasing its size.

Edgan · April 9, 2010, 6:57pm

Don’t permit that the cache variables are up to 90%. Fix it increasing its size.