High load on Connection Manager since move to NIO

I’'m performing some load test on the latest Connection Manager v3.3.0 and I have observed some strange behaviour concerning CPU usage.

My test configuration:

  • 2 CM with 1G RAM, Dual proc Xeon @ 1.86 Ghz (gadolinium and terbium on the graphs)

  • WF 8G Ram, Quad proc Xeon @ 2.0 Ghz (rhenium on the graphs)

  • 1 postgres DB (ruthenium on the graphs)

When I connect 10 000 users (meaning 5000 per CM) the load on the CM is about 80% CPU with a load average of 12 and I can’'t really understand why since these connections are not generating traffic.

The graphs generated by tsung shows that CPU on the CM is directly related to the number of opened connections.

I uploaded the graphs and config files of the tests I have performed with tsung:

  • 10 000 connected users that do nothing

  • 10 000 connected users chatting and exchanging presence with ~30 contacts

Please note that since upgrade to nio tsung doesn’'t detect client disconnection meaning that number of simultaneous user is wrong. But you can follow the CPU load of the CM to follow number of connected clients.

I’'ve changed some few parameters on the CM configuration:

  • allow machines to have more than 1 000 opened connection

  • set socket send buffer to 5K, receive to 10K, by default it was taking 65K/connections and was consuming all my java memory

  • CPU number 2, and I raise the number of thread to read incoming traffic from 16->32 (no noticeable changes on load).

My full config file: manager.xml

Am I missing some configuration parameter or is there a problem in the CM?

Any thought?

Still no one?