Pubsub: Why is .send() 10x slower than .publish()

Openfire Openfire Support
1

I have a server on EC2 with openfire and use pubsub to send messages to it

If my smack client uses node.publish() I can send about 150 messages/second (which might be the limit of my laptop…).

But if I switch to send using node.send I never get more than about 5-10 messages second.

Why is it so much slower?

I would expect some slowness but not that much. Is send so much heavier on my client or did I miss something on the server side? The server has plenty of spare CPU, memory and IO while sending from my client in a tight loop.

2

That’s odd. Have you been able to isolate the bottleneck to either Smack or Openfire?

3

As per the JavaDoc:

publish() is async, send() is sync.

In other words send() waits for the server response before the method returns. publish() just sends and forgets (does not block).

That would explain the performance difference.

4

Yes, very odd.

If I look on the server side while running there seems to be no lack of resources. I checked CPU, I/O to disk and memory.

As I run in EC2 I’m not sure about network bandwidth, but I switched to a m3.xlarge instance which is supposed to have good network performance.

So I can’t find anything obvious on the serverside that would need attention in the server infrastructure.

I don’t even know where to start looking …

5

Yes, some performance difference. That I can live with.

Come to think of it: How far into the server is it synchronous? Are there a database write with a commit involved before it returns to the client? That would explain why I don’t see much more performance with a bigger server.

6

Yes, usually PubSub items are persisted into the database. It depdends on your node configuration.

I expect the result is returned AFTER the publishing process is complete.

Even if the result is returned before persisting it always takes time for the client to wait for the response.

7

To Add to CSH’s answer: the XMPP action (within one client) is synchronous (the client will simply wait for the response to arrive), but most of the actual processing is not. Concurrent clients can happily send() simultaneously, without much of a performance hit.

send() does not do ‘more’ in comparison to publish(). It simply waits for a response from the server that the action succeeded, before continuing. The waiting bit does not consume any resources though.

8

The waiting bit does not consume any resources though.

It still blocks a thread, which could otherwise do other work.

9

Actually I tried to configure to node not to presist any messages. The pubsubitems table seems to be empty so it looks like it does not do any database writes. The disk I/O is also pretty low so…

10

I absolutely agree, there are important differences, primarily client-sided. The impact however won’t be noticeable in the sense that things will improve by adding more memory, bandwidth or CPU power server sided (which is what Marco was referring to).

11

The strange thing is that the same setup (hw/sw) does about 150 messages/s when I switch to send instead of publish. I did expect lot less of performance hit.

12

It is to be expected, for reasons described by @CSH above.

There’s a big difference between the two, which is especially noticeable when you repeatedly invoke them:

  • publish() will send out messages and continue immediately;
  • send() will send out a message, then wait for confirmation that it was delivered, before continuing.

So, send() will wait for the message to traverse the network, be processed by Openfire, and the response have transferred back, which obviously takes a lot longer than publish().