Offline messages: Do not receive all messages

Hi,

I have another problem with the offline store. I use one server to send messages to Jive Messenger 2.2.0 (SuSE 9.2 with mysql) and another server (smack 2.0.0) who connects each half an hour to retrieve the data. This time (not half an hour, just 20 minutes or so) one server wrote 194 messages to the other account which were stored in the jiveOffline table.

When the half-an-hour server connected it only received 72 messages but the jiveOffline table was empty afterwards. This is the code that gets the data:

int packetCount = 0;

// if no PacketCollector exists it is created this way:

// jabberConnection.createPacketCollector(new PacketTypeFilter(Message.class));

PacketCollector collector = connectorData.getPacketCollector();

Packet packet = collector.nextResult(timeout); // timeout is 5000 millisec

while (packet != null)

{

packetCount++;

processPacket((Message)packet);

packet = collector.nextResult(500);

}

log.finer(packetCount+" Messages received.");

return packetCount;

I need some help here.

Thanks,

Torsten.

PS: For another problem which describes my Server setup in greater detail please see here: http://www.jivesoftware.org/community/thread.jspa?threadID=16372&tstart=0

Have you specified large enough size of Offline Storage? Maybe max.size of storage was exceeded.

Yes, I have.

I have set the server to accept all messages even if the message space is exceeded.

And the server indeed stores the messages in its db. Then in one instant, when the other jabber client connects, all records are read from the jiveOffline table, but not all arive at the smack client.

This also happens with the embedded hsqldb, i just checked it.

So there must be some problem (probably in my code) which i don’'t see.

Torsten.

Hey Torsten,

So you have confirmed that all messages were stored in the offline table. Could you use the smack debugger and check in the raw received tab if all messages are being received by Smack? You may use the ConsoleDebugger that will print the captured info to the stdout.

Regards,

– Gato

I can reproduce the problem with a tiny test class.

First I send 500 messages with a simple JavaBean (empty for the test) to the server for a second jabber address.

Then I check that all 500 are put into the database.

I then start the receiver. Here it depends on the loop timeout how many messages are read. if the timeout is to low or I use pollResult() then I end up with only 12 of the 500 messages (depends on connection speed I guess). The problem I see is that the messages are lost. The jiveOffline table is empty. I have no chance to get hold of it.

Is there any security switch in the server that would delete from the db only those records that could be delivered to a client?

Thanks,

Torsten.

Hey Torsten,

We still have to locate the faulty section that is missing packets. If there is something wrong on the client then the server cannot figure that out and will consider the packets delivered and will remove them from the table. There is no ACK mechanism between the server and the client to prevent this.

I’'m attaching a debugger plugin that you can drop in the plugins folder. The plugin will print all sent and received packets by the server. With this debugging info + the smack debugger we should be able to pinpoint the error.

Regards,

– Gato

Hi,

I tested with the debugger (plugin test comes now).

The jiveOffline table held 524 records, the debugger showed a receive of 396. Afterwards the table was empty.

I’'m testing with the debugger plugin now.

I also have another problem described in this thread (http://www.jivesoftware.org/community/thread.jspa?messageID=106356&#106356)

I want to know if there are any messages in the queue of PacketWriter or PacketReader so that I close the connection only after the internal queue is empty but didn’'t yet succeed to find a way to this information. The PacketWriterListener notifies me for every packet that is SCHEDULED to be sent but not actually sent and received by the server.

Anyway, I’'m testing with the debugger.jar now…

I installed the debugger.jar and jive extracts it but the ./logs/debugger.log file is empty after a send and receive cycle. I have saved the packets that were sent and those that were received. How can I attach the files? Or can you give me your email address so I can mail them to you.

I have changed the mechanism that receives packets to use the PacketFilter event style. This way I now receive all messages. I just need to find a way to check if the queue is empty.

Torsten.

I think the problem with the lost data is between RAW data received from the server and the smack interpreted data. As I see in the debugger the RAW data comes in chunks, there is not one packet per offline message but a whole chunk of data in one. Then it is processed internally and given to the PacketListener Somewhere in between pollResult() says there is nothing to process and I close the XMPPConnection which deletes data that is still in the queue. This is how I understand the mechanism.

I’'ve implemented my own debugger, retrieve it via reflection and use it to asure that all messages that should be sent have indeed been given to the server.

It would be good if there where a more intuitive way to check the internal queues…