The XMPPSocketConnection drops messages when messages are received in rapid succession. For example, if two messages are sent back to back by a client OR if the server replays chat room history when a user first enters a room, not all of the messages are received. I am not sure yet where the problem resides, but it appears there may be a problem with the flash.net.Socket … if I put a trace statement in the onRead method of SocketConn it simply never gets called for many of the messages … as if the Socket class is not generating ProgressEvents for every message.
I have verified that it is a problem with the XMPPSocketConnection class because if I just swap in the old XMPPConnection which uses an XMLSocket things work just fine … i.e. no messages are ever dropped.
I may dig around a bit more to see if I can nail down why it isn’t working, but I don’t really have too much time to spend on it provided XMPPConnection continues to do what I need it to do. However, this should be logged as a defect.
I know more than I did an hour ago, but I still haven’t figured it out. I’m posting here just in case someone can think of something in the interim that I haven’t yet considered.
I suspected that because the messages were coming in rapidly, many of them may be concatenated into a single ProgressEvent and that somehow all the data was not being processed. This indeed appears to be the case, but the behavior is rather unexpected. Basically, I modified the onRead method as follows to get some debug info.
private function onRead( event: ProgressEvent ): void
{
trace(event);
trace("bytesAvailable: " + bytesAvailable);
var s:String = readUTFBytes(bytesAvailable);
trace("received: " + s);
trace("length: " + s.length);
onSockRead( s );
}
The output Ibelow) shows that the first ProgressEvent actually indicates far more data is available than is ever read. The strange thing is that it appears the XIFF code is doing the right thing … so is this a bug in the Socket.readUTFBytes() method? Or is there some usage trick that is not well-documented? Both bytesAvailable and bytesLoaded (in the event) show 9000+ bytes available, but the string read by readUTFBytes() is clearly much shorter than that even though the bytesAvailable is passed as the parameter telling readUTFBytes() how many bytes to read.
[ProgressEvent type=“socketData” bubbles=false cancelable=false eventPhase=2 bytesLoaded=9267 bytesTotal=0]
bytesAvailable: 9267
received: <presence to="bill@nacdmynl.northlandcc.net/flex" from="testroom@conference.nacdmynl.northlandcc.net/billy"><priority>0</priority><x xmlns=“http://jabber.org/protocol/muc#user”><item affiliation=“none” role=“participant”/></x></presence>
length: 237
: got an incoming data event
<presence from="testroom@conference.nacdmynl.northlandcc.net/billy" to="bill@nacdmynl.northlandcc.net/flex"><priority>0</priority><x xmlns=“http://jabber.org/protocol/muc#user”><item affiliation=“none” role=“participant” /></x></presence>
: got a presence event
: got a room join event
[ProgressEvent type=“socketData” bubbles=false cancelable=false eventPhase=2 bytesLoaded=1426 bytesTotal=0]
bytesAvailable: 1426
received: <message to="bill@nacdmynl.northlandcc.net/flex" id=“m_38” type=“groupchat” from="testroom@conference.nacdmynl.northlandcc.net/billy"><body>ddafdafdsafdsfdsa</body><x xmlns=“jabber:x:delay” stamp=“20070827T18:37:55” from="testroom@conference.nacdmynl.northlandcc.net/billy"/></message>
length: 285
Clearly, all my missing messages are buried in the thousands of bytes that are never read by readUTFBytes() … but why? What else should be necessary to get readUTFBytes() to read all of the data?
Any ideas out there?
Bill
I have found at least a couple of examples of simple Socket usage in ActionScript which suggest the code in onRead() SHOULD work. But it isn’t working for me. I’m curious to know if anyone else is finding that it does work for them … keep in mind this only happens when messags arrive very rapidly such that many discrete messages get concatenated into a single ProgressEvent.
However, I did notice a lot of examples DON’T use readUTFBytes() so I wonder if others ran into issues with it.
I can’t find anything anywhere that suggests a reason readUTFBytes() might stop reading prematurely, but I wonder if something in the stream might cause it to stop reading … maybe a byte that isn’t a valid UTF character? Beats me … I’m grasping at straws now.
I considered rewriting the onRead() method to use some other read methods (e.g. readBytes() or readByte() in a loop) but I just don’t know if I have the time to change it, debug it, fully test it … and once again it will mean my copy of the source diverges even further from the baseline.
Any chance any of the many bugs, including this one, I’ve already found is going to be fixed in the near future?
I realise that this isn’t the answer you want, but I am able to reproduce the problem. I have a message source, which turns data from a file into XMPP messages, and a Flex client application that reads them and displays the data contained in the message on a UI. I’m running ejabberd in between the two.
When I crank up the source, which produces around 14500 messages in the space of a few seconds, the Flex application seems to lose data because it’s not fast enough to consume the messages at the rate they’re being produced. Throttling the source so that there’s a 50ms delay between messages seems to be sufficient for data loss not to occur.
To investigate a bit further, I tried writing the smallest possible client: one that would connect, and then read from the connection and just increment a count of the number of messages received. Using the minimal version, I can crank up the source with no throttling, and the client can keep up, receiving a message event for every message that’s sent. However, I tried an alternative, which was to disconnect the client, and then connect the source to ejabberd, resulting in 14500 offline messages being saved for the receiving user id. Connecting the client at that point results in about a 14% message loss - it actually counts 12500-13000 message events.
So the problem definitely seems to be related to the rate at which the consumer can process the incoming data (the ‘drinking from a fire hose’ problem, as someone here called it).
Unfortunately I’m pretty new to ActionScript and Flex, so I’m not quite sure where to go next to solve this problem, but if I find out anything else I’ll post it here…
This may or may not be particularly helpful but I have run into a similar problem. I discovered that what appears to happen is only a certain number of bytes are read from the xml stream at a time and so the XIFF library bSocketReceiveData function gets triggered with only partially complete xml (in cases where the number of bytes read ends mid xml stanza). In all cases I have come across, the bSocketReceiveData method is called with the next segment of xml (i.e. none of the xml is dropped by the socket), however by this point it is too late because the XIFF library has already choked on the invalid partial xml and thus, all messages/iqs/whatever that existed in that portion of xml are ignored. What is worse, no error handling occurs here so no error event can be thrown.
I have not yet decided how I am going to resolve this issue but when I do, I will post my solution…
At least in my case, my previous post contained the correct analysis of the problem. Since the new AS 3.0 binary socket doesn’t have a concept of “xml” or any specific type of data, it doesn’t send progress events only when complete xml stanzas have been loaded. As a result, the bSocketReceiveData method in the XMPPSocketConnection class of the XIFF library occasionally gets called with an xml string that is truncated mid-stanza. I resolved the issue by adding to that method some validation - I essentially parse out all the complete stanzas and then store the remaining incomplete stanza until the next progress event in which the second half of the previously incomplete stanza is sent. At that point I concat the strings to form a complete xml stanza and then pass it along to the correct processing method.
This works flawlessly for me so far and resolves all issues I had with packets being “dropped” - note that my volume is nowhere near as high as that mentioned in the earlier posts in this thread but my suspicion is that this solution will work in those cases as well as there is no other logical reason I can think of why data would simply be lost.
Just a heads up for anyone who’s still having problems:
I thought I was experiencing this same problem with the latest trunk XIFF, using XMPPSocketConnection, but it turned out that I just had to get rid of the “flash” parameter that I was passing to XMPSocketConnection#connect. I’m pretty sure the “stream:flash” stream type that this parameter enables contains some XMLSocket-specific hackery (interspersing null characters between stanzas?).