JEP-0138 Support

I am assuming that the Stream Compression JEP-0138 is not currently integrated into any build of the server. Does anyone have any plans on supporting this JEP in the Jive server? I am thinking of adding it myself, but if someone is already doing the implementation I would hold off.

Stephen,

That’‘s correct – stream compression is not currently implemented. There is some work being done on stream feature negotiation so we’‘d need to integrate stream compression into that. However, we’‘d be more than happy for you to take on overall leadership on implementing compression. Should I create an issue in JIRA and note that you’'ll be working on this feature?

Regards,

Matt

Yes please do. I am not familiar with the Jive development process or codebase so it may take some time to get ramped up.

Actually it turned out to be easy to add, so I just did it. It seems to work with my client implementation, but before I submit it I would like to do some more testing. If anyone would like to test it out against their client implementation, send me a PM and I will send you the changes.

Nice. BTW, I created JM-333 for this issue. Please let me know when you’'re ready to have your changes reviewed.

-Matt

After some more testing I realized the initial approach may not work. The original flow is like this:

  • After the ");

  • In ClientSocketReader.java processUnknownPacket replace the socket with a ZipSocket

  • In ClientSocketReader.java processUnknownPacket call createSession(“jabber:client”); to create a new session

This works fine if the client is using a ZipSocket on the other end, but if they are using a regular socket transmitting zlib compressed data it won’'t work because the two are not (presumably) interoperable.

Another approach would be to create a CompressedSocket class that calls zlib to compress/uncompress the data as it is send/received. The question I have is what is the best way to do this? Where in the code would I need to create the CompressedSocket in lieu of the regular Socket that is created now?

Or is there a better way of doing this? Ideally there would be a single place where the client data is received and another single place where the client data is transmitted, and I could just add code to those methods to check to see if compression is enabled. If it is I would just call zlib to perform the operations and it would be transparent to the upper layers of the code.

I would like to use a method that touches the least amount of Jive Messenger code

Pending response to last question.

Hi Stephen,

Have read the JEP and yours and Matts comments. Given that feature negotiation does not seem to be fully supported in JM core, it would be very tricky for you to add this in “the best possible way”.

The key part of the specs the change from uncompressed to compressed: after you acknowledge with it states

“Both entities MUST now consider the previous stream to be null and void, just as with TLS negotiation and SASL negotiation (as specified in RFC 3920). Therefore the initiating entity MUST initiate a new stream to the receiving entity:”

Based on my reading (and this is of course subject to scrutiny:) it does not require you to replace the socket at all. It does require you to reset/ignore the previous stream. In JM this would be a new session (which is as you mention).

I would suggest creating an io compression decorator

CompressedInputStreamReader

CompressedOutStreamWriter

the reader (XPPPacketReader) input stream can then be replaced with the decorated input stream. And similar for the writer.

These classes can then be integrated when the core feature negotation is finalised.

I think thats how i would try it.

BTW as far I can tell java.util.zip package is zlib compatible, and there is also a pure java zlib alternative implementation (zlib web site)

Hope that gives you some more inspiration!

Conor.

Great suggestion. What I meant by “replacing” the socket was replacing the input and output socket streams with zipped socket streams. I didnt mean closing the socket and reopening it.

The way it is done now is:

  1. Compression is negotiated

  2. The client/server sockets’’ input and output streams are replaced with ZippedInputStream and ZippedOutputStream

  3. New session is created

I will take a look at the XPPPPacketReader and adding the compression decorators.

Where does this effort stand?

It has been implemented as above, however the feature negotiation code needs to be implemented correctly in a Jive view before it can be merged in.

Just to update everyone: this implementation is close to complete with the new codebase.

Let me be the first to say: w00t!

Do you have any numbers on how effective the compression is?

I don’'t have any data on that. I have run into a snag though. In the latest code base there is all this TLS stuff that is complicating things.

I was planning on using ZipInputStream for the compression, but now there is this TLSStreamReader class that I don’'t know what to do with.

Has anyone use the ZipInputStream class and know if using that class will work for this? I am no expert in Java. One idea is to do this:

private void saslSuccessful() throws XmlPullParserException, IOException {

XmlPullParser xpp = reader.getXPPParser();

// Reset the parser since a new stream header has been sent from the client

if (connection.getTLSStreamHandler() == null && connection.isCompressed ())

{

xpp.setInput(new InputStreamReader(new ZipInputStream (socket.getInputStream()), CHARSET));

}

else if (connection.isCompressed ())

{

xpp.setInput(new InputStreamReader(new ZipInputStream (connection.getTLSStreamHandler().getInputStream()),

CHARSET));

}

else if (connection.getTLSStreamHandler() == null)

{

xpp.setInput(socket.getInputStream(), CHARSET));

}

else {

xpp.setInput(new InputStreamReader(connection.getTLSStreamHandler().getInputStream(),

CHARSET));

}

It looks awkward to me.

Also, I think I fixed a bug in the SASL negotiation. It looks like the SASL negotiation code has the server resending the stream and then reading the stream:stream from the client. The server shouldn’'t send out the stream initiation until the client has sent the stream:stream element, as seen here in the RFC:

C: <?xml version=''1.0''?>

<stream:stream

to=’‘example.com’’

xmlns=’‘jabber:client’’

      xmlns:stream=''http://etherx.jabber.org/streams''

version=’‘1.0’’>

S: <?xml version=''1.0''?>

<stream:stream

from=’‘example.com’’

id=’‘someid’’

xmlns=’‘jabber:client’’

      xmlns:stream=''http://etherx.jabber.org/streams''

version=’‘1.0’’>

The affected code is in:

SocketReader::saslSuccessful()

OK, this effort is complete. The negotiation code is complete, and also the compression code is implemented using the ZipInputStream. Hopefully the ZipInputStream is compatible with most zlib based client implementations.

I do not know the proper procedure to merge the code in, but until I find that out if you are interested in getting the modified files please send me a private message and I will send you a zip file of the changed code files.

Hi Stephen, that’'s good to hear. To get your code merged with JiveMessenger, you can send your code to Matt, I guess. His email address is in his profile.

If you could send me your code too, that’‘d be great. I’‘d love to test stuff out immediately, but I’‘m really in the middle of my graduation at the moment. So, although I’‘m interested, it’'s going to take me a while to respond.

Hey guys,

Stephen’‘s contribution has been merged into the codebase. It will be available in the next nightly build. However, since I don’'t have any client that supports compression I was not able to test the new functionality. But all regression tests passed just fine.

Let me know if you were able to use/test the new compression feature.

Thanks,

– Gato

One thing I forgot to mention is that in order to activate compression support you need to add the following system property: xmpp.client.compression.policy and set it to “optional”.

By default the compression feature is disabled and will not be offered to connecting entities in the element.

Any idea if or when Spark will support compression?