Hi,
I want to use the in wildfire implemented JEP-0138, with a new (self-developed) compression method.
I don’'t want to edit the source code, so is it possible to add this compression method using a plugin?
Hi Tim,
There’'s no need to, Wildfire already implements it - JM-333.
Cheers,
Ryan
I think Tim is referring to adding a compression algorithm of his own making.
Yes, I want to add my own compression algorithm. JM-333 only implements zlib.
Hi Tim (Guus),
Sorry for the confusion before.
Unfortunately, writing a plugin (that uses a PacketInterceptor) to un/compress the packet won’‘t work since Wildfire won’‘t be able to transform the packet before it reaches the interceptor, and Wildfire does not provide a mechanism to add other stream compression methods to its JEP-0138 implementation. Looks like Gato did the work on JM-333 so maybe he could provide some insight into what it would take to make Wildfire’'s compression methods “plugable”.
Hope that helps,
Ryan
Hi Tim,
it’'s interesting:
Resolution Date: 12/14/05 of JM-333 …
and if we look at http://www.jabber.org/registrar/compress.html
2005-12-14 Added LZW (DCLZ) method. (psa)
So the JEP was implemented in Wildfire when LZW was added to it as an optional compression method. I assume that you want to implement LZW and it would be really fine if you would modify the existing core code instead of using a plugin and contribute the changes to Gato (dombiak_gaston).
LG
Hi LG,
I’‘d be curious to know if Tim really does want to implement LZW or if it is completely custom compression metho. I was thinking if it’‘s the latter that’'s where it would be nice if Wildfire supported plugable compression methods since JEP-0138 supports it.
Cheers!
Ryan
Hi Ryan,
Indeed I want to add a simple self-made compression method, which replaces , and so on. Then I want to modify a client at mobile phone, to use the shorter XML-tags. The advantage of this compression is, that it doesn’'t need a lot of CPU-power.
thanks in advance!
Hi Tim,
You shouldn’‘t have called that a compression because compression conveys a different meaning altogether. You have made everybody here confused. That is more like a shortcut to me. Shortcut, like you mentioned, severely breaks the XMPP spec and I wouldn’‘t recommend it. If you see an unresistable advantage of using shortcut, I think what you can do is to develop a component that provides the “gateway” between Wildfire and your mobile clients, like what’'s been done for Yahoo, MSN etc. And never modify Wildfire source code for such purpose.
Adding:
Looks like I’‘m the only one who’'s confused here - overlooking that the LZW in the JEP is actually an option. Sorry =/
Message was edited by: aznidin
Although a very simple one, (but as he stated, that could be an advantage) a dictionary coder with a static dictionary is a perfectly valid compression algorithm, as far as I see. If you implement it as a compression method as described in JEP-0138, the original stream would be reconstructed and would be valid XMPP again, so breaking XMPP isn’‘t an issue. His algorithm won’‘t be listed as a valid algorithm in JEP-0138, but there’'s no ‘‘MUST’’ dependency on registered algorithms, if I read the specs correctly.
The real issue here is how to hook a plugin this early in the processing of incoming streams, as far as I see.
What about going the binary XML route instead? That would be a standard-route.
Hi Guus,
I can see your point there. However, rather than hooking a plugin - can’‘t think of how without modifying the source - that early in the processing of incoming streams, developing a (mediation) component seems to be a more viable option. It’'s modular, isolated and the implementation would not be affected when he wants to upgrade Wildfire.
Binary XML is not a standard route to compress XMPP streams, JEP 138 is.
Notice that ZLib probably does a much better job than the static dictionary of Tim’'s scheme (which can indeed be implemented as a profile of JEP-138), but CPU is the main issue for him. It might get a bit tricky due to incomplete XML stanzas being compressed, but a byte-by-byte parser integrated with the compression algorithm can solve this.
Hi Tim,
zip does also not use much CPU power. Depending on the content of the packet a zipped packet may be smaller or larger than one where you replace “presence” with “p”. If you use two characters like “pr” then a zipped packet will likely be smaller and thus faster and cheaper.
I didn’'t look at HTTP-Binding but this could be an option for clients especially if the server sends and accepts compressed HTTP content.
LG
Thanks for all advices!
But after all I think using the standard protocol JEP-0138, is the best way to use my compression method. Doesn’'t anyone now how to use my own compression using a plugin in wildfire?
Tim,
I’‘m assuming that you actually tested to make sure that zlib wouldn’‘t work due to CPU usage? Your best bet is to dive into the compression code. It’'s probably not setup to be pluggable right now. You could make changes to make it pluggable, submit that as a patch, then build your own logic as a plugin.
Regards,
Matt
Tim,
It is now sometime later, but did you have any joy?
I understand where you are coming from, but for those that don’'t: implementing zip on javame adds ~60K of bytecode (cf. 40KB for parser, 20KB for comms and state machine, and maybe < 40KB for your UI); and if you need to decompress the stanza before you decode it you need to allocate memory for the whole XML fragment, versus only allocating as much as you need as you go via a SAX or StAX parser - which when you may only have 128K heap, is a big deal…
So from a device point of view using a event parser is best, but you then you have to pay for the bandwidth…
Personally i had thought a JSON (http://json.org) encoding would have been a good option for encoding for mobile; and perhaps still use a ‘‘tiny’’ profile like you are/were seeking to do (like the SVG tiny profile).
So, any progress?
– derek