I agree with your assessments.
1 (profiling and stress testing)
JMeter seems like a good foundation for building out a tool. I created a similar setup for testing with NNTP and once you get beyond the initial pain of learning the framework it can go pretty quickly.
One thing to consider is a partnership with Winfessor to build the tool - possiblying building on their existing one. They use .NET and I’‘m not sure how many Messenger people own visual studio .NET (I don’‘t). But maybe we could port their existing stuff and expand from there? Or request tweaks to Winfessor’‘s tool that would allow us to use it as-is but coordinate distributed tests using some glue code whether .NET, perl or java or whatever. In any case, testing tools is something that can benefit both of us and the Winfessor guys are very cool about sharing technology and ideas… And neither of us is focused on testing itself - it’'s just an ends to a means so anything that helps us get better tools would probably be interesting to everyone.
Agreed… Thinking about it, maybe we need to design the transport integration points in Messenger CM and Core rather than the actual protocol. That would let people try different protocol implementations quickly and in parallel. If someone comes up with a custom binary protocol, or wants to use straight XMPP XML, or wants some other scheme they can try it out in parallel.
That way we can explore as many solutions as possible and use the testing tools to determine the best solution. I imagine it will depend a lot on the specifics of the task at hand which could suggest we need to negotiate protocols or at least provide for a few reasonable options. I also have a hope that such a design would encourage academic research using the Messenger codebase which could result in really innovative technology in the future.
I prefer to see design by indivdual or very small group rather than design by committee. If we could create a pluggable transport to sidestep issue #2 I think hacking out a fast first prototype asap is the best way to proceed. Keep it fast and simple and be prepared for refactoring and rewriting and get something to start with seems like the best strategy. Motion in any direction is better than no motion at all (and all that jazz).
Since you’‘re ready to go i’‘d say you should just code a quick and dirty first crack 0.0.0.1 and check it in. Use anything for the CM <–> Core communication… I’‘d suggest just using RMI for now. That’‘s probably the quickest and dirtiest first pass - our packets implement Serializable don’‘t they? The strawman should give us something to at least talk against, figure out where the real protocol integration points can be, etc. It would also give us something to test the testing tools agaist (comparing this strawman against standard Messenger). By the time we get the build system, testing, and basic design idea done from this, we’'ll be at a point where we can refactor the strawman, know something about the tradeoffs with actual protocols, and be in a good spot to take a crack at pampero 1.0… IMO