Based on the interest of a lot of community members, we’‘re launching an ambitious project to greatly improve the scalability of Jive Messenger. Our goal is to let Jive Messenger handle at least 100K connections. To do this, we’'re going to create an open protocol for a two-tier XMPP architecture – multiple “connection manager” servers will aggregate client connections into a core router server (standard Jive Messenger instance).
Major areas of development for this project:
Document protocol for two-tier communication (probably as an extension of JEP-0114).
Create high performance connection manager server implementation using Java NIO. The target is for each connection manager to handle 10K client connections.
Modify Jive Messenger to accept connection manager connections.
We’‘ll be deploying more information and resources in the near future. For the time being, please reply if you’'re interested in helping out with this project!
Huh, Pampero. A nasty wind BTW, i think it sounds better when you say “Pampero Project” (the way you wrote in this thread’‘s title) and not “Project Pampero”. It’‘s only an IMHO I’'m talking about Forum name
Thanks for the offer. Feel free to send me an email if you’‘d like to discuss any details. We’'re definitely looking for companies that contribute to the implementation or sponsor implementation.
I am definatly interested in the high-availability aspect this would provide. I do not have an unusually large number of users (250) but I am very interested in making the service fault-tolerant.
I am definatly interested in the high-availability
aspect this would provide. I do not have an
unusually large number of users (250) but I am very
interested in making the service fault-tolerant.
A question for you on this. Pampero and scalability we know we’‘ll be able to build out without 3rd party technology. However, clustering and fault-tolerance may require a 3rd party commercial component. That would mean we’‘d need to sell a clustering plugin. Is that something you’'d be willing to pay for as a value add-on on top of the Open Source version?
Thanks for the info. I guess I am still looking for an answer the underlying question of scalability of the wildfile server. I am hoping it will scale to 10k on a HP-UX machine with 4g of memory.
Out of the box, Wildfire should scale to 7-8k. I think this posted somewhere, but obviously your mileage my vary.
Is that HP-UX machine 64bit? I’'ve long believed that wildfire could scale to 10s of thousands of users if someone was willing to invest in the right hardware. For example, a Sun E1000 with 32 cpus and 512gb of memory running a single jvm, should provide more bandwidth for the single application.
In the grand scheme of things, Pampero will attempt to allow commodity hardware to be clustered together to achieve such scalablity. It’'s a few month away from a alpha. Keep watching this forum for details.
Thanks for the information. I understand that ‘‘mileage may vary’’ but 7-8k range is what I was looking for. Is there any published baseline tests for these number or others on the forums that may be able to share their scalability outcomes?
A question for you on this. Pampero and scalability
we know we’'ll be able to build out without 3rd party
technology. However, clustering and fault-tolerance
may require a 3rd party commercial component. That
would mean we’'d need to sell a clustering plugin. Is
that something you’'d be willing to pay for as a value
add-on on top of the Open Source version?
Regards,
Matt
Personally speaking, I feel that a true and reliable clustering solution should be something that is paid for. It is not really available anywhere else and it also allows for people to use the system without clustering/fault tolerance to its fullest to find out if it fits with their needs. If they want clustering and fault tolerance (by no means an easy thing to do) then yes they should be willing to pay for the solution or write their own. Most people will not need clustering or fault tolerance so for most people this is not really even an issue. They may want it just for the coolness factor but realistically they probably don’'t need it.
Personally speaking, I feel that a true and reliable
clustering solution should be something that is paid
for. It is not really available anywhere else and it
also allows for people to use the system without
clustering/fault tolerance to its fullest to find out
if it fits with their needs. If they want clustering
and fault tolerance (by no means an easy thing to do)
then yes they should be willing to pay for the
solution or write their own. Most people will not
need clustering or fault tolerance so for most
people this is not really even an issue. They may
want it just for the coolness factor but
realistically they probably don’'t need it.
I dont think that “true” high availability need be a commercial component. It is a complicated thing to implement, and as such I think a great benefit could come from sharing the implementation with the community. There are many smart people that can provide useful experience in setting up a HA product. Bugs get fixed very quickly when the person spotting the bug can submit a patch at the same time. For groups tight on time, this is a great feature of Open Source.
I think the model used for Wildfire currently is a great method, that real product support is an add-on service you pay for. But if you are smart, and willing to get your hands dirty, you can have a great product. I often think the cost is about the same either way, its just a difference between paying someone else or doing it yourself.