Hi Folks. I’‘m currently building a rather unusual IM application that requires the ability to obtain presence information on, and send messages to, hundreds of thousands of users, and being an XMPP newbie I’‘m at a loss as to how to implement this. Our company, VoodooVox, Inc., created a web service call MyVox ( www.myvox.com ) which provides free Flash-based widgets for community web pages. We’‘re creating a new widget which lets web visitors see if the widget owner is currently “on-line” and send the owner an IM. When someone creates one of these widgets for their web page they will give us their IM username and network (Jabber, AIM, MSN, etc) and we will show their “presence” status to visitors to their web page and let them send them IM messages without exposing the owner’'s IM info or requiring that the visitor have an IM account.
I’‘ve been playing with the OpenFire server and the Smack API and have been able to successfully create a test OpenFire account, associate an AIM account with it (I’‘ve installed the IM Gateway), add other AIM users to my roster, see their presence, and send them messages … all good so far :-). The problem I’‘m facing is how to scale this little hack up to a production-level service to handle our current user base, which is over 250,000 and growing. The only way I can see to get presence information on a specific IM account, knowing only their username and legacy network, is to add them to the roster of a Jabber user account that we own. The ideal solution would be to dynamically create one Jabber account per widget when a new widget is created, add the widget owner’‘s IM account to the widget’'s IM roster, and make a connection to the widget IM acount whenever the widget is active on a page (these Flash widgets talk to our web servers via XML messages, so this would be relatively easy to implement).
Unfortunately it seems that in order to be able to talk to a user on a legacy IM network, or see their presence status, these Jabber accounts have to also have their own legacy account associated with it, and those can not be created dynamically (ie: at the time a MyVox user creates a new widget). The only way I can see around this is to pre-register a relatively small number of Jabber accounts, each with their own legacy accounts associated with them, and multi-plex them as needed for all these hundreds of thousands of widgets. The problem with this approach is that to get presence information these accounts would end up with tens of thousands of users on their rosters, which is obviously not a typical user case. Is there a practical limit on roster size? Is there a more elegant XMPP solution I’'m missing here?
I’'m not sure if you did take a look at the existing Presence Plugin which is as far as I know only for XMPP and allows to expose this information via HTTP.
It could be more easy to write a plugin for Openfire which is only accessible by your local servers via HTTP, maybe as a web service which allows your servers to query presence information of all users in Openfire.
As you want also be able to send messages to random users you may want to create another plugin which allows your servers to do this.
Openfire plugins run within the same JVM as Openfire and have full access to all data and the database. So you don’'t need Smack and rosters if you are running the Openfire server.
Is there a practical limit on Roster size?
I guess there is, a a long roster takes much longer to load than an empty one. Even if you are in the same LAN the server needs to create the roster and your client has to receive the roster as XML and parse the packet. This may take a lot of memory both on client and server side.
I did install the Presence Plug-in, but I think you’‘re correct that it only exposes the presence of XMPP users on the OpenFire server. Unfortunately for our application the majority of our widget users will already have an IM account on one of the “legacy IM networks”, and as far as I can tell, the only way to detect their presence is by adding them to the roster of an XMPP account which also has an account on the same legacy network registered with it. So, for example, if I want to see the presence of an AIM user ‘‘FOO’’ using my local XMPP account ‘‘BAR’’, I would have to first register my own AIM account with my XMPP ‘‘BAR’’ account and then add AIM user ‘‘FOO’’ to my ‘‘BAR’’ XMPP roster. In standard IM client applications this isn’‘t a problem, but attempting to do this from an automated web service seems to be impossible, since I don’‘t think I can auto-generate AIM (or MSN, YAHOO, etc) accounts programmatically. If I were to restrict our IM and presence to strictly XMPP users, then I could auto-generate an XMPP account for each widget, add the widget owner’‘s XMPP username to the widget’‘s XMPP roster, and then I’‘d be able to reflect the widget owner’'s presence on the widget. The problem seems to be dealing with legacy IM accounts in an automated, programmatic way.
What you said about roster length makes perfect sense, and further supports my hunch that huge rosters would be an unworkable solution. So it appears that on one hand I can’‘t dynamically create XMPP accounts that can communicate with the legacy IM networks, and on the other hand I can’‘t use a small number of pre-created XMPP accounts with legacy IM support because of the roster size limitations. I think this means I’‘m screwed ;-). If there are other possible solutions anyone can think of I’'d sure like to hear about them ;-).