Broadcasting messages to Yahoo/issues

Openfire Plugins Kraken - IM Gateway
#1

I posted about this on the main openfire discussion area several weeks ago but it probably is better here, plus I have more details. To make a long story short, we use openfire and the IM gateway to connect to Yahoo from our own proprietary system (not using spark client, in other words). We need the ability to implement a broadcast mode, similar to what both Yahoo’s own client and the spark client can do, so that users can send a message to a number of users (mostly in the 40-50 range) that are logged into Yahoo as a broadcast.

Here is the problem. When I try to send to a group of 30 from our own proprietary system to yahoo, via Openfire and IM gateway, we see only a limited number of users getting the messages.

When I tried this using the spark clients broadcast mode, connected into our openfire server, and send to yahoo buddies I see exactly the same thing, and the magic number appears to be 19. The first 19 messages that are sent get to the destination buddy fine, but after that nothing gets out (I verified the order in the spark debug log and in our own servers logs, to make sure). This seems to indicate there is some sort of rate limiting on Yahoo, that is dropping the messages after 19 because some limit has been broached.

However, if I do the same test from the Yahoo client, using their ‘send instant message to all in folder’, it seems to work fine (not surprisingly). My suspicion is that the yahoo client signs in with some sort of identifier that turns off rate limiting, something that the IM gateway does not/cannot do, but I don’t know.

Any ideas?

Thanks!

#2

We did find a workaround with this (Yahoo definitely has a broadcast mode, but at the moment we don’t know exactly how we could implement it).

The basic problem with Yahoo and doing broadcast is rate limiting throttling; in our testing we found that broadcasting to more then 19 users would see anything above 19 dropped.

What we did was a couple of things. We first of all broke up broadcasts into groups of 15 (yahoo in their broadcast mode appears to use 10), and then we also put in throttling to stop putting out too many messages in a given time (basically, putting a delay between messages, a variable one based on how many messages we had sent). In doing that, we have been able to get it to work. I am not posting the actual delay algorithm because I suspect it probably would not work for other systems with their own unique propogation delays and such, it takes some trial and error to find the right way to do it. In theory, you could simply do a strict delay between groups of messages, but making it variable means that in the end it will probably be more efficient and also won’t run into trouble if the number of people being sent to grows large.