Looking for the best way to broadcast IQs

Openfire Openfire Dev
1

Hi all,

I’m creating an internal Component for handling location-based updates.

Briefly: users would have the capability to send an “update” IQ to the component, which in turn would broadcast the IQ to “some” of the users.

As far as I understand, I can’t use a system like pubsub, because we talk about conditional dispatch (say, based on the user’s location…)

I’m currently thinking of implementing this in a similar fashion as in Jive’s BroadcastPlugin:

Sending a packet to each of the relevant users, in public void processPacket(Packet packet)

My question is: what would happen if the number of target users is high?

I would prefer to avoid “blocking my component” for too long and I wonder if there is another mechanism, e.g. like a queue that could be filled with IQs, without the need to wait for all the them to be sent.

Or maybe that’s exactly what’s already going on behind the scenes?

Thanks in advance for any advice,

Ariel

2

Hi Ariel,

I lost my long previous reply so trying again…

I have a few questions and answers to you.

Why are you thinking of using IQ packets for notifying users? Do you need some ACK from them?

Why is pubsub not good for your needs? What do you mean by conditional dispatch? Do you have an example of why pubsub is not good?

Which users do you need to notify? Would a solution based on roster (ie. presence subscription) would work for you? Users can include their coordinates in their presence packet as an extension and then when sending that presence to the server, the server will broadcast their presence including the coordinates info to users subscribed to their presences.

If you want to go with an entire custom solution then you can take care of scalability all by yourself. You can have users send a Presence packet to your component that will include their coordinate information. Your component can then decide the best strategy for notifying users. If the number of users is small or the total broadcast load is small then it can just broadcast the notifications and if the number is load is big then you can just queue the things and have a thread that will consume that queue and send the notifications. BTW, the notifications can be in the form of Presences since IQ packets expect a reply unless you need some kind of ACK as I asked before.

– Gato

3

Gaston,

Thank you for your detailed answer!

I guess the most helpful thing would be for me to explain the “big picture”…

I’m building a (mobile-based) system where users can buy and sell spots on a map.

A) As a seller, you can:

  1. Define a new spot to sell

  2. Define a price for the spot

  3. Define a duration time for the bid

Once the bid has started, time is starting to tick. If the duration of the bid is over, the spot is discarded.

During the bid, the seller can:

  1. Cancel the bid

  2. Change the price

  3. Change the duration

If someone buys the spot, the seller is receiving a notification, and his account is getting credited.

B) As a buyer, you can:

  1. Watch on the the map new spots being created or discarded

  2. See how the price is changing and how much time is left, for each spot.

  3. Buy a spot

  4. Chat with the seller

C) Facts to take in count

  1. There can be many potential buyers watching the map, and many real-time interactions (e.g. new spot created, spot-price changed, etc.)

  2. The map is dynamic: every buyer can watch a different area of the city on his map (think of an interactive Google Maps view…)

D) Requirements and considerations

  1. Sellers should only be notified of the real-time interactions taking place around the “area of the map” they are currently watching.

  2. When a seller is, for example, changing the price of a spot: the UI widget used for that purpose should be in a wait state until a response comes back from the server.

  3. In term of fairness, when someone buys a spot, he will always pay for the amount he saw on screen (which can be different than what the seller asked for, e.g. due to propagation latency…)

E) System in use

  1. I’m currently using the Bitnami TomcatStack (Apache + MySQL + Tomcat) running on Amazon EC2, with Openfire 3.7.1 installed on top.

  2. So far, it is not clear to me if regular Tomcat servlets and Openfire are running on the same Java VM at all.

I have tried to access XMPPServer from a servlet, but without success…

I should create another topic for that, but it is important to understand that I would prefer to avoid a situation where some of the user transactions take place as Tomcat based web-services interacting with the “application DB”, while some others take place as interactions with Openfire (which will probably have to access the “application DB” as well…)

F) Implementation strategies

  1. In order to solve issue D1, I’m planning to divide the earth into tiles. Each of those tiles correspond to a mutilple of Google Maps tiles at the maximum zoom level. In short, it means that I don’t need to transmit to the server (and/or the other users) the region covered by each user’s map every time he is interacting with it, but instead: only when a new tile is entered or leaved.

It implies to maintain on the server side a map of “active tiles”, containing reference to the “registered” buyers. Ideally, this map should be held in memory (i.e. no costly DB transactions nor disk-based persistence…)

a) So for instance, if a seller is creating a new spot: the information should be propagated only to the the sellers “registered” to the same spot’s tile.

b) Or when a seller is entering a new area of the map: he should be notified about the bids taking place in that area…

I don’t think that pubsub is enough to solve the problem here, mainly because of issue b: at any time, you should be able to select an arbitrary area on the map and retrieve the relevant information / state.

Therefore, my plan is an attempt to solve issues D1, D2 and E2 at the same time by:

  1. Using a custom Openfire component to centralize (almost) all of the communication with the users.

Obviously, the original “account creation” stage will have to take place as a Tomcat-based web-service, but from that point, I’m hoping to centralize everything via the the custom component.

The component will probably extend AbstractComponent, and it will listen to custom IQs, which look to me like perfect candidates for passing custom XML data, and for returning responses.

From what I understand from Gaston’s answer, I will have to use some thread-based mechanism when there is a need to broadcast IQs to many users at a time, in order to avoid the blocking incoming queue… I may try to look at the pubsub implementation code for ideas on how to implement such a system…

Regarding presence, I guess should use it less for passing custom data, but more for receiving automatic available/unavailable stanzas from the users. So far, I could not make it work from inside my custom component. That should be the topic of a whole new thread…

Phew, that’s it I guess!

Thanks for reading so far.

In some way, my proposition is like trading the old paradigma of communicating via http to a web-service, with something like: keeping sockets open and doing all the transactions (including DB access, etc.) via some custom XMPP-based setup.

Is it a risky approach? Am I reinventing the wheel or missing something obvious?

Feedback is, again, greatly appreciated!

Ariel