We upgraded our server to 4.7.1 recently and thought it would be nice to build some resiliency to the system so we installed Hazelcast. Now, it seems that pade also supports clustering since 1.6.1 but we can’t seem to make it work.
Our setup is two centos nodes connected to a galera database cluster. Client connections to the nodes come via a load balancer. Essentially you have a Front End Load Balancer → Two Openfire Nodes → DB Load Balancer → Galera Cluster.
Pade on both nodes seems to be up and running. Both videobridges appear to be up and you have focus on the senior node also up.
For clarity lets say that node01 has focus and jvb up and node02 has only jvb up.
What we are seeing is that conferencing is possible only when the clients are directed to the node that has jicofo running (in our example node01). As long as clients are directed to node01 they can join and create conferences just fine.
If a client however gets directed to node02 which only has videobridge running the connection is rejected. Also that client cannot join a room that has been created at node01.
The questions we have are the following:
Should it not be able for clients to connect and create conferences regardless of the node the load balancer directs them to?
Should it not be able for clients to join existing conferences regardless of the node the load balancer directs them to?
How can we load balance media streams? From what I understand the only way this would work is if we could force tcp media streams and make sure there are sticky sessions to the load balancer. But how can we do this (force media streams)? Can we disable udp media streams completely and force clients to a specific tcp port? How would the clients know where to connect in this case?
Are there other ports required to be open between the two nodes besides the one used by Hazelcast for this setup to work? We have selinux on both nodes and have ports tcp 5701, 5222, 7443 and udp 10000 open. Do we need any extra ports?
Let me know if any further clarification is required for our setup.
Thank you for the feedback on trying to use pade with clustering.
Should it not be able for clients to connect and create conferences regardless of the node the load balancer directs them to?
Yes, that should be the case. However, my testing was limited. You might be encountering a new issue. Check your Openfire MUC permissions and confirm that normal group-chat occurs in the sames rooms ok.
Should it not be able for clients to join existing conferences regardless of the node the load balancer directs them to?
Yes, that is what should happen.
How can we load balance media streams?
That is what the focus does by making sure that all participants of a meeting are allocated the same videobridge. Jitsi also does geographical allocation, but Pade is not yet supporting that. I have a bit of homework to do for that.
Are there other ports required to be open between the two nodes besides the one used by Hazelcast for this setup to work?
No. Just the usual required 7443 for Bosh/webSockets and 10000 for WebRTC
Enable openfire debug log mode on both nodes and have a look at the log files to spot any errors that might give an insight on why node02 is rejecting JVB connections. Confirm XMPP connections are working fine on node02 and MUC group-chat can occur in the same rooms that Jitsi is trying to use.
Hi Dele,
Thank you for taking the time to respond to this.
We have made some progress here and it seems that from what we can see clients connecting to node02 can join meetings sourced at node01. Regardless of where the connection is attempted users can create conference rooms etc.
In other words everything works fine if the clients go directly to any node and the load balancer (nginx) in this case is circumvented.
If however you put a loadbalancer in the equation the whole set up falls apart. We are forwarding port 443 to 7443 for both nodes and are also load balancing udp 10000. However from what we can see it does not get to the udp protocol ever. I have attached screenshots from our network configuration page maybe this will help?
Please note that ip address mapping we see that everything is blank. If I am not mistaken in our previous single-node setup we could see the local ip of the node as well as the public ip there on the fields. Should they be showing up maybe?
If so which one of the nodes would you have in local address?
Thanks again.
I don’t use a load balancer in front of Openfire. You would need some one with better network experience than me to help you with this. Try posting this as issue on Github and request for the attention of gjaekel (Guido Jäkel) · GitHub
If both your nodes have public internet static IP addresses, you can leave this blank. However if they are behind on a network with NAT, you have to use the openfire.xml to set their private and public IP addresses because Openfire DB will store the same value for both. See:
thank you very much for this.
Just wanted to let you know that when we define the network interfaces using the openfire.xml file then neither the video bridges nor jicofo comes up at all.
I will leave as is and will try to contact Guido as you suggested.
I will post back if we manage to get to the bottom of this.
Thanks again.
