Last Friday 06/05/2026, I updated to 5.1. afterwards everything seemed to be working fine, so I left for the weekend. I get to work today and Openfire is down. I login to the VPS with Putty and restart the server first. After the restart, Openfire ran for a minute, and then it was down again. I restarted the service and it came up for a couple of minutes again and crashed.
Then I noticed the Java Memory, which I have configured for 1024MB was at 99%, and at that moment, Spark lost connection, and the Admin Console stopped working againā¦
Anyone have any ideas? I am not sure what to even look for in the logs, or where the logs are stored for that matterā¦
That goes on for THOUSANDS of Lines for that one user after serviceID=. If I restart the server, it does the same thing, but it might be another user in the serviceID area. But this goes on for the entire log
Iām sorry to hear this. Would it be possible for you to private a heap dump (maybe in private) of an Openfire instance that is close to crashing, so that we can analyze whatās taking up all that memory?
My VPS is an older version of CentOS 8. My host now offers Ubuntu 22.04 or Debian 12.0. What would I need to save from the current CentOS installation, in order to move it all to a new Ubuntu/Debian install?
I use MySQL, so I can copy the database easy enough. Any of the /opt/openfire folders that would need to go as well?
Pretty much all you need is in the /opt/openfire/ directory (and the database). As Openfire is a Java-based application, itās pretty OS-agnostic. Copying over the entire /opt/openfire/ folder to the new system (and manually making sure that your start/stop scripts work - or alternatively, install the same version of Openfire on the new OS, then overwrite it with /opt/openfire from your backup) should get you pretty far.
Obviously, ensure that you have backups for your entire old system, so that you can restore things just in case the migration doesnāt work out.
We recently upgraded to 5.1.0 and noticed the same kind of crashing.
We are on openSUSE 16.0 and were running Java 17.x, but also tried upgrading to Java 25 to see if it corrected (did not help). We auth against LDAP and hook to mysql/maria. We are using plugins: Bookmarks, Broadcast, Certficate Manager, Client Control, Monitoring Service, Pade, Search, and User Status Plugin. We have around 100 users. Initially the crashing took several hours (also over the weekend), but after restarting openfire we would notice high java memory usage almost right away.
For reference, I manage another similar setup (but without LDAP, Pade, and a smaller user-base of around 10 users). It is not currently experiencing this issue.
@guus Can you point me in the direction to capture a heap dump? If I know what you are wanting, maybe I can temporarily upgrade my environment and hope for a crash.
Right now what I have is the attached log, does it tell you anything helpful?
As far as my instance goes, I did a database dump. Did a clean install of Debian 12. Setup Java(Java Version: 21.0.11 Oracle Corporation ā Java HotSpotā¢ 64-Bit Server VM), and installed Openfire 5.1. Then once that was all done and up and running, and setup the Registration plugin and sent out an email to everyone to sign back up. Was easier to go clean, assign users to their groups again, than to figure out a way to get a MySql 5.5 database to āJust workā in MariaDB 15.1, which came preinstalled with Debian 12.
Nearly all of the users are added back in and assigned to their respective groups, and all of the Group Chatrooms are now active. Been running for around 6 hours with 74 users logged in and sitting at about ā64.15 MB of 512.00 MB (12.5%) usedā of Java Memory. Weāll see how this goes after a few days.
Iām having the same problem. I updated to 5.1.0 along with updating my Debian 12 packages. The OpenFire server ate up all my RAM and used 100% of my CPU. The last thing I see before it reboots is this log output: java.lang.OutOfMemoryError: Java heap space.
I tried increasing the minimum and maximum memory when starting the server, but it still consumes all of it.
I created a dump as recommended online, but since Iām not an expert in debugging, I canāt understand anything.
My guess is that the loading of MUC history at startup is to blame here. Can you tell how large some of your database relations are? namely ofmucconversationlog
Thanks everyone for providing these details. Most of it points to an unexpectedly high number of pubsub subscriptions.
I wonder why these have been created. Have people that suffer from this issue a configuration where, through shared groups, everyone is on everyoneās elseās contact list?
Are there duplicate subscriptions in the database perhaps? Could you try this query?
SELECT
serviceid, nodeid, owner, jid, COUNT(*) AS count
FROM
ofPubSubSubscription
GROUP BY
serviceid, nodeid, owner, jid
ORDER BY
count DESC, serviceid
@joelgb The repeated org.jrobin.core.RrdException: Datasource name [ratelimit.newconnections.socket_s2s.accepted] to long (44 chars found, only 20 allowed error in your logfile is unrelated. It is an indication that you need to update the Monitoring Service plugin.
In LDAP, all of our users allowed to use openfire are in a specific group (done to filter out āserviceā type of LDAP users). In the openfire web console we went to Users/Groups ā Edit Group ā (edit the group) and set Contact List (Roster) Sharing. First āGroup Nameā is our LDAP group. āEnter contact list group nameā is an āAll usersā name representing our orgās name.
So in everyoneās Spark client, they have a group with all IM users and we only have to manage this by adding/removing people from the LDAP group (maybe a restart of openfire too). The āAll usersā group is needed so people can āfindā each easily. Weāve had this setup since we started using openfire.
in case it matters: I have ofPubsubSubscription. I have 7994 records.The record with the highest ācountā of 4075. looks like this:
In the picture, serviceid and nodeid are the same values/user for all pictured records.
The ofMucConversationLog.ibd file that was in my instance, was 132MB. The ofPubsubSubscription.ibd was 884MB. It was on an instance that was started in 2019, and only had this problem after upgrading to 5.1. There was another update a while back that caused an issue, but increasing the Java Memory to 1024MB from 512MB fixed that.
In our deployment, we have 5 different locations that need to communicate, so I created 5 groups, and enabled āContact list group sharingā for each of the groups so that everyone can see everyone else. And then each user is assigned to their respective groups so that everyone sees which location that they are in. It has been this way for years.
It was only when I updated to 5.1 that the issue arose. I went from 5.0.5 to 5.1. 5.0.4 ran like a champ. Then I updated to 5.0.5 and it was fine for a few days until 5.1 was released and I updated again. Then the crashing started.
I am reasonably confident that the out-of-memory problem occurs because Openfire is trying to load to many āpubsub subscriptionā entities from the database.
We can clearly see in @Govorun 's query output that the ofPubsubSubscription table has an unreasonable amount of entries. That is very likely to be part of the problem.
The logs (on level TRACE) that @Brandon20 shared show that (presumably on startup) a lot of new subscriptions are being created - which is highly suspicious.
Those two clearly relate to a similar issue. Iāve also received a heap dump, that again shows a lot of āpubsub subscriptionā instances (org.jivesoftware.openfire.pubsub.NodeSubscription) living in memory. That too points to the same source for this issue.
Sadly, Iāve not found a way to reproduce this problem yet. Iāve tried a local setup with and without LDAP, with and without shared groups - so far, I didnāt have any luck.
From Brandonās log lines, I assume that upon server start with 5.1.0, the amount of rows in the ofPubsubSubscription table is growing. Can someone check if that happens every time the server is (re)started, and if the same growth is visible with Openfire 5.0.5?
Does anyone that is now affected by this problem have a database backup available? It would be useful to know if earlier versions of Openfire also had these amount of database rows. Even though thatās certainly not correct, thereās a chance that in older versions, all of those duplicates maybe didnāt get loaded, preventing the out-of-memory issue from occurring?
