Monitoring Plugin (MAM) — doubled/quadrupled messages after relogin

Openfire Plugins
1

Hello.

I’m getting duplicate messages in 1:1 chats (but not in MUC rooms) after restarting a client or logging out and back in. It happens with both Gajim and Conversatons, so it is probably not client-specific. Sometimes it’s two of the same message, sometimes four.

The issue seems to be related with the monitoring plugin, as it stops if I remove the plugin. I don’t remember changing any settings for the monitoring plugin from default, apart from making sure that all the checkboxes are enabled.

I tried to look into the protocol log on the client, and the same message was indeed received four time from the server.

(user2@example.org sends two messages to user1@example.com: “320test” and “321test”. The former doubled, the latter quadrupled.)

Спойлер 
<!-- Incoming Thu 20 Jun 2024 01:19:45 PM +07 (user1@example.com) -->
<message xmlns="jabber:client" from="user2@example.org/gajim.XXXXXXXX" to="user1@example.com" xml:lang="en" type="chat" id="340df5e9-3ecf-48cd-8069-2c65202d60d9">
  <body>320test</body>
  <origin-id xmlns="urn:xmpp:sid:0" id="340df5e9-3ecf-48cd-8069-2c65202d60d9" />
  <request xmlns="urn:xmpp:receipts" />
  <active xmlns="http://jabber.org/protocol/chatstates" />
  <markable xmlns="urn:xmpp:chat-markers:0" />
</message>


<!-- Incoming Thu 20 Jun 2024 01:19:49 PM +07 (user1@example.com) -->
<message xmlns="jabber:client" from="user2@example.org/gajim.XXXXXXXX" to="user1@example.com" xml:lang="en" type="chat" id="bc6b3162-2971-422a-a50d-998fc42994de">
  <body>321test</body>
  <origin-id xmlns="urn:xmpp:sid:0" id="bc6b3162-2971-422a-a50d-998fc42994de" />
  <request xmlns="urn:xmpp:receipts" />
  <active xmlns="http://jabber.org/protocol/chatstates" />
  <markable xmlns="urn:xmpp:chat-markers:0" />
</message>


<!-- I relogined here -->


<!-- Outgoing Thu 20 Jun 2024 01:20:25 PM +07 (user1@example.com) -->
<iq xmlns="jabber:client" to="user1@example.com" type="set" id="0396c955-c892-4e34-af61-e096dda9054e">
  <query xmlns="urn:xmpp:mam:2" queryid="410d78b8-5190-4ad3-bd58-f1d85131c771">
    <x xmlns="jabber:x:data" type="submit">
      <field type="hidden" var="FORM_TYPE">
        <value>urn:xmpp:mam:2</value>
      </field>
    </x>
    <set xmlns="http://jabber.org/protocol/rsm">
      <max>70</max>
      <after>2040</after>
    </set>
  </query>
</iq>


<!-- Incoming Thu 20 Jun 2024 01:20:25 PM +07 (user1@example.com) -->
<message xmlns="jabber:client" from="user2@example.org/gajim.AJWQJWI3" to="user1@example.com" xml:lang="en" type="chat" id="bc6b3162-2971-422a-a50d-998fc42994de">
  <body>321test</body>
  <origin-id xmlns="urn:xmpp:sid:0" id="bc6b3162-2971-422a-a50d-998fc42994de" />
  <request xmlns="urn:xmpp:receipts" />
  <active xmlns="http://jabber.org/protocol/chatstates" />
  <markable xmlns="urn:xmpp:chat-markers:0" />
  <delay xmlns="urn:xmpp:delay" stamp="2024-06-20T06:19:49.306Z" from="example.com" />
</message>


<!-- Incoming Thu 20 Jun 2024 01:20:26 PM +07 (user1@example.com) -->
<message xmlns="jabber:client" to="user1@example.com/gajim.YYYYYYYY">
  <result xmlns="urn:xmpp:mam:2" queryid="410d78b8-5190-4ad3-bd58-f1d85131c771" id="2041">
    <forwarded xmlns="urn:xmpp:forward:0">
      <delay xmlns="urn:xmpp:delay" stamp="2024-06-20T06:19:46.016Z" />
      <message xmlns="jabber:client" from="user2@example.org/gajim.XXXXXXXX" to="user1@example.com" xml:lang="en" type="chat" id="340df5e9-3ecf-48cd-8069-2c65202d60d9">
        <body>320test</body>
        <origin-id xmlns="urn:xmpp:sid:0" id="340df5e9-3ecf-48cd-8069-2c65202d60d9" />
        <request xmlns="urn:xmpp:receipts" />
        <active xmlns="http://jabber.org/protocol/chatstates" />
        <markable xmlns="urn:xmpp:chat-markers:0" />
</message>
</forwarded>
</result>
</message>


<!-- Incoming Thu 20 Jun 2024 01:20:26 PM +07 (user1@example.com) -->
<message xmlns="jabber:client" to="user1@example.com/gajim.YYYYYYYY">
  <result xmlns="urn:xmpp:mam:2" queryid="410d78b8-5190-4ad3-bd58-f1d85131c771" id="2042">
    <forwarded xmlns="urn:xmpp:forward:0">
      <delay xmlns="urn:xmpp:delay" stamp="2024-06-20T06:19:49.306Z" />
      <message xmlns="jabber:client" from="user2@example.org/gajim.XXXXXXXX" to="user1@example.com" xml:lang="en" type="chat" id="bc6b3162-2971-422a-a50d-998fc42994de">
        <body>321test</body>
        <origin-id xmlns="urn:xmpp:sid:0" id="bc6b3162-2971-422a-a50d-998fc42994de" />
        <request xmlns="urn:xmpp:receipts" />
        <active xmlns="http://jabber.org/protocol/chatstates" />
        <markable xmlns="urn:xmpp:chat-markers:0" />
</message>
</forwarded>
</result>
</message>


<!-- Incoming Thu 20 Jun 2024 01:20:26 PM +07 (user1@example.com) -->
<message xmlns="jabber:client" to="user1@example.com/gajim.YYYYYYYY">
  <result xmlns="urn:xmpp:mam:2" queryid="410d78b8-5190-4ad3-bd58-f1d85131c771" id="2043">
    <forwarded xmlns="urn:xmpp:forward:0">
      <delay xmlns="urn:xmpp:delay" stamp="2024-06-20T06:20:02.950Z" />
      <message xmlns="jabber:client" from="user2@example.org/gajim.XXXXXXXX" to="user1@example.com" xml:lang="en" type="chat" id="bc6b3162-2971-422a-a50d-998fc42994de">
        <body>321test</body>
        <origin-id xmlns="urn:xmpp:sid:0" id="bc6b3162-2971-422a-a50d-998fc42994de" />
        <request xmlns="urn:xmpp:receipts" />
        <active xmlns="http://jabber.org/protocol/chatstates" />
        <markable xmlns="urn:xmpp:chat-markers:0" />
        <delay xmlns="urn:xmpp:delay" stamp="2024-06-20T06:19:49.306Z" from="example.com" />
</message>
</forwarded>
</result>
</message>


<!-- Incoming Thu 20 Jun 2024 01:20:26 PM +07 (user1@example.com) -->
<iq xmlns="jabber:client" type="result" id="0396c955-c892-4e34-af61-e096dda9054e" from="user1@example.com" to="user1@example.com/gajim.YYYYYYYY">
  <fin xmlns="urn:xmpp:mam:2" queryid="410d78b8-5190-4ad3-bd58-f1d85131c771" complete="true">
    <set xmlns="http://jabber.org/protocol/rsm">
      <first>2041</first>
      <last>2043</last>
      <count>220</count>
</set>
</fin>
</iq>

I have cut everything that didn’t seem related for reading convenience. I’m not fluent with XMPP, so I could miss something important. I can upload the full protocol log if required.

Is it maybe a normal server behavior and it’s client’s responsibility to check for duplicates? Or maybe client request was somehow incorrect? Or is it something that can be fixed on the server side?

2

bump

3

Upgraded to Openfire 4.9.0 and Monitoring Plugin 2.6.0. Behavior seems to be changed a bit, the first one or the first two or three messages don’t duplicate. It is not consistent.

Спойлер

image
image203×525 8.55 KB

No such problem with Ejabberd or Prosody with their respective MAM modules enabled. Could I have misconfigured Openfire somehow?

4

Thanks for reporting this. I do concur that it looks like a bug in our Monitoring plugin, but I don’t know what is causing it.

I created a new issue in the bugtracker for this, here: Duplicated one-on-one messages · Issue #391 · igniterealtime/openfire-monitoring-plugin · GitHub

1 Like
5

This issue occurs only in Gajim when used with Openfire’s MAM (Message Archive Management) plugin. Other XMPP clients like Spark or Pade do not have this problem.

The key problem is that Gajim and Openfire MAM handle message identification differently, which leads to duplicate or even triplicate messages when retrieving archived conversations.

⸻

Why Does This Happen?

To understand the root cause, it’s important to know how XMPP messages are stored and retrieved:
1. How Gajim Sends Messages:
• When Gajim sends a message, it assigns a unique stanza ID (an identifier for each message).
• This stanza ID is generated by Gajim itself and stored in its local database.
2. How Openfire MAM Stores Messages:
• When Openfire’s MAM plugin is enabled, it archives messages server-side.
• However, Openfire generates its own stanza ID when storing these messages in MAM.
• This means Openfire’s archived message has a different stanza ID than what Gajim originally assigned.
3. How Gajim Retrieves Messages from MAM:
• When Gajim reconnects or re-syncs, it fetches archived messages from Openfire’s MAM.
• Since Openfire’s MAM returns messages with different stanza IDs, Gajim treats them as new messages instead of recognizing them as duplicates.
• As a result, you see the same message multiple times—once from Gajim’s local database and once (or more) from Openfire’s MAM archive.

⸻

How I Identified the Problem
1. I downloaded Gajim’s source code and examined all Python scripts related to:
• How messages are stored.
• How messages are retrieved.
• How Gajim interacts with the MAM plugin in Openfire.
2. I found that Gajim’s local message storage and Openfire’s MAM storage were using different stanza IDs.
3. I verified this by disabling the MAM plugin in Openfire, and the issue disappeared. This confirmed that the stanza ID mismatch was the root cause.

⸻

How I Fixed It

Since I am not a programmer, I used ChatGPT to help analyze and modify the relevant Python scripts in Gajim.
• The fix involved modifying Gajim’s message retrieval system so that it properly handles the stanza IDs assigned by Openfire’s MAM.
• After applying these changes, I recompiled Gajim and tested it.
• With this modified version, Gajim now correctly recognizes archived messages from Openfire, preventing duplicates.

⸻

Final Notes
• I do not update Gajim in my company because newer versions still have this problem.
• If anyone needs the modified Gajim executable for Windows, I can provide it.
• This issue likely needs to be addressed by Gajim’s developers for a proper long-term fix.

Edited by GPT.

2 Likes
6

Thanks for your reply.

This issue occurs only in Gajim

We had the same behavior in Conversations (Android), too. And I just tried whatever other XMPP client I had installed, which happened to be Psi+. Strangely it happened there as well, although I don’t think Psi+ supports MAM at all…

Спойлер

image
image448×188 13.8 KB

7

I suspect that the problem is caused by Openfire not adding a ‘stable and unique’ stanza identifier to one-on-one messages (it does add these identifiers to group chat messages).

1 Like
8

Thank you for sharing your research, but I couldn’t repeat it.
I would like to ask you to provide a modified executable file Gajim for Windows
And if it’s not difficult, the file and the lines of code where you made the changes.

9

I solved the issue of duplicate messages when using Gajim with Openfire.