Smack 4.4.0-beta2: OMEMO message received with unknown identityKey due to mismatched fingerprints

During recent testing of OMEMO in aTalk, one of the device (Device-A) is locked into a state unable to exchange OMEMO messages with Device-B or any other OMEMO devices:

Test environment:

  1. Device A: Note-10; swordfish@atalk.sytes.net with omemo device ID 2116796881
  2. Device B: Note-5; leopard@atalk.sytes.net with omemo device ID 2022975859

After further investigation on the above two devices, it is found that the SQL database in Device A, actually contains an incorrect fingerprint value for ‘2022975859’ which is different from the correct fingerprint saved in Device-B database. I did not know when and how this situation come about, seems the data corruption was there some time ago.

In aTalk current implementation, I found that there is no defined way for aTalk to break out from this locked state. Neither of the current available options “Regenerate OMEMO Identities” nor “Purge unused Identities” able to resolve the problem. Only the aTalk debug version has a method to completely clear the OMEMO database and rebuild it from scratch able to solve this issue.

With reference to the debug log below: sending a message from Device B to Device A, did indeed trigger an UntrustedOmemoIdentityException. My understanding from the log, OMEMO routine just ignores the message. Not sure if this UntrustedOmemoIdentityException is propagated to the app layer for action.

Any good recommendation how to resolve this locked state in the field.

========== aTalk debug log: OMEMO message sent from Device B to Device A =========
2020-10-29 12:45:11.780 9781-21522/org.atalk.android D/SMACK: SENT (0):         
    <message to='leopard@atalk.sytes.net' id='1603946711465183634752' type='chat'>
      <encrypted xmlns='eu.siacs.conversations.axolotl'>
        <header sid='2116796881'>
          <key rid='2022975859'>
           MwohBf4uNHjhtb0KpcNkO56Vp66XcIu1qZJsOHTu5NsqoroYEAcYASIwZYFJ8hraXNaOMhazgDhcVW+1fYO08lS53ePp3ib9RPyz6pxjO6/Eiqeakvtk7KP02F/Vu63Bj/k=
          </key>
          <iv>
            pF0e6T2kQhPAiabt
          </iv>
        </header>
        <payload>
          e5mofCdL4g==
        </payload>
      </encrypted>
      <body>
        I sent you an OMEMO encrypted message but your client doesn't seem to support that. Find more information on https://conversations.im/omemo
      </body>
      <store xmlns='urn:xmpp:hints'/>
      <encryption xmlns='urn:xmpp:eme:0' namespace='eu.siacs.conversations.axolotl' name='OMEMO'/>
      <request xmlns='urn:xmpp:receipts'/>
      <active xmlns='http://jabber.org/protocol/chatstates'/>
      <origin-id xmlns='urn:xmpp:sid:0' id='GKRJ-7W46-N6UA-W'/>
    </message>
          
2020-10-29 12:45:12.826 9781-21523/org.atalk.android D/SMACK: RECV (0):          
    <message xml:lang='en' to='swordfish@atalk.sytes.net/atalk' from='leopard@atalk.sytes.net/atalk' id='E314T-62'>
      <encrypted xmlns='eu.siacs.conversations.axolotl'>
        <header sid='2022975859'>
          <key prekey='true' rid='2116796881'>
            MwheEiEFutZSoWNmJJLRIErGii3YZL8XarLoGO+/j+UF0Zlex0kaIQVUoD3h6j9Hvadji3FnpPYEsQKk1CsFa+YTfZwMFQAyDCJSMwohBRnpTs7O8p3Lxib+5SHQtkibdORmco98sx64wYr1MGRyEAAYACIgTnDXtKalHegnQaaW9G0W48d2+/a0GuxBUv9I652wjxcBXKg5m7Ts0ygAMAE=
          </key>
          <iv>
            mqbVvez5yuIk+SVJ
          </iv>
        </header>
      </encrypted>
      <origin-id xmlns='urn:xmpp:sid:0' id='4S41-GG51-VTYH-F'/>
    </message>          
          
2020-10-29 12:45:12.846 9781-21841/org.atalk.android W/aTalk: [154858] org.jivesoftware.smackx.omemo.OmemoRatchet.retrieveMessageKeyAndAuthTag() Received message from leopard@atalk.sytes.net:2022975859 contained unknown identityKey. Ignore message.
    org.jivesoftware.smackx.omemo.exceptions.UntrustedOmemoIdentityException: Untrusted OMEMO Identity encountered:
    Fingerprint of trusted key:
    24633a58 43923c28 1f56bcb1 e38fd8fd 8a7bd544 f3846e9a dcea6ac7 4e5f0d72
    Fingerprint of untrusted key:
    54a03de1 ea3f47bd a7638b71 67a4f604 b102a4d4 2b056be6 137d9c0c 1500320c
        at org.jivesoftware.smackx.omemo.signal.SignalOmemoRatchet.doubleRatchetDecrypt(SignalOmemoRatchet.java:97)
        at org.jivesoftware.smackx.omemo.OmemoRatchet.retrieveMessageKeyAndAuthTag(OmemoRatchet.java:107)
        at org.jivesoftware.smackx.omemo.OmemoService.decryptMessage(OmemoService.java:457)
        at org.jivesoftware.smackx.omemo.OmemoService.onOmemoMessageStanzaReceived(OmemoService.java:1192)
        at org.jivesoftware.smackx.omemo.OmemoManager$3.run(OmemoManager.java:959)
        at java.lang.Thread.run(Thread.java:919)
2020-10-29 12:45:12.850 9781-21841/org.atalk.android W/aTalk: [154858] org.jivesoftware.smackx.omemo.OmemoService.onOmemoMessageStanzaReceived() Could not decrypt incoming message: 
    org.jivesoftware.smackx.omemo.exceptions.CryptoFailedException: Transported key could not be decrypted, since no suitable message key was provided. Provides keys: [2116796881]
        at org.jivesoftware.smackx.omemo.OmemoRatchet.retrieveMessageKeyAndAuthTag(OmemoRatchet.java:127)
        at org.jivesoftware.smackx.omemo.OmemoService.decryptMessage(OmemoService.java:457)
        at org.jivesoftware.smackx.omemo.OmemoService.onOmemoMessageStanzaReceived(OmemoService.java:1192)
        at org.jivesoftware.smackx.omemo.OmemoManager$3.run(OmemoManager.java:959)
        at java.lang.Thread.run(Thread.java:919)

Trying to find the cause of the problem, so I added in another device i.e. avd-28: parrot@atalk.sytes.net:899121650 to test. But it too faces a similar problem.
I have taken screenshots of the three test devices (attached below). What I see is that each device has some mismatches in the fingerprints of the its buddies’ devices. Seems that these incorrect IdentityKey fingerprints are from the previous sessions.

snapshot Color Code:
Red: The physical devices and its omemoDevice Identity: its fingerprint is show in cyan color.
White: buddy active/inactive omemoDevices and their fingerprints

====================
From the captured debug log below (the log is captured as exception for tracing back to the caller); when I send an omemo message from avd-28 device to Note-10, the recipients’ identityKey fingerprints are being updated before the actual message is being sent. However the problem is that all the updated IdentityKey fingerprints are always the same incorrect values (presumably generated from a previous known sessions). This identityKey fingerprint update stage does not seem to help resolving the fingerprint mismatched problem.

While doing the test, I observed another strange behavior on Note-10 as below:
a. For unknown reason, its own omemodevice fingerprint is being cleared to null.
b. When I exited and relaunched aTalk; aTalk pop up an alert message indicates that its identityKey is missing. On checking the DB, what I see is that a new and different fingerprint is being generated for its own identityKey.

  1. Are the above two observations a & b are part of the omemo normal behavior?
  2. If so, how is this new identityKey fingerprint being informed and get updated to its buddies’ previously saved identityKey fingerprint.
  3. What I see is that, any new omemo messages sent from Note-10 to its buddies will contain this new identityKey, hence all the buddy recipients will throw UntrustedOmemoIdentityException.

Any advice?

====== aTalk debug log on sending an omemo message ============
2020-10-31 09:15:30.479 19763-19763/org.atalk.android E/(SQLiteOmemoStore.java:443)#storeOmemoIdentityKey: java.lang.Exception: Store device IdentityKey: swordfish@atalk.sytes.net:1245216054: 7ab3fae4268332bf26f7503d9caa5c8f5ce40a8d64a4d2bde6817b97ac7d3038
        at org.atalk.crypto.omemo.SQLiteOmemoStore.storeOmemoIdentityKey(SQLiteOmemoStore.java:443)
        at org.atalk.crypto.omemo.SQLiteOmemoStore.storeOmemoIdentityKey(SQLiteOmemoStore.java:63)
        at org.jivesoftware.smackx.omemo.signal.SignalOmemoStoreConnector.saveIdentity(SignalOmemoStoreConnector.java:109)
        at org.whispersystems.libsignal.SessionCipher.encrypt(SessionCipher.java:125)
        at org.jivesoftware.smackx.omemo.signal.SignalOmemoRatchet.doubleRatchetEncrypt(SignalOmemoRatchet.java:147)
        at org.jivesoftware.smackx.omemo.util.OmemoMessageBuilder.addRecipient(OmemoMessageBuilder.java:236)
        at org.jivesoftware.smackx.omemo.OmemoService.encrypt(OmemoService.java:409)
        at org.jivesoftware.smackx.omemo.OmemoService.createOmemoMessage(OmemoService.java:542)
        at org.jivesoftware.smackx.omemo.OmemoManager.encrypt(OmemoManager.java:340)
        at org.jivesoftware.smackx.omemo.OmemoManager.encrypt(OmemoManager.java:313)
        at net.java.sip.communicator.impl.protocol.jabber.OperationSetBasicInstantMessagingJabberImpl.sendInstantMessage(OperationSetBasicInstantMessagingJabberImpl.java:522)
        at org.atalk.android.gui.chat.MetaContactChatTransport.sendInstantMessage(MetaContactChatTransport.java:414)
        at org.atalk.android.gui.chat.ChatController.sendMessage(ChatController.java:314)
        at org.atalk.android.gui.chat.ChatController.onClick(ChatController.java:481)
        at android.view.View.performClick(View.java:6597)
        at android.view.View.performClickInternal(View.java:6574)
        at android.view.View.access$3100(View.java:778)
        at android.view.View$PerformClick.run(View.java:25885)
        at android.os.Handler.handleCallback(Handler.java:873)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at android.os.Looper.loop(Looper.java:193)
        at android.app.ActivityThread.main(ActivityThread.java:6669)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)
2020-10-31 09:15:30.484 19763-19763/org.atalk.android D/(SQLiteOmemoStore.java:466)#storeOmemoIdentityKey: Skip Update duplicated identityKey for: swordfish@atalk.sytes.net:1245216054; org.whispersystems.libsignal.IdentityKey@25960e73; 7ab3fae4268332bf26f7503d9caa5c8f5ce40a8d64a4d2bde6817b97ac7d3038
2020-10-31 09:15:30.509 19763-19763/org.atalk.android E/(SQLiteOmemoStore.java:443)#storeOmemoIdentityKey: java.lang.Exception: Store device IdentityKey: parrot@atalk.sytes.net:1566361092: 3c9d0a7fe93a1bef350112f8056f34d9bece8547781fe01dda99f9754b0b800d
        at org.atalk.crypto.omemo.SQLiteOmemoStore.storeOmemoIdentityKey(SQLiteOmemoStore.java:443)
        at org.atalk.crypto.omemo.SQLiteOmemoStore.storeOmemoIdentityKey(SQLiteOmemoStore.java:63)
        at org.jivesoftware.smackx.omemo.signal.SignalOmemoStoreConnector.saveIdentity(SignalOmemoStoreConnector.java:109)
        at org.whispersystems.libsignal.SessionCipher.encrypt(SessionCipher.java:125)
        at org.jivesoftware.smackx.omemo.signal.SignalOmemoRatchet.doubleRatchetEncrypt(SignalOmemoRatchet.java:147)
        at org.jivesoftware.smackx.omemo.util.OmemoMessageBuilder.addRecipient(OmemoMessageBuilder.java:236)
        at org.jivesoftware.smackx.omemo.OmemoService.encrypt(OmemoService.java:409)
        at org.jivesoftware.smackx.omemo.OmemoService.createOmemoMessage(OmemoService.java:542)
        at org.jivesoftware.smackx.omemo.OmemoManager.encrypt(OmemoManager.java:340)
        at org.jivesoftware.smackx.omemo.OmemoManager.encrypt(OmemoManager.java:313)
        at net.java.sip.communicator.impl.protocol.jabber.OperationSetBasicInstantMessagingJabberImpl.sendInstantMessage(OperationSetBasicInstantMessagingJabberImpl.java:522)
        at org.atalk.android.gui.chat.MetaContactChatTransport.sendInstantMessage(MetaContactChatTransport.java:414)
        at org.atalk.android.gui.chat.ChatController.sendMessage(ChatController.java:314)
        at org.atalk.android.gui.chat.ChatController.onClick(ChatController.java:481)
        at android.view.View.performClick(View.java:6597)
        at android.view.View.performClickInternal(View.java:6574)
        at android.view.View.access$3100(View.java:778)
        at android.view.View$PerformClick.run(View.java:25885)
        at android.os.Handler.handleCallback(Handler.java:873)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at android.os.Looper.loop(Looper.java:193)
        at android.app.ActivityThread.main(ActivityThread.java:6669)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)
2020-10-31 09:15:30.511 19763-19763/org.atalk.android D/(SQLiteOmemoStore.java:466)#storeOmemoIdentityKey: Skip Update duplicated identityKey for: parrot@atalk.sytes.net:1566361092; org.whispersystems.libsignal.IdentityKey@932a825b; 3c9d0a7fe93a1bef350112f8056f34d9bece8547781fe01dda99f9754b0b800d
2020-10-31 09:15:30.520 19763-19763/org.atalk.android D/(AbstractOperationSetBasicInstantMessaging.java:116)#fireMessageEvent: Dispatching Message Listeners = 4 evt = net.java.sip.communicator.service.protocol.event.MessageDeliveredEvent[source=net.java.sip.communicator.impl.protocol.jabber.MessageJabberImpl@dce69e5]
2020-10-31 09:15:30.523 19763-19899/org.atalk.android D/SMACK: SENT (0): 
    <message to='swordfish@atalk.sytes.net/atalk' id='1604106930434231631333' type='chat'>
      <encrypted xmlns='eu.siacs.conversations.axolotl'>
        <header sid='889121650'>
          <key prekey='true' rid='1245216054'>
            MwgREiEFreBH/19FDml4XqMsLRiZfAVFQzfLw2ltpjBvszh/dW4aIQX/z+aSp/8GRa1Nyoxc05o55/zmFjq61sd/ND6hQkllOSJiMwohBeS6ORV9y+DWepfTx9uEfB3/N1BZG49NN4Ua1ZrQUYULEBAYACIwKl+BI0kTAkZsy7l6CZ87Xdr3Vyescdsb8G8wiRzYSRVcbmrs3CJAfQoz2Gf3FrP2YRCxeenARDsoADAB
          </key>
          <key prekey='true' rid='1566361092'>
            Mwg/EiEFBlcQ3cdASXgNL3/NQPy4wteFHkBYVz1vUHL5FOMd3n0aIQV8IPoxhhEu2RedUV73jbhtdHFtml2PkilCxyvOJcWwKiJiMwohBTLgrW2TrL+CrMzaSPSpNRvI6U3CgnOseMsGkdE0J0o7EAgYACIwpqXdTv0Vn2r0h3VSnSp8UHQE84QcRvy9sHnsBBq2yCR8vIaWuwZaO9rNYIi7MJv6t2pC/uj0/k8oADAB
          </key>
          <iv>
            lAPk/ge60n2Wn0RH
          </iv>
        </header>
        <payload>
          79nOKQ==
        </payload>
      </encrypted>
      <body>
        I sent you an OMEMO encrypted message but your client doesn't seem to support that. Find more information on https://conversations.im/omemo
      </body>
      <store xmlns='urn:xmpp:hints'/>
      <encryption xmlns='urn:xmpp:eme:0' namespace='eu.siacs.conversations.axolotl' name='OMEMO'/>
      <active xmlns='http://jabber.org/protocol/chatstates'/>
      <origin-id xmlns='urn:xmpp:sid:0' id='Y1BK-XC8T-ADYQ-M'/>
      <request xmlns='urn:xmpp:receipts'/>

Did you write some unit tests for your OmemoStore implementation?
I’d say there is a bug somewhere in your code, as I’m pretty sure that Smacks code works as intended.
You might want to hook your OmemoStore into Smacks unit tests here to get started.

Thanks, I will try and see how to integrate and perform the unit test in aTalk.

Actually I have performed another detailed source code review of aTalk implementation i.e.

public class SQLiteOmemoStore extends SignalOmemoStore implements OmemoManager.InitializationFinishedCallback;

and compared every implemented method against smack FileBasedOmemoStore class.

I see that there are two function calls from smack that make changes to the fingerprints (2nd is of interest in this investigation) i.e.:

*storeOmemoIdentityKeyPair(OmemoDevice userDevice, IdentityKeyPair identityKeyPair)
*storeOmemoIdentityKey(OmemoDevice userDevice, OmemoDevice contactDevice, IdentityKey contactKey).

I also verified aTalk Omemo Regeneration implementation, and found that is some shortfall in the actual operations. Has requested advice from ejabberd team:
XEP-0060: Publish-Subscribe: Delete node and purge node operations in ejabberd #3418.

Actually this round of the OMEMO re-testing is to investigate a field ANR report from google console when user performs a OMEMO regeneration. I now made the operation a background thread to resolve ANR.

During the aTalk source review (fingerprint mismatch), I concluded that an OMEMO regeneration operation on the device should not have corrupted its buddy devices fingerprints contents in the DB; as the contents are left untouched.

I am now thinking, whether there is a possibility the incorrect update of the identity fingerprints on the device is due a race condition, when its buddy device is performing an OMEMO regeneration. I have observed and confirmed that one of the test device, its DB contains the buddy newly generated
OMEMO deviceId, but its fingerprint is of the old deviceId.

In all mismatched fingerprint occurrence, it is because the fingerprint is not being updated, and not due to data corrupted.

I now need to understand why there is always a function call from smack to storeOmemoIdentityKey() of all the message’s recipients, before the OMEMO message is being sent; and when/how the recipients’ fingerprints are being generated.
Do you have any info on this process flow?

To be clear: when performing an OMEMO regeneration, the device-ID is also regenerated. Please make sure that you don’t reuse an old device-ID with new keys. This might otherwise cause your issues.

The problem may have arised due to aTalk incorrect OMEMO regeneration implement. I will close the issue. If need to, will raise again if problem re-appear.

See
Smack 4.4.0-beta2: OMEMO continues using old OmemoDevice Id even after the regeneration process with new DeviceID specified

1 Like

This topic was automatically closed 100 days after the last reply. New replies are no longer allowed.