Smack 4.4.7: Omemo publish bundles with its default deviceId and deviceId = -1

This was observed on swn@atalk.sytes.net device with omemoDevice = ‘swn@atalk.sytes.net:534057247’.

The weird behaviour was first observed while doing aTalk system testing i.e. on aTalk handling of offline omemo encrypted messages. Only 3 out of 5 offline messages were received by swan, and was found that smack thrown the following errors after received the 3rd offline messages on investigation.

08:34:33.134  E  Corrupted Omemo IdentityKeyPair: Invalid IdentityKeyPairs for omemoDevice: swan@atalk.sytes.net:-1.
                 Exception: com.google.protobuf.InvalidProtocolBufferException: Protocol message contained an invalid tag (zero)..
--------- beginning of crash
08:34:33.138  E  [188] org.jivesoftware.smackx.omemo.signal.SignalOmemoStoreConnector.getIdentityKeyPair() IdentityKeyPair seems to be invalid.
                 org.jivesoftware.smackx.omemo.exceptions.CorruptedOmemoKeyException: Invalid IdentityKeyPairs for omemoDevice: swan@atalk.sytes.net:-1.
                 Exception: com.google.protobuf.InvalidProtocolBufferException: Protocol message contained an invalid tag (zero)..
                 	at org.atalk.crypto.omemo.SQLiteOmemoStore.loadOmemoIdentityKeyPair(SQLiteOmemoStore.java:360)
                 	at org.atalk.crypto.omemo.SQLiteOmemoStore.loadOmemoIdentityKeyPair(SQLiteOmemoStore.java:79)
                 	at org.jivesoftware.smackx.omemo.signal.SignalOmemoStoreConnector.getIdentityKeyPair(SignalOmemoStoreConnector.java:82)
                 	at org.whispersystems.libsignal.SessionBuilder.processV3(SessionBuilder.java:129)
                 	at org.whispersystems.libsignal.SessionBuilder.process(SessionBuilder.java:107)
                 	at org.whispersystems.libsignal.SessionCipher.decrypt(SessionCipher.java:180)
                 	at org.whispersystems.libsignal.SessionCipher.decrypt(SessionCipher.java:150)
                 	at org.jivesoftware.smackx.omemo.signal.SignalOmemoRatchet.doubleRatchetDecrypt(SignalOmemoRatchet.java:101)
                 	at org.jivesoftware.smackx.omemo.OmemoRatchet.retrieveMessageKeyAndAuthTag(OmemoRatchet.java:110)
                 	at org.jivesoftware.smackx.omemo.OmemoService.decryptMessage(OmemoService.java:457)
                 	at org.jivesoftware.smackx.omemo.OmemoService.onOmemoMessageStanzaReceived(OmemoService.java:1194)
                 	at org.jivesoftware.smackx.omemo.OmemoManager$3.run(OmemoManager.java:976)
                 	at java.lang.Thread.run(Thread.java:1012)
08:34:33.141  E  FATAL EXCEPTION: Thread-79
                 Process: org.atalk.android, PID: 10654
                 java.lang.IllegalArgumentException: Null value!
                 	at org.whispersystems.libsignal.ratchet.BobSignalProtocolParameters.<init>(BobSignalProtocolParameters.java:38)
                 	at org.whispersystems.libsignal.ratchet.BobSignalProtocolParameters$Builder.create(BobSignalProtocolParameters.java:110)
                 	at org.whispersystems.libsignal.SessionBuilder.processV3(SessionBuilder.java:141)
                 	at org.whispersystems.libsignal.SessionBuilder.process(SessionBuilder.java:107)
                 	at org.whispersystems.libsignal.SessionCipher.decrypt(SessionCipher.java:180)
                 	at org.whispersystems.libsignal.SessionCipher.decrypt(SessionCipher.java:150)
                 	at org.jivesoftware.smackx.omemo.signal.SignalOmemoRatchet.doubleRatchetDecrypt(SignalOmemoRatchet.java:101)
                 	at org.jivesoftware.smackx.omemo.OmemoRatchet.retrieveMessageKeyAndAuthTag(OmemoRatchet.java:110)
                 	at org.jivesoftware.smackx.omemo.OmemoService.decryptMessage(OmemoService.java:457)
                 	at org.jivesoftware.smackx.omemo.OmemoService.onOmemoMessageStanzaReceived(OmemoService.java:1194)
                 	at org.jivesoftware.smackx.omemo.OmemoManager$3.run(OmemoManager.java:976)
                 	at java.lang.Thread.run(Thread.java:1012)

When checking database, found that ‘swn@atalk.sytes.net:-1’ was instead stored in the omemo tables.
aTalk purgeInactiveUserDevices() was modified to include ‘-1’ in the inactiveDevices list, then executed to purge inactiveDevices from the database. The database was checked and confirmed clean.

aTalk was then restarted, but found smack publish two bundles: first with ‘swn@atalk.sytes.net:534057247’, then ‘swn@atalk.sytes.net:-1’ with the exact same prekeys. The omemo db tables were again filled with ‘swn@atalk.sytes.net:-1’ parameter. Only after 2 rounds of purgeInactiveUserDevices(), the system returns to normal operation.

Not sure how in the first place the problem was started. The information is kept here for update if more info is uncovered.

Any comment from the smack team.

// ======== Pusub with own deviceID = 534057247 ===========
09:42:01.532  D  SENT (1): 
                 <iq id='AQ1HD-37' type='set'>
                   <pubsub xmlns='http://jabber.org/protocol/pubsub'>
                     <publish node='eu.siacs.conversations.axolotl.bundles:534057247'>
                       <item>
                         <bundle xmlns='eu.siacs.conversations.axolotl'>
....                         
                           </prekeys>
                         </bundle>
                       </item>
                     </publish>
                   </pubsub>
                 </iq>

09:42:11.809  D  SENT (1): 
                 <iq id='AQ1HD-45' type='set'>
                   <pubsub xmlns='http://jabber.org/protocol/pubsub'>
                     <publish node='eu.siacs.conversations.axolotl.bundles:-1'>
                       <item>
                         <bundle xmlns='eu.siacs.conversations.axolotl'>
                           <signedPreKeyPublic signedPreKeyId='1'>
                             Bd2gfYIfUm68NloSMY6UwU67PtrlzWYmEopMLsqSHT1b
                           </signedPreKeyPublic>
.....
                             <preKeyPublic preKeyId='100'>
                               BZ8Oqa6Uk0SgOt3Op6mwU4IoO2BVGjYfyEpdVCC2k0Fd
                             </preKeyPublic>
                           </prekeys>
                         </bundle>
                       </item>
                     </publish>
                   </pubsub>
                 </iq>

09:42:09.923  D  RECV (1): 
                 <iq xml:lang='en-US' to='swan@atalk.sytes.net/atalk-2huivli' from='swan@atalk.sytes.net' type='result' id='AQ1HD-37'>
                   <pubsub xmlns='http://jabber.org/protocol/pubsub'>
                     <publish node='eu.siacs.conversations.axolotl.bundles:534057247'>
                       <item id='6AE5CE766231D'/>
                     </publish>
                   </pubsub>
                 </iq>

// ======== Re-Pusub with own deviceId = -1 ===========

// public with -1
09:42:21.124  D  RECV (1): 
                 <message to='swan@atalk.sytes.net' from='swan@atalk.sytes.net' type='headline'>
                   <event xmlns='http://jabber.org/protocol/pubsub#event'>
                     <items node='eu.siacs.conversations.axolotl.bundles:-1'>
                       <item id='6AE5CE818EB86'>
...
                       </item>
                     </items>
                   </event>
                   <addresses xmlns='http://jabber.org/protocol/address'>
                     <address jid='swan@atalk.sytes.net/atalk-2huivli' type='replyto'/>
                   </addresses>
                 </message>

09:42:21.141  D  RECV (1): 
                 <iq xml:lang='en-US' to='swan@atalk.sytes.net/atalk-2huivli' from='swan@atalk.sytes.net' type='result' id='AQ1HD-45'>
                   <pubsub xmlns='http://jabber.org/protocol/pubsub'>
                     <publish node='eu.siacs.conversations.axolotl.bundles:-1'>
                       <item id='6AE5CE818EB86'/>
                     </publish>
                   </pubsub>
                 </iq>


09:42:21.141  D  RECV (1): 
                 <iq xml:lang='en-US' to='swan@atalk.sytes.net/atalk-2huivli' from='swan@atalk.sytes.net' type='result' id='AQ1HD-45'>
                   <pubsub xmlns='http://jabber.org/protocol/pubsub'>
                     <publish node='eu.siacs.conversations.axolotl.bundles:-1'>
                       <item id='6AE5CE818EB86'/>
                     </publish>
                   </pubsub>
                 </iq>                 	
                 	
// ======= Resulted =========
09:42:21.183  D  SENT (1): 
                 <iq id='AQ1HD-50' type='set'>
                   <pubsub xmlns='http://jabber.org/protocol/pubsub'>
                     <publish node='eu.siacs.conversations.axolotl.devicelist'>
                       <item>
                         <list xmlns='eu.siacs.conversations.axolotl'>
                           <device id='-1'/>
                           <device id='534057247'/>
                         </list>
                       </item>
                     </publish>
                   </pubsub>
                 </iq>
                                 	
09:42:22.284  D  RECV (1): 
                 <message to='swan@atalk.sytes.net' from='swan@atalk.sytes.net' type='headline'>
                   <event xmlns='http://jabber.org/protocol/pubsub#event'>
                     <items node='eu.siacs.conversations.axolotl.devicelist'>
                       <item id='6AE5CE82C0852'>
                         <list xmlns='eu.siacs.conversations.axolotl'>
                           <device id='-1'/>
                           <device id='534057247'/>
                         </list>
                       </item>
                     </items>
                   </event>