Smack 4.4.0-alpha2-20190206: XmlStringBuiler#wirte method throws null pointer reference exception

cmeng · February 8, 2019, 6:00am

The XmlStringBuiler#wirte throws null pointer reference exception. I have modified the write method (attached below) to capture this exception. Apparently stanza to be sent may contain null array element i.e. The first stanza of the attached sending <present> stanzas below does contain a null element after the line

<c xmlns='http://jabber.org/protocol/caps' hash='sha-1' node='http://android.atalk.org' ver='SPANeHrMhKxipyDRRXFZu1T21So='/>

Without the null check, aTalk throw exception during
CarbonManager.setCarbonsEnabled(true); and
Omemo init()

I need further checking to ensure that the null element in the stanza is not caused by aTalk.

2019-02-08 13:20:01.361 10059-10768/org.atalk.android W/System.err:     at org.jivesoftware.smack.util.XmlStringBuilder.write(XmlStringBuilder.java:588)
2019-02-08 13:20:01.361 10059-10768/org.atalk.android W/System.err:     at org.jivesoftware.smack.tcp.XMPPTCPConnection$PacketWriter.writePackets(XMPPTCPConnection.java:1273)
2019-02-08 13:20:01.361 10059-10768/org.atalk.android W/System.err:     at org.jivesoftware.smack.tcp.XMPPTCPConnection$PacketWriter.access$3100(XMPPTCPConnection.java:1087)
2019-02-08 13:20:01.361 10059-10768/org.atalk.android W/System.err:     at org.jivesoftware.smack.tcp.XMPPTCPConnection$PacketWriter$1.run(XMPPTCPConnection.java:1135)
2019-02-08 13:20:01.361 10059-10768/org.atalk.android W/System.err:     at java.lang.Thread.run(Thread.java:764)
2019-02-08 13:20:01.362 10059-10768/org.atalk.android D/SMACK: SENT (0): 
    <presence id='Dk1jF-95'>
      <status/>
      <priority>
        30
      </priority>
      <c xmlns='http://jabber.org/protocol/caps' hash='sha-1' node='http://android.atalk.org' ver='SPANeHrMhKxipyDRRXFZu1T21So='/>
    </presence>
2019-02-08 13:20:01.448 10059-10769/org.atalk.android D/SMACK: RECV (0): 
    <presence xml:lang='en' to='swordfish@atalk.org/atalk' from='swordfish@atalk.org/atalk' id='Dk1jF-95'>
      <c xmlns='http://jabber.org/protocol/caps' hash='sha-1' node='http://android.atalk.org' ver='SPANeHrMhKxipyDRRXFZu1T21So='/>
      <x xmlns='vcard-temp:x:update'>
        <photo>
          1bfa4a6e285b3f609ddeeb7c95de851dcf3eb56c
        </photo>
      </x>
      <priority>
        30
      </priority>
      <status/>
    </presence>

    public void write(Writer writer, String enclosingNamespace) throws IOException {
        for (CharSequence csq : sb.getAsList()) {
            if (csq instanceof XmlStringBuilder) {
                ((XmlStringBuilder) csq).write(writer, enclosingNamespace);
            }
            else if (csq instanceof XmlNsAttribute) {
                XmlNsAttribute xmlNsAttribute = (XmlNsAttribute) csq;
                if (!xmlNsAttribute.value.equals(enclosingNamespace)) {
                    writer.write(xmlNsAttribute.toString());
                    enclosingNamespace = xmlNsAttribute.value;
                }
            }
            else if (csq == null)
                new Exception("csq is null").printStackTrace();
            else {
                writer.write(csq.toString());
            }
        }
    }

cmeng · February 9, 2019, 2:41am

After further testing, found that the problem is due to a call in aTalk VcardAvatarManager to addExtension to the stanza <presence>

Below are the source extractions of the aTalk/Smack (with added debug info) to find the root cause.

============== aTalk VcardAvatarManager =============
    /**
     * Creates a filter to only listen to presence stanza with the element name "x" but without the
     * namespace "vcard-temp:x:update".
     */
    private static final StanzaFilter PRESENCES_WITHOUT_VCARD
            = new AndFilter(PresenceTypeFilter.AVAILABLE, new NotFilter(new StanzaExtensionFilter(ELEMENT, NAMESPACE)));

    connection.addStanzaInterceptor(this, PRESENCES_WITHOUT_VCARD);

    /**
     * Intercepts sent presence packets in order to add VCardTempXUpdate extension.
     *
     * @param stanza The sent presence packet.
     */
    @Override
    public void processStanza(Stanza stanza)
    {
        // Do not add an elementExtension without hash value
        if (vCardTempXUpdate.getAvatarHash() != null) {
            LOGGER.warning("Stanza EE before add: " + stanza.toXML() + "\n" + stanza.getExtensions()
                    + "\n" + vCardTempXUpdate.toXML());
            stanza.addExtension(vCardTempXUpdate);
            LOGGER.warning("New Stanza EE after add: " + stanza.toXML() + "\n"  + stanza.getExtensions());
        }
    }

============== smack XmlStringBuilder ===============
    /**
     * Write the contents of this <code>XmlStringBuilder</code> to a {@link Writer}. This will write
     * the single parts one-by-one, avoiding allocation of a big continuous memory block holding the
     * XmlStringBuilder contents.
     *
     * @param writer
     * @throws IOException
     */
    public void write(Writer writer, String enclosingNamespace) throws IOException {
        for (CharSequence csq : sb.getAsList()) {
            if (csq instanceof XmlStringBuilder) {
                ((XmlStringBuilder) csq).write(writer, enclosingNamespace);
            }
            else if (csq instanceof XmlNsAttribute) {
                XmlNsAttribute xmlNsAttribute = (XmlNsAttribute) csq;
                if (!xmlNsAttribute.value.equals(enclosingNamespace)) {
                    writer.write(xmlNsAttribute.toString());
                    enclosingNamespace = xmlNsAttribute.value;
                }
            }
            else if (csq == null)
                LOGGER.warning("Stanza contains null: " + sb.getAsList());
            else {
                writer.write(csq.toString());
            }
        }
    }

Following is the debug log captured. It is found that statement below actually added a “null” instead of the desired extension i.e. vCardTempXUpdate. Look like this problem only happen in the latest release of the smack source (believe smack now uses CharSequence toXML(XmlEnvironment xmlEnvironment - will change aTalk source) i.e. presently return null form aTalk, instead of the toXml()); the previous release did not cause the null reference exceptions to be thrown.

While testing, I also found that it is not necessary for aTalk to add the CardTempXUpdate elementExtension; it seems that the ejabberd server has implementation that will automatically add CardTempXUpdate to the <presence> stanza sending to all the contacts. Will remove the addExtension from aTalk.

!!!new!!!: Just confirmed new smack release call the method below. No more problem after aTalk source changes to implement the method instead of return null.

public CharSequence toXML(XmlEnvironment xmlEnvironment)

Not sure if smack will consider to check csq for null before writer.write(csq.toString());

stanza.addExtension(vCardTempXUpdate);

2019-02-09 10:01:27.322 18641-19267/org.atalk.android W/aTalk: [3793] org.jivesoftware.smackx.avatar.vcardavatar.VCardAvatarManager.processStanza() Stanza EE before add: <presence xmlns='jabber:client' id='9caj4-96'><status/><priority>30</priority><c xmlns='http://jabber.org/protocol/caps' hash='sha-1' node='http://android.atalk.org' ver='SPANeHrMhKxipyDRRXFZu1T21So='/></presence>
   [org.jivesoftware.smackx.caps.packet.CapsExtension@9547c26]
   <x xmlns='vcard-temp:x:update'><photo>1bfa4a6e285b3f609ddeeb7c95de851dcf3eb56c</photo></x>
2019-02-09 10:01:27.325 18641-19267/org.atalk.android E/aTalk: [3793] org.jivesoftware.smack.util.LazyStringBuilder.length() The following LazyStringBuilder threw a NullPointerException:  <presence xmlns='jabber:client' id='9caj4-96'><status/><priority>30</priority><c xmlns='http://jabber.org/protocol/caps' hash='sha-1' node='http://android.atalk.org' ver='SPANeHrMhKxipyDRRXFZu1T21So='/>null</presence>
   java.lang.NullPointerException: Attempt to invoke interface method 'int java.lang.CharSequence.length()' on a null object reference
       at org.jivesoftware.smack.util.LazyStringBuilder.length(LazyStringBuilder.java:78)
       at org.jivesoftware.smack.util.LazyStringBuilder.toString(LazyStringBuilder.java:112)
       at org.jivesoftware.smack.util.XmlStringBuilder.toString(XmlStringBuilder.java:556)
       at java.lang.String.valueOf(String.java:2827)
       at java.lang.StringBuilder.append(StringBuilder.java:132)
       at org.jivesoftware.smackx.avatar.vcardavatar.VCardAvatarManager.processStanza(VCardAvatarManager.java:319)
       at org.jivesoftware.smack.AbstractXMPPConnection.firePacketInterceptors(AbstractXMPPConnection.java:1107)
       at org.jivesoftware.smack.AbstractXMPPConnection.sendStanza(AbstractXMPPConnection.java:766)
       at org.jivesoftware.smackx.caps.EntityCapsManager.updateLocalEntityCaps(EntityCapsManager.java:564)
       at org.jivesoftware.smackx.caps.EntityCapsManager.access$500(EntityCapsManager.java:82)
       at org.jivesoftware.smackx.caps.EntityCapsManager$8.onEntityCapailitiesChanged(EntityCapsManager.java:407)
       at org.jivesoftware.smackx.disco.ServiceDiscoveryManager.renewEntityCapsVersion(ServiceDiscoveryManager.java:854)
       at org.jivesoftware.smackx.disco.ServiceDiscoveryManager.addFeature(ServiceDiscoveryManager.java:389)
       at org.jivesoftware.smackx.omemo.OmemoManager.initialize(OmemoManager.java:251)
       at org.jivesoftware.smackx.omemo.OmemoManager$2.run(OmemoManager.java:266)
       at java.lang.Thread.run(Thread.java:764)
2019-02-09 10:01:27.327 18641-19267/org.atalk.android E/aTalk: [3793] org.jivesoftware.smack.AbstractXMPPConnection.firePacketInterceptors() Packet interceptor threw exception
   java.lang.NullPointerException: Attempt to invoke interface method 'int java.lang.CharSequence.length()' on a null object reference
       at org.jivesoftware.smack.util.LazyStringBuilder.length(LazyStringBuilder.java:78)
       at org.jivesoftware.smack.util.LazyStringBuilder.toString(LazyStringBuilder.java:112)
       at org.jivesoftware.smack.util.XmlStringBuilder.toString(XmlStringBuilder.java:556)
       at java.lang.String.valueOf(String.java:2827)
       at java.lang.StringBuilder.append(StringBuilder.java:132)
       at org.jivesoftware.smackx.avatar.vcardavatar.VCardAvatarManager.processStanza(VCardAvatarManager.java:319)
       at org.jivesoftware.smack.AbstractXMPPConnection.firePacketInterceptors(AbstractXMPPConnection.java:1107)
       at org.jivesoftware.smack.AbstractXMPPConnection.sendStanza(AbstractXMPPConnection.java:766)
       at org.jivesoftware.smackx.caps.EntityCapsManager.updateLocalEntityCaps(EntityCapsManager.java:564)
       at org.jivesoftware.smackx.caps.EntityCapsManager.access$500(EntityCapsManager.java:82)
       at org.jivesoftware.smackx.caps.EntityCapsManager$8.onEntityCapailitiesChanged(EntityCapsManager.java:407)
       at org.jivesoftware.smackx.disco.ServiceDiscoveryManager.renewEntityCapsVersion(ServiceDiscoveryManager.java:854)
       at org.jivesoftware.smackx.disco.ServiceDiscoveryManager.addFeature(ServiceDiscoveryManager.java:389)
       at org.jivesoftware.smackx.omemo.OmemoManager.initialize(OmemoManager.java:251)
       at org.jivesoftware.smackx.omemo.OmemoManager$2.run(OmemoManager.java:266)
       at java.lang.Thread.run(Thread.java:764)

2019-02-09 10:01:27.330 18641-19263/org.atalk.android W/aTalk: [3789] org.jivesoftware.smack.util.XmlStringBuilder.write() Stanza contains null: [<, presence,  , id, =', 9caj4-96, ', >, <, status, />, <, priority, >, 30, </, priority, >, <c xmlns='http://jabber.org/protocol/caps' hash='sha-1' node='http://android.atalk.org' ver='SPANeHrMhKxipyDRRXFZu1T21So='/>, null, </, presence, >]
2019-02-09 10:01:27.332 18641-19263/org.atalk.android D/SMACK: SENT (0): 
   <presence id='9caj4-96'>
     <status/>
     <priority>
       30
     </priority>
     <c xmlns='http://jabber.org/protocol/caps' hash='sha-1' node='http://android.atalk.org' ver='SPANeHrMhKxipyDRRXFZu1T21So='/>
   </presence>
2019-02-09 10:01:27.396 18641-19264/org.atalk.android D/SMACK: RECV (0): 
   <presence xml:lang='en' to='swordfish@atalk.org/atalk' from='swordfish@atalk.org/atalk' id='9caj4-96'>
     <c xmlns='http://jabber.org/protocol/caps' hash='sha-1' node='http://android.atalk.org' ver='SPANeHrMhKxipyDRRXFZu1T21So='/>
     <x xmlns='vcard-temp:x:update'>
       <photo>
         1bfa4a6e285b3f609ddeeb7c95de851dcf3eb56c
       </photo>
     </x>
     <priority>
       30
     </priority>
     <status/>
   </presence>

Flow · February 9, 2019, 7:49am

This is usually a sign that the XmlStringBuilder was invoked with a null argument which should not be null. For example:

XmlStringBuilder xml = …
String text = null;
xml.append(text);

this will later cause an NPE when the XML string is requested from XmlStringBuilder. It is up to the user that he does not hand null to the XmlStringBuilder when not use one of the methods that are prefixed with opt.

cmeng · February 9, 2019, 9:06am

Thanks for the clarification, the problem is aTalk just accepts AS auto inserted default method which returns null.

CharSequence toXML(XmlEnvironment xmlEnvironment);

New Smack makes call to this method instead of the previously implemented

XmlStringBuilder toXML(String enclosingNamespace)

This is my oversight. Have changed and everything is now working with the new smack release.

system · May 20, 2019, 9:09am

This topic was automatically closed 100 days after the last reply. New replies are no longer allowed.