powered by Jive Software

Smack 4.4.0: Smack BOSH receives "400 Bad Request' from server when sending large amount of data via mobile network

Attached is the latest patch for the XMPPBOSHConnection to resolve two new problems found i.e.:

  1. Call notifyConnectionError() while it is not connected
  2. Problem in handling the “error” in BOSHPacketReader#responseReceived()
  3. Source cleanup

XMPPBOSHConnection.patch (15.9 KB)

The patched XMPPBOSHConnected has been tested using aTalk client with the following observations i.e.

a. When android device is connected to network via local WiFi network:

  1. The XMPPBOSHConnection is able and always has no problem of a successfully login via BOSH protocol.
  2. ReconnectionManager is also working when WiFi connection is temporary disrupted; or when switching from Mobile network to WiFi network.
  3. Steam:error is correctly reported to the upper app level.

b. When android device is connected to network via mobile network:

  1. The BOSH connection always (>95%) failed when sending large amount of data to server (publish prekeys <body> size is 9476 bytes); at a very specific location as shown in the log below during i.e.
<body rid='4370261655789057' sid='f7ee00539b64d8961421a28b33508d71998a10ca' xmlns='http://jabber.org/protocol/httpbind'>
  <iq xmlns='jabber:client' id='XXKUG-218' type='set'>
    <pubsub xmlns='http://jabber.org/protocol/pubsub'>
      <publish node='eu.siacs.conversations.axolotl.bundles:1175061843'>
  1. Subsequent attempts to reconnect by ReconnectionManager, they too and always failed at the exact same location.

  2. BoshClient does not provide any special handler when server reported
    <h1>400 Bad Request</h1>
    There is no attempt by jbosh to retry to resend the last bad-request stanza. jBosh just throws BOSHException. This causes the whole login process to restart by ReconnectionManager.
    Wonder whether it is appropriate for jbosh to reconsider to make a reattempt to resend the last failed stanza.
    Actually I modified the BOSHClient source, so it re-attempts to send the last stanza on bad-request failure, but without any success. The resend ‘published prekeys’ always failed no matter how many attempts has been made.

Found that ejabberd reports the error incorrectly; should be as per
17.2 Terminal Binding Conditions i.e.

<body type='terminate' condition='bad-request' xmlns='http://jabber.org/protocol/httpbind'/>

BOSHClient will do the following without re-attempt,

            checkForTerminalBindingConditions(body, respCode);
            if (isTermination(body)) {
                // Explicit termination
                lock.unlock();
                dispose(null);
                return;
            }
  1. The problem does not seem to be caused by (my initial suspect)
    https://stackoverflow.com/questions/47723973/strictmode-java-lang-throwable-untagged-socket-detected

  2. ejabberd debug log shows that there is no data being received (reported by low level mod_bosh.erl).

======= mod_bosh.erl ======
process([], #request{method = 'POST', data = <<>>}) ->
    ?DEBUG("Bad Request: no data", []),
    {400, ?HEADER(?CT_XML),
     #xmlel{name = <<"h1">>, attrs = [],
	    children = [{xmlcdata, <<"400 Bad Request">>}]}};

and at ejabberd debug log:
2019-10-28 10:06:51.053 [debug] <0.7362.0>@mod_bosh:process:63 Bad Request: no data

no-data’ refers to the <body/> contents. There is actually no trace of any pubsub prekeys data in the ejabberd debug log. It seems that the body data is not being sent at all.

  1. Instead of using the registered test accounts on my home server i.e. atalk.styes.net (with certificate signed by https://letsencrypt.org/); when switches to a registered account on dismail.de, so far they seem to have no problem.

No sure what else can I try. Appreciate some advice on this.

========= aTalk BOSH connection via mobile network =============

2019-10-28 11:14:23.763 32334-1543/org.atalk.android D/SMACK: SENT (1): 
    <body rid='4370261655789056' ack='4370261655789054' sid='f7ee00539b64d8961421a28b33508d71998a10ca' xmlns='http://jabber.org/protocol/httpbind'>
      <iq xmlns='jabber:client' id='XXKUG-207' type='set'>
        <enable xmlns='urn:xmpp:carbons:2'/>
      </iq>
    </body>
2019-10-28 11:14:23.919 32334-1548/org.atalk.android D/SMACK: RECV (1): 
    <body xmlns='http://jabber.org/protocol/httpbind'>
      <iq xml:lang='en' to='swordfish@atalk.org/atalk' from='swordfish@atalk.org' type='result' id='XXKUG-205' xmlns='jabber:client'/>
    </body>
2019-10-28 11:14:23.935 32334-1548/org.atalk.android D/SMACK: RECV (1): 
    <body xmlns='http://jabber.org/protocol/httpbind'>
      <iq xml:lang='en' to='swordfish@atalk.org/atalk' from='swordfish@atalk.org' type='result' id='XXKUG-206' xmlns='jabber:client'/>
    </body>
2019-10-28 11:14:18.996 32334-1528/org.atalk.android D/SMACK: SENT (1): 
    <body rid='55246617515818' sid='eab2f64eefbae9810348abdbe1b7745ea4e3fb0c' xmlns='http://jabber.org/protocol/httpbind'>
      <iq xmlns='jabber:client' id='XXKUG-194' type='set'>
        <pubsub xmlns='http://jabber.org/protocol/pubsub'>
          <publish node='eu.siacs.conversations.axolotl.bundles:1175061843'>
            <item xmlns='http://jabber.org/protocol/pubsub'>
              <bundle xmlns='eu.siacs.conversations.axolotl'>
                <signedPreKeyPublic signedPreKeyId='7'>
                  BYO0TfZBS5Y309M6oYcWqc6mtukZoD8Vou0qRg4WzdRT
                </signedPreKeyPublic>
                <signedPreKeySignature>
                  qWok+fvQjhOm+7nP0LLPqFpcVZ7f6ls41gURHroFqVpBbBNhNQUCSBpS7l/Odnu/wcQxpggsTanVFJXkCHOICw==
                </signedPreKeySignature>
                <identityKey>
                  BZzrSoMmL0L29FVpJFVO8UrjE7UAYNYWU5pzSYNQDn1T
                </identityKey>
                <prekeys>
                  <preKeyPublic preKeyId='1'>
                    BWnG9r6a2SNH8qvfcL9u+RIdXvOPt1ITHyH/cE5UcLNi
                  </preKeyPublic>
                  <preKeyPublic preKeyId='2'>
                    BS/Az4hezhq+cqXYIciFiqqjQ4GeuvRavvtK7eecubBu
                  </preKeyPublic>
...
                  <preKeyPublic preKeyId='23'>
                    BWo3LzQ1Wi9Yxj+1e1lbREdcAnJ8tIVzXDLOKuKlvSlC
                  </preKeyPublic>
                  <preKeyPublic preKeyId='24'>
                    BRS1eE/9pfADbcjzdpYC71zydVvFiCQ+Py2eA2sndjpf
2019-10-28 11:14:18.996 32334-1528/org.atalk.android D/SMACK: SENT (1):
2019-10-28 11:14:19.049 32334-1522/org.atalk.android V/aTalk: [20365] org.igniterealtime.jbosh.BOSHClient.processExchange() Could not obtain response
    org.igniterealtime.jbosh.BOSHException: Could not parse body:
    <?xml version='1.0'?>
    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
    <h1>400 Bad Request</h1>
        at org.igniterealtime.jbosh.BodyParserXmlPull.parse(BodyParserXmlPull.java:132)
        at org.igniterealtime.jbosh.StaticBody.fromString(StaticBody.java:114)
        at org.igniterealtime.jbosh.ApacheHTTPResponse.awaitResponse(ApacheHTTPResponse.java:246)
        at org.igniterealtime.jbosh.ApacheHTTPResponse.getBody(ApacheHTTPResponse.java:192)
        at org.igniterealtime.jbosh.BOSHClient.processExchange(BOSHClient.java:1124)
        at org.igniterealtime.jbosh.BOSHClient.processMessages(BOSHClient.java:1000)
        at org.igniterealtime.jbosh.BOSHClient.access$300(BOSHClient.java:102)
        at org.igniterealtime.jbosh.BOSHClient$RequestProcessor.run(BOSHClient.java:1729)
        at java.lang.Thread.run(Thread.java:764)
     Caused by: java.lang.IllegalStateException: Root element was not 'body' in the 'http://jabber.org/protocol/httpbind' namespace.  (Was 'h1' in '')
        at org.igniterealtime.jbosh.BodyParserXmlPull.parse(BodyParserXmlPull.java:98)
        at org.igniterealtime.jbosh.StaticBody.fromString(StaticBody.java:114) 
        at org.igniterealtime.jbosh.ApacheHTTPResponse.awaitResponse(ApacheHTTPResponse.java:246) 
        at org.igniterealtime.jbosh.ApacheHTTPResponse.getBody(ApacheHTTPResponse.java:192) 
        at org.igniterealtime.jbosh.BOSHClient.processExchange(BOSHClient.java:1124) 
        at org.igniterealtime.jbosh.BOSHClient.processMessages(BOSHClient.java:1000) 
        at org.igniterealtime.jbosh.BOSHClient.access$300(BOSHClient.java:102) 
        at org.igniterealtime.jbosh.BOSHClient$RequestProcessor.run(BOSHClient.java:1729) 
        at java.lang.Thread.run(Thread.java:764) 
2019-10-28 11:14:24.064 32334-1547/org.atalk.android V/aTalk: [20384] org.igniterealtime.jbosh.BOSHClient.processMessages() Processing thread exiting: 0
2019-10-28 11:14:24.068 32334-1554/org.atalk.android W/(AndroidOmemoService.java:125)#initializationFailed: OmemoManager init failed: The connection XMPPBOSHConnection[swordfish@atalk.org/atalk] (1) is no longer connected while waiting for response with IQReplyFilter: iqAndIdFilter (AndFilter: (OrFilter: (IQTypeFilter: type=error, IQTypeFilter: type=result), StanzaIdFilter: id=XXKUG-218)), : fromFilter (OrFilter: (FromMatchesFilter (full): null, FromMatchesFilter (ignoreResourcepart): swordfish@atalk.org, FromMatchesFilter (full): atalk.org)) because of org.igniterealtime.jbosh.BOSHException: Could not parse body:
    <?xml version='1.0'?>
    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
    <h1>400 Bad Request</h1>

Below are the tests on Bosh Connection carried out on 3 android devices i.e. Note-10 (API-28), Note-5 (API-21) and SM-J730GM (API-28) and the observations:

The size of the pubsub prekeys stanza is 9477 bytes with 100 prekeys, or a total of 9637 with https headers added.

Note-10 (API-28):
#. Server is atalk.sytes.net with letsencrypt certification

  1. Always passed when connected via home WiFi network
  2. 100% failure when connected via mobile network
  3. 100% failure when connected to WiFi hot-spot on Note-5 via mobile network
  4. High rate of success when reduce the prekeys to size of 8 (size ~ 1Kbytes) in #2 test condition.
  5. Note-10 does not allow http scheme transport, so unable to perform http scheme connection.

#. When server is dismail.de
6. Pass when connected to dismail.de

SM-J730GM (API-28)
Same results as Note-10

Note-5 (API-21)

  1. pass when tested with dismial.de with WiFi or mobile network (https)
  2. android denies the atalk.sytes.net letsencrypt certification (https), so cannot test https connection

Changed ejabberd server to use http scheme i.e. no TLS encryption - clear-text connection
3. Pass with atalk.sytes.net when connected via mobile network or WiFi network
4. Pass with atalk.sytes.net when connected to WiFi hotspot on Note-10 via mobile network
On wireshark: It is seen that the pubsub prekeys stanza is segmented to 7 packets for tcp transfer

Conclusion:
a. The failure does not seems to come from mobile network
b. When connected using XMPPTCPConnection, androids will throws certifications exception. aTalk catches and allows user to manual accept the custom certificate.
c. It seems that the XMPPBOSHconnection problem is due to letsencrypt certification; though android accepts and allows small amount of data transfer (single tcp packet). It refuses and blocks large amount of data transfer when sent via mobile network.
d. I am unable to figure out how much data is being transferred before android actually blocks further data transfer; as the data is encrypted when view on wireshark.

Can aTalk catches the certification exception during XMPPBOSHConnection, to allow user manual acceptance?

Please provide a brief instruction on how and where to do this if this is possible, Would like to try out on aTalk.

Please see reply from ejabberd team on ‘400 Bad Request’

Ejabberd bosh reports ‘400 Bad Request’ with incorrect stanza format to the bosh client #3073