File Transfer fails on Smack 4.1.0-beta 1

NOTE: This question is also posted on StackOverflow. Flow posted a link to the specs, which I have read. I have received no useful responses from that forum, which is why I am posting my question here.

I am working on an application involving two Smack clients, one of which attempts to send a jar file to the other client. I have read the specs for XEP-0096: SI File Transfer, XEP-0095: Stream Initiation, XEP-0047: In-Band Bytestreams, and XEP-0065: SOCKS5 Bytestreams. From what I can tell from the XML stanzas, it looks like everything is being done correctly.

I am using Openfire 3.9.3 as the XMPP server. For all code examples I am using the following Smack libraries (version 4.1.0- beta 1): smack-java7, smack-tcp, smack-extensions, and smack-sasl-provided. Here’s the code for the OutgoingFileTransfer (note in all code examples info such as Smack usernames have been anonymized):

// LOG is of type org.apache.logging.log4j.Logger

// transferManager is the FileTransferManager for this XMPPTCPConnection

private void sendJarFile(final String to)

{

    LOG.info("Sending " + jarFileName + " to " + to);

    OutgoingFileTransfer jarTransfer =

            transferManager.createOutgoingFileTransfer(to);

    final File jarFile = new File(jarFileName);

    try

    {

        jarTransfer.sendFile(jarFile, "The current jar file");

        while (!jarTransfer.isDone())

        {

            LOG.info("File transfer status: " + jarTransfer.getStatus());

            Thread.sleep(500);

        }

        LOG.info("File transfer to " + to + " is done");

        // Now that the file transfer is done check for errors

        // or exceptions

        FileTransfer.Error error = jarTransfer.getError();

        if (error != null)

        {

            LOG.error(error.getMessage());

        }

        Exception exception = jarTransfer.getException();

        if (exception != null)

        {

            if (exception instanceof XMPPException.XMPPErrorException)

            {

                XMPPException.XMPPErrorException errorException =

                        (XMPPException.XMPPErrorException)exception;

                XMPPError xmppError = errorException.getXMPPError();

                LOG.error(xmppError);

                LOG.error("Descriptive text: " + xmppError.getDescriptiveText());

                LOG.error("Condition: " + xmppError.getCondition());

                LOG.error("Type: " + xmppError.getType());

            }

        }

    }

    // For now, just catching and logging exceptions. Exception handling

    // will be added in top-level classes

    catch (SmackException e)

    {

        LOG.error("Exception trying to send jar file", e);

    }

    catch (InterruptedException e)

    {

        // Do nothing

    }

    catch (Exception e)

    {

        LOG.error("Exception trying to send jar file", e);

    }

}

The output from this code is:

Sending test.jar to receiver@local_openfire/Smack

14:09:43.748 INFO - File transfer status: Initial

14:09:44.249 INFO - File transfer status: Negotiating Stream

14:09:44.751 INFO - File transfer status: Negotiating Stream

// This message continues for several seconds, until finally

14:09:53.805 INFO - File transfer status: Negotiating Stream

14:09:54.308 INFO - File transfer to receiver@local_openfire/Smack

is done

14:09:54.309 ERROR - org.jivesoftware.smack.packet.XMPPError@30e95075

14:09:54.310 ERROR - Descriptive text: null

14:09:54.310 ERROR - Condition: service-unavailable

14:09:54.310 ERROR - Type: cancel

For the IncomingFileTransfer, the code is:

@Override

public void fileTransferRequest(FileTransferRequest request)

{

    final String requestorId = request.getRequestor();

    LOG.info("FileTransferRequest from: " + requestorId);

    // Only respond to requests from the sender

    if (requestorId.contains(senderId))

    {

        final IncomingFileTransfer transfer = request.accept();

        LOG.info("FileTransferRequest accepted");

        try

        {

            final String fileName = transfer.getFileName();

            transfer.recieveFile(new File(fileName));

            LOG.info("Incoming file transfer: " + fileName);

            LOG.info("Transfer status is: " + transfer.getStatus());

            while (!transfer.isDone())

            {

                final double progress = transfer.getProgress();

                final double progressPercent = progress * 100.0;

                String percComplete = String.format("%1$,.2f", progressPercent);

                LOG.info("Transfer status is: " + transfer.getStatus());

                LOG.info("File transfer is " + percComplete + "% complete");

                Thread.sleep(1000);

            }

            // Now that the file transfer is done check for errors

            // or exceptions

            FileTransfer.Error transferError = transfer.getError();

            if (transferError != null)

            {

                LOG.error("Transfer error occurred: " + transferError.getMessage());

            }

            Exception transferException = transfer.getException();

            if (transferException != null)

            {

                LOG.error("Transfer exception occurred: " + transferException);

                if (transferException instanceof SmackException.NoResponseException)

                {

                    SmackException.NoResponseException smackException = (SmackException.NoResponseException)transferException;

                    smackException.printStackTrace();

                }

            }

            LOG.info("FileTransfer complete");

            provisioningComplete = true;

        }

        // For now just logging exceptions

        catch (SmackException e)

        {

            LOG.error("SmackException trying to receive jar file", e);

        }

        catch (InterruptedException e)

        {

            // Do nothing

        }

        catch (IOException e)

        {

            LOG.error("IOException trying to receive jar file", e);

        }

    }

    else

    {

        LOG.warn("FileTransferRequest rejected");

        try

        {

            request.reject();

        }

        catch (NotConnectedException e)

        {

            LOG.warn("NotConnectedException when rejecting FileTransferRequest");

        }

    }

}

The output from this code is:

14:09:43.766 INFO - FileTransferRequest from:

sender@local_openfire/Smack

14:09:43.767 INFO - FileTransferRequest accepted

14:09:43.768 INFO - Incoming file transfer: test.jar

14:09:43.769 INFO - Transfer status is: Negotiating Transfer

14:09:43.770 INFO - Transfer status is: Negotiating Stream

14:09:43.770 INFO - File transfer is 0.00% complete

14:09:44.771 INFO - Transfer status is: Negotiating Stream

14:09:44.771 INFO - File transfer is 0.00% complete

14:09:45.776 INFO - Transfer status is: Negotiating Stream

14:09:45.776 INFO - File transfer is 0.00% complete

14:09:46.778 INFO - Transfer status is: Negotiating Stream

14:09:46.778 INFO - File transfer is 0.00% complete

14:09:47.782 INFO - Transfer status is: Negotiating Stream

14:09:47.783 INFO - File transfer is 0.00% complete

14:09:48.784 ERROR - Transfer exception occurred:

org.jivesoftware.smack.SmackException: Error in execution

14:09:48.784 INFO - FileTransfer complete

After this code has run, on the receiver end, I have a file named “test.jar” in the current working directory with a file size of 0 bytes. I’ve tried this both with the sender and receiver on different machines, and the sender and receiver on the same machine. I was initially using Smack 4.0.6, but switched to the latest code base (4.1.0- beta 1 as of this writing) in hopes that perhaps this bug would have been resolved.

I used the Smack debugging tool and captured the raw XML stanzas. For the sake of brevity, I only included those that were relevant to file transfer (i.e, not presence or roster packets). Here they are:

<iq to=“receiver@smack_server/Smack” id=“NK8Lh-11” type=“set”

from=“sender@smack_server/Smack”>

    <si xmlns="[http://jabber.org/protocol/si](http://jabber.org/protocol/si)" id="jsi_3077759398544954943"

    mime-type="text/plain" profile="[http://jabber.org/protocol/si/profile/file-transfer](http://jabber.org/protocol/si/profile/file-transfer)">

        <file xmlns="[http://jabber.org/protocol/si/profile/file-transfer](http://jabber.org/protocol/si/profile/file-transfer)" name="test.txt"

size=“37”>

A test file

        <feature xmlns="[http://jabber.org/protocol/feature-neg](http://jabber.org/protocol/feature-neg)">
                        <value>[http://jabber.org/protocol/bytestreams](http://jabber.org/protocol/bytestreams)</value>
                        <value>[http://jabber.org/protocol/ibb](http://jabber.org/protocol/ibb)</value>
  <si xmlns="[http://jabber.org/protocol/si](http://jabber.org/protocol/si)">

    <feature xmlns="[http://jabber.org/protocol/feature-neg](http://jabber.org/protocol/feature-neg)">
          <value>[http://jabber.org/protocol/bytestreams](http://jabber.org/protocol/bytestreams)</value>

          <value>[http://jabber.org/protocol/ibb](http://jabber.org/protocol/ibb)</value>
    <query xmlns="[http://jabber.org/protocol/disco#info](http://jabber.org/protocol/disco#info)"/>
  <query xmlns="[http://jabber.org/protocol/disco#info](http://jabber.org/protocol/disco#info)">
    <feature var="[http://jabber.org/protocol/disco#items](http://jabber.org/protocol/disco#items)"/>
    <feature var="[http://jabber.org/protocol/bytestreams](http://jabber.org/protocol/bytestreams)"/>

    <feature var="[http://jabber.org/protocol/ibb](http://jabber.org/protocol/ibb)"/>

    <feature var="[http://jabber.org/protocol/si](http://jabber.org/protocol/si)"/>

    <feature var="[http://jabber.org/protocol/xhtml-im](http://jabber.org/protocol/xhtml-im)"/>
    <feature var="[http://jabber.org/protocol/si/profile/file-transfer](http://jabber.org/protocol/si/profile/file-transfer)"/>
    <feature var="[http://jabber.org/protocol/commands](http://jabber.org/protocol/commands)"/>

    <feature var="[http://jabber.org/protocol/muc](http://jabber.org/protocol/muc)"/>

    <feature var="[http://jabber.org/protocol/xdata-validate](http://jabber.org/protocol/xdata-validate)"/>

    <feature var="[http://jabber.org/protocol/xdata-layout](http://jabber.org/protocol/xdata-layout)"/>

    <feature var="[http://jabber.org/protocol/disco#info](http://jabber.org/protocol/disco#info)"/>
    <query xmlns="[http://jabber.org/protocol/bytestreams](http://jabber.org/protocol/bytestreams)" sid="jsi_3077759398544954943" mode="tcp">
    <open xmlns="[http://jabber.org/protocol/ibb](http://jabber.org/protocol/ibb)" block-size="4096" sid="jsi_3077759398544954943" stanza="iq"/>

From the best that I can tell having read the spec, it looks like everything is proceeding as it should. The sender sends the initial SI request, the receiver responds with the supported protocols (i.e., byte streams and IBB), the send then queries the receiver for all the disco items, the receiver responds with the list of features, the sender then sends various stream hosts, and then the sender sends a block of data via IBB. From there, the receiver gets the SmackException: Error in execution method, which is caused by a SmackException.NoResponseException with a message that a response was not received in 5 seconds. It looks like this question is basically being overlooked at this point, but in the hopes that someone does check it out I’d really appreciate any help. Thanks!

If there is no response to the NK8Lh-26 IQ, then I suggest you look at the entity which is supposed to respond to the IQ. If you find out why it does not, then you are one step further.

Do you use Prosody or ejabberd server? I recently discovered strange behavior with these servers.

Prosody does not deliver larger stanzas (IBB data) until another stanza or whitespace is sent. (I am not yet sure, if it’s really a Prosody issue, but Openfire and ejabberd don’t behave like this, maybe it’s also a configuration in the deployment).

With ejabberd I saw similar strange behavior. IBB is very slow, it takes a few seconds for a 4KB chunk to get transferred, which is also strange.

But from your stack trace it seems that no packet is transferred at all. weird…

the receiver responds with the supported protocols (i.e., byte streams and IBB)

As a side note, this is actually a bug. SMACK-561

1 Like

I have tried many different configurations. I have Openfire 3.9.3 installed on Mac OS X (my primary machine), Ubuntu Server 12.04 LTS, and a Raspberry Pi running Raspbian. As far as devices running the code, I have one computer running Mac OS X Yosemite and another running Windows 7. All firewalls are off and the ports needed are available (7777 and 7778).I have tried running this app using each Openfire server, and have also tried running the app with both sender and receiver on the same machine, and sender and receiver on different machines. Regardless of which machine runs the sender and which runs the receiver, I encounter the same issue. I can’t figure out why the receiver is not sending the response to NK8Lh-26 IQ. Is there anything on the receiver side that needs to be done aside from calling FileTransferRequest.accept() and IncomingFileTransfer.receiveFile(). Yesterday I tried extending InBandBytestreamListener, as it seemed like the receiver might need to accept the bytes stream request. This did not resolve the issue. The other thing I find especially strange is that the SmackException.NoResponseException occurs on the receiver, while the sender is stuck in the Negotiating Stream state indefinitely. Since the receiver is not accepting the byte stream, it would seem like the sender would timeout. I should also note that I’ve tried this with FileTransferNegotiator.IBB_ONLY set to true and it made no difference.

I am using Openfire 3.9.3. After reading the spec, I realized that the receiver should only respond with one protocol. In order to eliminate this as a possible cause I set FileTransferNegotiator.IBB_ONLY to true on both the sender and receiver side. I then verified that sender and receiver both sent only IBB in the SI IQs.

Have you tried attaching a debugger and follow the trace of the request IQ on the receiver? This should give you an hint why it does not respond. Hint: It may be a good idea if you first think about who the intended recipient of the IQ request in Smack is. Then you can compare the actual route of the incoming IQ request, with the expected one (if they are not the same that is).

I’ve used a debugger in Eclipse and the Smack Debug Window. I see that the sender has this message under the “All Packets” menu as having been sent to receiver:

The message does not appear under the receiver’s “All Packets” menu, but does appear under “Raw Received Packets.” I even tried to implement PacketListener and used an IQTypeFilter, but I don’t see the IQs from the sender.

I honestly don’t know if you’re asking questions to help lead me to the correct answer, or if you actually know what’s going wrong but aren’t saying anything. You stated:

Hint: It may be a good idea if you first think about who the intended recipient of the IQ request in Smack is. Then you can compare the actual route of the incoming IQ request, with the expected one (if they are not the same that is).

I’m either completely lost or I don’t know what question you’re asking. My understanding of the process is this:

  1. Sender sends a SI to the receiver. In this IQ, there is a listed of supported protocols for sending data
  2. Receiver receives the SI IQ and responds with the chosen protocol (as CSH mentioned, it may respond with a list of protocols).
  3. The sender then sends an IQ with a block of data. It seems like the sender waits for the receiver to accept this block of data.

So my answer would be that the receiver should receiver an IQ Set with a block of data from the sender. In my case, the receiver never receives the IQ with a block of data from the sender. That is to say, according to the Smack Debugger Window, the IQ never appears in the “All Packets” menu, but does appear under “Raw Received Packets”. Translating the steps above into psuedo-code, when the sender calls sendFile on an object of type OutgoingFileTransfer, this is equivalent to Step 1 above. This creates a FileTransferRequest, and the receiver is notified of this request. The receiver the calls accept() on the FileTransferRequest and then calls receiveFile() on the IncomingFileTransfer (which I believe is equivalent to Step 2 above). From there, according to the Javadocs for the methods I’ve called, the Transfer Negotiation and Stream Negotiation are supposed to be taken care of. I’ve literally tried every method for sending a file on the OutgoingFileTransfer object and every method for receiving a file on the IncomingFileTransfer object, all to no avail. If I’m missing something really obvious or just being thick I have no problem with you saying so.

I honestly don’t know if you’re asking questions to help lead me to the correct answer, or if you actually know what’s going wrong but aren’t saying anything.
I’m sorry, I hope I didn’t gave the impression that I’m hiding some knowledge from you. I honestly have no idea what’s going wrong. I’m only trying to point you into the right direction so that we can solve your issue. And your posts shows that this works to a certain degree.

I think you are focusing to much on the Smack Debugger. You should use a Java Debugger to follow the trace of the IQ request on the receiver side. See where it gets handled. beta2 should always send a response to an IQ. Such a response is missing in your provided traces. That’s all I know.

Thanks for the clarification. I see that you are quite active on the forum and thought you might be used to dealing with people who haven’t really looked into the documentation before posting questions, so I thought you might be trying to get me to do a little more work on my end. Anyways, I will take your suggestion and dive in to the Java debugger.

I thought you might be trying to get me to do a little more work on my end.
Well that’s indeed basically what I’m trying to do. See, I can’t obviously debug your problem. And even if I could somehow via some remote technology, I don’t have the time to do so. Also given that both Smack is open source and XMPP an open standard, nothing prevents you from solving those issues yourself (and sending a patch to me )

Here’s the latest update. Using a debugger, I dove into the code for IncomingFileTransfer.receiveFile() and OutgoingFileTransfer.sendFile(). Everything seems to be going as it should. I finally came upon a realization: The IQ

    <open xmlns="[http://jabber.org/protocol/ibb](http://jabber.org/protocol/ibb)" block-size="4096" sid="jsi_3077759398544954943" stanza="iq"/>

showed up in the receiver’s “Raw Received Packets”, but not “All Packets.” This means it received the IQ but didn’t know how to parse it. Sure enough, there was not an IQProvider for the open element or "http://jabber.org/protocol/ibb" namespace. So I added one (ProviderManager.addIQProvider()) and while I was at it, added Providers for “data” and “close” as well. I restarted the application and verified that the Providers were added. However, the open IQ is still not being parsed by the receiver. I made sure to use the constants defined for the element and namespace (i.e., Open.ELEMENT, Open.NAMESPACE, etc.) in order to avoid a typo. This is the root cause of the issue. The sender actually times out, but for some reason doesn’t have an Error or Exception status, and the receiver times out and does have an Exception status, all because the open IQ is never parsed by the receiver. Is this possibly a known bug or is there something else I can look into to verify that all the Providers were added correctly? Thanks!

You could check if a provider is available for the open IQ by calling ProviderManager.getIQProvider(“open”, “http://jabber.org/protocol/ibb”) and see if it returns a non null value. Providers get configured by Smack’s initialization process.

The provider itself is unit tested: OpenIQProviderTest

Even if no provider was in place, then an UnparsedIQ should get created and Smack should answer with an error IQ (at least using beta2).

Therefore, I’m not sure if this is really the root cause. But you appear to be on the right track.

The receiver does not send an Error IQ, it doesn’t respond to the open IQ at all. I reverted back to 4.0.6 last night and found that now the Open IQ Packets are being parsed and the receiver is responding to them. Now it appears (at least in 4.0.6) that the Data IQs are not being parsed. The sender sends an IQ with bytes, but the IQ ends up in the receiver’s “Raw Received Packets” bin but is not parsed.

I have resolved the issue! After downloading the Smack 4.0.6 source code and debugging the packet parsing on the Receiver side, I found that everything was being done correctly. The IQs and Packets were being parsed, and the XMPPTCPConnection was correctly handling the processing of the packets. The issue turned out to be some kind of race condition. I believe the problem was here:

    // transfer is of type IncomingFileTransfer, created by

    // FileTransferRequet.accept()

    final String fileName = transfer.getFileName();

    transfer.recieveFile(new File(fileName));

    LOG.info("Incoming file transfer: " + fileName);

    LOG.info("Transfer status is: " + transfer.getStatus());

    while (!transfer.isDone())

    {

        final double progress = transfer.getProgress();

        final double progressPercent = progress * 100.0;

        String percComplete = String.format("%1$,.2f", progressPercent);

        LOG.info("Transfer status is: " + transfer.getStatus());

        LOG.info("File transfer is " + percComplete + "% complete");

        Thread.sleep(1000);

    }

For some reason, the while loop was consuming all the cycles and the XMPPTCPConnection was not able to respond to the Open IQs and Data IQs in a timely manner. So I moved the progress monitoring into a new Thread, and everything worked perfectly. For reference, I was using Java version 1.8.0_31 (64-bit) on Mac OS X and also tested on Java version 1.8.0_31 (64-bit) on Windows 7 Professional (64-bit).

Using 4.0.6 or 4.1.0-beta2?

Good question. It was on 4.0.6. I haven’t tried running the app with 4.1.0-beta 1 as I refactored the code to use 4.0.6 last night and have decided to stick with that version for now. Given what I saw when I was debugging with 4.1.0- beta 1, it looks like it may have been the same (or a similar) issue. I also verified that it works correctly with Java 7 (1.7.0_51) 64-bit.

Hmm, receiveFile starts a new thread to receive the data. I also found nothing that would explain why adding the monitoring into a new thread would “unblock” the thread somehow.

Glad that it did work for you, though we still have no clue about what went wrong.

Using smack-4.1.0-beta2-SNAPSHOT-2015-01-14:

Debugging/single-stepping into the receiveFile, I have observed streamNogotiatorTask,get(…) throws an ExecutionException “Error in execution”. To contrast, when using PSI client to receive and negotiate the exact same file transfer sent by beta2 Sender, PSI functions without error.

As another data point, I tried using 4.1.0-rc2 in my app and the file transfer failed. Specifically, the sender encountered a SmackException.NoResponseException No response received within reply timeout. Timeout was 5000ms. All configurations remained the same with only the Smack library being different.

Everything works as expected with 4.0.7