Here is everything logged by Openfire around the time of one of these lost/missed messages:
2016.08.30 10:28:41 WARN [pool-29-thread-4]: org.jivesoftware.openfire.net.SocketUtil - Unable to create a socket connection to XMPP domain 'Domain' using remote host: Domain:5269. Cause: Connection refused: connect (a full stacktrace is logged on debug level)
2016.08.30 10:28:41 WARN [pool-29-thread-4]: org.jivesoftware.openfire.net.SocketUtil - Unable to create a socket connection to XMPP domain 'Domain': Unable to connect to any of its remote hosts.
2016.08.30 10:28:41 INFO [pool-29-thread-4]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Create outgoing session for: OpenFireServer to Domain] - Unable to create new session: Cannot create a plain socket connection with any applicable remote host.
2016.08.30 10:28:41 WARN [pool-29-thread-4]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Authenticate local domain: 'OpenFireServer' to remote domain: 'Domain'] - Unable to authenticate: Fail to create new session.
2016.08.30 10:30:48 WARN [pool-29-thread-5]: org.jivesoftware.openfire.net.SocketUtil - Unable to create a socket connection to XMPP domain 'Domain' using remote host: Domain:5269. Cause: Connection refused: connect (a full stacktrace is logged on debug level)
2016.08.30 10:30:48 WARN [pool-29-thread-5]: org.jivesoftware.openfire.net.SocketUtil - Unable to create a socket connection to XMPP domain 'Domain': Unable to connect to any of its remote hosts.
2016.08.30 10:30:48 INFO [pool-29-thread-5]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Create outgoing session for: OpenFireServer to Domain] - Unable to create new session: Cannot create a plain socket connection with any applicable remote host.
2016.08.30 10:30:48 WARN [pool-29-thread-5]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Authenticate local domain: 'OpenFireServer' to remote domain: 'Domain'] - Unable to authenticate: Fail to create new session.
2016.08.30 10:33:49 WARN [pool-29-thread-1]: org.jivesoftware.openfire.net.SocketUtil - Unable to create a socket connection to XMPP domain 'Domain' using remote host: Domain:5269. Cause: Connection refused: connect (a full stacktrace is logged on debug level)
2016.08.30 10:33:49 WARN [pool-29-thread-1]: org.jivesoftware.openfire.net.SocketUtil - Unable to create a socket connection to XMPP domain 'Domain': Unable to connect to any of its remote hosts.
2016.08.30 10:33:49 INFO [pool-29-thread-1]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Create outgoing session for: OpenFireServer to Domain] - Unable to create new session: Cannot create a plain socket connection with any applicable remote host.
2016.08.30 10:33:49 WARN [pool-29-thread-1]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Authenticate local domain: 'OpenFireServer' to remote domain: 'Domain'] - Unable to authenticate: Fail to create new session.
Guus, correct me if Iām wrong, but that looks like a DNS and S2S issue. my guess is the client is using a cname or something as āserverā address, instead of the xmpp domain, which is prob causing the issue
Here are all of the raw packets received today when I received a message from a user, Jill, that didnāt display although the chat window opened and flashed. You can see in the data below that the message itself did come in to spark, it just wasnāt displayed.
<message to="gretchen@OpenfireServer" id="8h9tm-613" type="chat" from="jill@OpenfireServer/Spark 2.6.3"><thread>eSYGso</thread><composing xmlns="http://jabber.org/protocol/chatstates"/></message>
<message to="gretchen@OpenfireServer" id="8h9tm-614" type="chat" from="jill@OpenfireServer/Spark 2.6.3"><thread>eSYGso</thread><composing xmlns="http://jabber.org/protocol/chatstates"/></message>
<message to="gretchen@OpenfireServer" id="8h9tm-615" type="chat" from="jill@OpenfireServer/Spark 2.6.3"><thread>eSYGso</thread><composing xmlns="http://jabber.org/protocol/chatstates"/></message>
<message to="gretchen@OpenfireServer" id="8h9tm-616" type="chat" from="jill@OpenfireServer/Spark 2.6.3"><thread>eSYGso</thread><composing xmlns="http://jabber.org/protocol/chatstates"/></message>
<message to="gretchen@OpenfireServer" id="8h9tm-617" type="chat" from="jill@OpenfireServer/Spark 2.6.3"><body>i am having some printer issues - could you pls take a look?</body><thread>eSYGso</thread><x xmlns="jabber:x:event"><offline/><composing/></x><active xmlns="http://jabber.org/protocol/chatstates"/></message>
<message to="gretchen@OpenfireServer" id="8h9tm-619" type="chat" from="jill@OpenfireServer/Spark 2.6.3"><thread>eSYGso</thread><paused xmlns="http://jabber.org/protocol/chatstates"/></message>
<message to="gretchen@OpenfireServer" id="8h9tm-620" type="chat" from="jill@OpenfireServer/Spark 2.6.3"><thread>eSYGso</thread><gone xmlns="http://jabber.org/protocol/chatstates"/></message>
<iq type="get" id="89-27224" from="OpenfireServer" to="gretchen@OpenfireServer/Spark 2.6.3"><ping xmlns="urn:xmpp:ping"/></iq>
Although I doubt that it relates to the issue at hand, each of these log messages are all relate to pretty much the same thing: Your Openfire (which is running the XMPP domain āOpenFireServerā), is trying, and failing, to create a connection to another server (might be yours, might be someone elseās, might not exist at all) that is running the XMPP domain āDomainā.
A common cause for your Openfire to try and connect to another server is to allow users from both networks to talk to each-other. Assuming that your username is āgretchenā (which would make your JID āgretchen@OpenFireServerā), it is very possible that you want to chat with your friend named āoliverā thatās on another XMPP network, named āDomainā (Oliverās JID would be āoliver@Domainā). You could have done so by adding āoliver@Domainā to your contact list, after which your domain will attempt to make contact with Oliverās domain. These attempts are what we see failing in the log file that you provided.
Note that it is possible that you tried to add a contact to your contact list, but used an incorrect JID to do so. For instance, Oliver could have an account on the same XMPP domain as yours (eg: āoliver@OpenFireServerā), but you accidentally used āoliver@Domainā instead. Even if the XMPP domain āDomainā does not exist, or is currently not reachable, your domain will attempt to contact it.
The names of both XMPP domains are suspicious: āOpenFireServerā and āDomainā look odd. Did you search/replace these values before you posted the log files here? If not, your setup might need to be looked at - although you can probably make things work with these names, odd values like this can lead to confusion, and thus, problems.
I was somewhat able to reproduce the problem. Spark is very extensible in nature. All kinds of plugins can add functionality. Iāve found that Spark would stop processing messages completely, if one of the āadd-onsā would throw an unexpected problem. Iāve addressed this issue here: https://issues.igniterealtime.org/browse/SPARK-1798 and provided a fix here: SPARK-1798: Gracefully handle exceptions from event listeners. by guusdk Ā· Pull Request #215 Ā· igniterealtime/Spark Ā· Giā¦
With the fix for this issue, the root cause of the problem wonāt be fixed (I donāt know what it is), but the effect would now be isolated to that āadd-onā. It would no longer prevent further processing of the message.
Thereās a good chance that this will āfixā your problem (not being able to see an incoming message). As a side-effect, the root cause should now be logged in the error logs of Spark. That will allow us to address that too.
I did a search and replace to replace the name of the server that Open Fire is installed on to OpenFireServer, but Domain is just how it is in the logs. We have only one domain and our openfire server shouldnāt be trying to contact another server or domain. Iāll have to look into that to figure out what is going on.
Iāve got the latest build installed now. Here are the raw packets received from a message that again didnāt show in my chat window, but did make my chat window pop up and flash. The second message that she sent me, that you see at the end, did appear in the chat window:
Bah, I had hoped that your problem would have been resolved with the new build. Apparently not.
This new build did improve logging. Does this version write anything related to this issue in the Spark log files?
One last thing that you could try is to start Spark from the command line / dosshell. Thereās a chance that an error is printed to the console, instead of the error log. Perhaps that could give us a new insights.
I was hoping this would fix it too! But I suppose having the problem happen fairly consistently to me, and not very often to any of my users, is the best possible troubleshooting scenario.
Nothing relevant in the spark log files. Iāll give running spark from the command line a shot.
Iām running out of ideas, sadly. As long as thereās no way for me to reproduce this problem, itās going to be hard to debug it.
You could try switching off plugins that you might have installed, toggle some settings, see if that makes a difference. Perhaps youāre lucky and find a cause that way. If your users donāt have this problem, then whatās different between your client and theirs?
No, Iād be surprised if itās anything outside of Spark. Your client receives the message, it āsimplyā does not display it. Spark has plugins of its own - I was referring to those.