S2S not working, test fails in one direction

Openfire
1

hi, please forgive the double post, i will go back and close out the other thread. I have been working on this for months and cannot find the answer.
After an upgrade (several ago), i noticed that s2s (which was working) stopped. i have tried everything i know but i cannot get it working again. Both servers are currently on 4.5.1. here is everything i can think of that would be helpful.
any help would be greatly appreciated.
the 2 sites involved are uedi-gbs.com and ultra-fei.com…both chat servers are centos 7 currently patched. i am using a CA issued wild card cert on each.
in short, in one direction, i get this error from the test tool
XMPP:

<iq type="error" id="267-26868" to="ultra-fei.com" from="uedi-gbs.com"><ping xmlns="urn:xmpp:ping"/><error code="404" type="cancel"><remote-server-not-found xmlns="urn:ietf:params:xml:ns:xmpp-stanzas"/></error></iq>

logs:

Sending server to server ping request to uedi-gbs.com
Successful server to server response received.
Primary packet routing failed
org.jivesoftware.openfire.PacketException: Cannot route packet of type IQ or Presence to bare JID: <iq type="error" id="267-26868" to="ultra-fei.com" from="uedi-gbs.com"><ping xmlns="urn:xmpp:ping"/><error code="404" type="cancel"><remote-server-not-found xmlns="urn:ietf:params:xml:ns:xmpp-stanzas"/></error></iq>
	at org.jivesoftware.openfire.spi.RoutingTableImpl.routeToLocalDomain(RoutingTableImpl.java:306) ~[xmppserver-4.5.1.jar:4.5.1]
	at org.jivesoftware.openfire.spi.RoutingTableImpl.routePacket(RoutingTableImpl.java:239) [xmppserver-4.5.1.jar:4.5.1]
	at org.jivesoftware.openfire.server.OutgoingSessionPromise$PacketsProcessor.returnErrorToSender(OutgoingSessionPromise.java:343) [xmppserver-4.5.1.jar:4.5.1]
	at org.jivesoftware.openfire.server.OutgoingSessionPromise$PacketsProcessor.addPacket(OutgoingSessionPromise.java:361) [xmppserver-4.5.1.jar:4.5.1]
	at org.jivesoftware.openfire.server.OutgoingSessionPromise$1.run(OutgoingSessionPromise.java:134) [xmppserver-4.5.1.jar:4.5.1]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
Failed to establish server to server session.

Here is as much info as i could cram into this thread

uedi-gbs.com:
Server:
2020-04-14 11_50_58-Window
Server 2 Server Settings:
2020-04-14 11_55_00-Window
DNS:
2020-04-14 11_23_36-Window
dns lookups from gbs-vic-chat-1

[tonyg@uedi-gbs.com@gbs-vic-chat-1 ~]$ dig _xmpp-client._tcp.uedi-gbs.com. any +short
0 0 5222 gbs-vic-chat-1.uedi-gbs.com.
[tonyg@uedi-gbs.com@gbs-vic-chat-1 ~]$ dig _xmpp-server._tcp.uedi-gbs.com. any +short
0 5 5269 gbs-vic-chat-1.uedi-gbs.com.
[tonyg@uedi-gbs.com@gbs-vic-chat-1 ~]$ dig _xmpps-server._tcp.uedi-gbs.com. any +short
0 5 5269 gbs-vic-chat-1.uedi-gbs.com.
[tonyg@uedi-gbs.com@gbs-vic-chat-1 ~]$ dig  gbs-vic-chat-1.uedi-gbs.com. any +short
172.30.0.15

[tonyg@uedi-gbs.com@gbs-vic-chat-1 ~]$ dig _xmpp-client._tcp.ultra-fei.com. any +short
0 0 5222 fei-vic-chat-1.ultra-fei.com.
[tonyg@uedi-gbs.com@gbs-vic-chat-1 ~]$ dig _xmpp-server._tcp.ultra-fei.com. any +short
0 5 5269 fei-vic-chat-1.ultra-fei.com.
[tonyg@uedi-gbs.com@gbs-vic-chat-1 ~]$ dig _xmpps-server._tcp.ultra-fei.com. any +short
0 5 5269 fei-vic-chat-1.ultra-fei.com.
[tonyg@uedi-gbs.com@gbs-vic-chat-1 ~]$ dig fei-vic-chat-1.ultra-fei.com. any +short
172.30.0.14

server to server test FROM uedi-gbs.com TO ultra-fei.com (success!)
2020-04-14 11_44_19-Window

ultra-fei.com:
Server:
2020-04-14 11_50_38-Window
Server 2 Server Settings:
2020-04-14 11_55_36-Window
DNS:
2020-04-14 11_24_27-Window
DNS lookups from fei-vic-chat-1

[tonyg@ultra-fei.com@fei-vic-chat-1 ~]$ dig _xmpp-client._tcp.ultra-fei.com. any +short
0 0 5222 fei-vic-chat-1.ultra-fei.com.
[tonyg@ultra-fei.com@fei-vic-chat-1 ~]$ dig _xmpp-server._tcp.ultra-fei.com. any +short
0 5 5269 fei-vic-chat-1.ultra-fei.com.
[tonyg@ultra-fei.com@fei-vic-chat-1 ~]$ dig _xmpps-server._tcp.ultra-fei.com. any +short
0 5 5269 fei-vic-chat-1.ultra-fei.com.
[tonyg@ultra-fei.com@fei-vic-chat-1 ~]$ dig fei-vic-chat-1.ultra-fei.com. any +short
172.30.0.14
[tonyg@ultra-fei.com@fei-vic-chat-1 ~]$ dig _xmpp-client._tcp.uedi-gbs.com. any +short
0 0 5222 gbs-vic-chat-1.uedi-gbs.com.
[tonyg@ultra-fei.com@fei-vic-chat-1 ~]$ dig _xmpp-server._tcp.uedi-gbs.com. any +short
0 5 5269 gbs-vic-chat-1.uedi-gbs.com.
[tonyg@ultra-fei.com@fei-vic-chat-1 ~]$ dig _xmpps-server._tcp.uedi-gbs.com. any +short
0 5 5269 gbs-vic-chat-1.uedi-gbs.com.
[tonyg@ultra-fei.com@fei-vic-chat-1 ~]$ dig gbs-vic-chat-1.uedi-gbs.com. any +short
172.30.0.15

Server to Server test FROM ultra-fei.com TO uedi-gbs.com (fail!)
2020-04-14 11_47_25-Window

txt:
XMPP:

<iq type="error" id="267-26868" to="ultra-fei.com" from="uedi-gbs.com"><ping xmlns="urn:xmpp:ping"/><error code="404" type="cancel"><remote-server-not-found xmlns="urn:ietf:params:xml:ns:xmpp-stanzas"/></error></iq>

logs:

Sending server to server ping request to uedi-gbs.com
Successful server to server response received.
Primary packet routing failed
org.jivesoftware.openfire.PacketException: Cannot route packet of type IQ or Presence to bare JID: <iq type="error" id="267-26868" to="ultra-fei.com" from="uedi-gbs.com"><ping xmlns="urn:xmpp:ping"/><error code="404" type="cancel"><remote-server-not-found xmlns="urn:ietf:params:xml:ns:xmpp-stanzas"/></error></iq>
	at org.jivesoftware.openfire.spi.RoutingTableImpl.routeToLocalDomain(RoutingTableImpl.java:306) ~[xmppserver-4.5.1.jar:4.5.1]
	at org.jivesoftware.openfire.spi.RoutingTableImpl.routePacket(RoutingTableImpl.java:239) [xmppserver-4.5.1.jar:4.5.1]
	at org.jivesoftware.openfire.server.OutgoingSessionPromise$PacketsProcessor.returnErrorToSender(OutgoingSessionPromise.java:343) [xmppserver-4.5.1.jar:4.5.1]
	at org.jivesoftware.openfire.server.OutgoingSessionPromise$PacketsProcessor.addPacket(OutgoingSessionPromise.java:361) [xmppserver-4.5.1.jar:4.5.1]
	at org.jivesoftware.openfire.server.OutgoingSessionPromise$1.run(OutgoingSessionPromise.java:134) [xmppserver-4.5.1.jar:4.5.1]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
Failed to establish server to server session.
2

You’re requiring mutual authentication on both servers. Although I don’t think that this is the problem (as it works one-way), it’s an uncommon settings. Do you explicitly want cert-based authentication?

3

hi, i just disabled mutual auth on both servers and retested, same result

4

Try removing the DNS SRV records for _xmpps-server (and leave the ones for _xmpp-server).

If you want to use _xmpps-server you should point them at port 5270, not 5269, as configured here:

image

For now, try disabling that legacy-mode type s2s connection as well as removing the _xmpps-server record.

5

with having both xmpps-server as well as xmpp-server records, Openfire will arbitrarily* pick one. If one of them is misbehaving, this will lead to very random errors.

* arbitrary in so far DNS results are arbitrary - there is probably some caching going on, etc.

6

Tonyg: I had the same issue and found that the SSL cert was not built correctly with the intermediate certs. Once I manually built the certificate and didn’t use the p7b file it worked. Some SSL certs have 2 intermediate certificates. You must load them all in the correct order to get them to work. I will check where I saved the document I made that has the steps needed. I think it is the same as apache if I remember correctly.

7

Can you get me a keystore or a certificate chain to reproduce that problem? I’ve put in code that explicitly tries to sort unsorted chains.

8

guus, i removed the xmpps record, cleared cache on the dns servers, restarted both chat servers. same results.

9

i guess this could be the issue, i thought i was pretty careful to chain properly, however, i might have made a mistake…how can i tell?

10

another comment about chains. i normally leave out the root cert in the chain and just put in the intermediaries. In the past, cert audits call this out as being “bad form”. should i anchor the cert chain in this case?

11

It should not be needed, but I generally do.

12

ok, so i re-added the identity cert chain to both sides and this time i included the root cert. i restarted httpd and it did not fix the issue. here is the identity cert chain i entered (not including the private key obviously)…is there anyway you can check to make sure i did this right?

uedi-gbs.com-2019.pem (8.1 KB) ultra-fei.com-2020.pem (8.1 KB)

13

any idea’s?

much appreciated!

14

bump