Openfire 4.0.1 - Cannot connect to jabber.org

Michael42 · February 16, 2016, 12:55am

Hello,

I’ve been using previous versions of Openfire to connect to other Openfire servers as well as to a jabber.org server without any issues. However, once I upgraded from Openfire version 3.10.2 to 4.0.1, I ran into some problems. The two Openfire servers I upgraded were unable to authenticate to the jabber.org server. Under “server sessions”, it only showed an incoming session stream from jabber.org, but no outgoing sessions. Here is what the log showed:

2016.02.12 20:11:40 org.jivesoftware.openfire.session.LocalOutgoingServerSession[Create outgoing session for: our-domain.net to jabber.org] - Unable to create a new session: exhausted all options (not trying dialback as a fallback, as server dialback is disabled by configuration.

2016.02.12 20:11:40 org.jivesoftware.openfire.session.LocalOutgoingServerSession[Authenticate local domain: ‘our-domain.net’ to remote domain: ‘jabber.org’] - Unable to authenticate: Fail to create new session.

2016.02.12 20:12:27 org.jivesoftware.openfire.spi.LegacyConnectionAcceptor - Configuration allows for up to 16 threads, although implementation is limited to exactly one.

Here is what is displayed under “server sessions”:

**Remote Server Connections Details **

Below are details about the sessions with the remote server .

**Remote Server Connections Details **

Connection

Incoming

Remote server
IP / Hostname:

208.68.163.218
/ 208.68.163.218

Incoming Session Details

Stream ID

Authentication

Cipher Suite

Creation Date

Last Activity

Packets RX

Packets TX

aw5s84p3tb

Dialback

TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

7:55 PM

1

0

Both Openfire servers are configured to use self-signed certificates. I have the following parameters configured:

xmpp.server.cert.policy = disabled

xmpp.server.certificate.accept-self = true

xmpp.server.certificate.verify = false

xmpp.server.dialback.enabled = false

xmpp.server.tls.enabled = true

xmpp.server.tls.policy = optional

xmpp.socket.ssl.active = true

xmpp.socket.ssl.certificate.accept = true

xmpp.socket.ssl.certificate.verify = false

Cipher suites and protocols are left at their defaults. When I sign into my jabber.org account using Spark, I am unable to see any users on the Openfire servers and they are unable to see me. If I switch back to Openfire version 3.10.2, then bidirectional communication between the Openfire servers and jabber.org works just fine. Clearly, something has changed in the S2S code between those versions. Is anyone else having this issue or is it just me? ;-(

Any help is greatly appreciated. Thank you all!

Michael

guus · February 16, 2016, 8:30am

TLS management was indeed changed a lot in 4.0.0. Some functional behavior was likely introduced. You appear to suffer from that.

At the very least, the XMPP domain at igniterealtime.org can successfully federate with jabber.org. That makes it likely that the problem is related to configuration, and not a programming bug (that’s the good news)

When you enable debug logging, and look at the ‘all.log’ log file, you’ll most likely find a lot more data. Any clues in that?

My first thought is that the TLS certificate chain validation fails. The most obvious reason for this would be that your instance of Openfire does no have a (valid) root CA certificate for Jabber.org.

Assuming that jabber.org uses the same certificate for s2s validation as it does for c2s, below is the certificate chain for jabber.org. Can you verify that your s2s truststore contains the required startcom certificates (and that those certificates are still valid)?

$ openssl s_client -connect jabber.org:5222 </dev/null -starttls xmpp
CONNECTED(00000003)
depth=2 C = IL, O = StartCom Ltd., OU = Secure Digital Certificate Signing, CN = StartCom Certification Authority
verify error:num=19:self signed certificate in certificate chain
verify return:0
---
Certificate chain 0 s:/C=US/CN=conference.jabber.org/emailAddress=stpeter@jabber.org
   i:/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA 1 s:/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA
   i:/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority 2 s:/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority
   i:/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIHxzCCBq+gAwIBAgIHBVMO/QVsqTANBgkqhkiG9w0BAQsFADCBjDELMAkGA1UE
BhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBE
aWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxODA2BgNVBAMTL1N0YXJ0Q29tIENs
YXNzIDEgUHJpbWFyeSBJbnRlcm1lZGlhdGUgU2VydmVyIENBMB4XDTE1MDMwNjA0
NTI1OFoXDTE2MDMwNjE0MDYyMVowUDELMAkGA1UEBhMCVVMxHjAcBgNVBAMTFWNv
bmZlcmVuY2UuamFiYmVyLm9yZzEhMB8GCSqGSIb3DQEJARYSc3RwZXRlckBqYWJi
ZXIub3JnMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA1KG/b3T7YIe7
aQW3TUCCvHaVjHmfqM8DSKowxZJaWyhYirNWqvcCkXgk+ZUWwb35iCgc3bi1CZVz
+DG0+cx80sMSs071PIuBoRXxyt3RRgh6ioMdfedvjzo/GSR3Hhy4KQ8VhPkRtevl
eHUH4ArRd8J+1uXSWsUGsesxr9YEFGPzmYYm7g//Ob8xoUhj2e0OPSDuLrapgplJ
0o0GIZD//dM3lLnjwkGOnEENMQKbeOENeLXA8ysHrbMqfbHMqMc4ciool/WUsu9I
EPsdXSAxH5C9Pq5+of9tM1uUcau5y6mIy1nb25MqG2AvlQTTnQDHEjs9FD9kF3fl
LlECuNiOYHdPi6ipyuU29l0VesEQ4nvzI2Q1fErgFurzNBOQmgoGlzDDV2NMiVOl
MmKXBXlYTuV4IUZcMWD4NehGfYjXAIDFFRbGVyPaQzyj/wlrv4W3RlZD16T6Q6pa
NGQSG3n3YJ5bT3EQCRjSQRPM8nEmlZK5SCcrF8oZRB+U12UW3m4J9q2EwWM0JGek
JKeiIFmYxeW/C1QEKoV2vTRfb2xhSJs2kNKUCPygUubf7XGOEZjSO4lyrk1FtdtB
teE4gEZXr+AuuDNrTBnI/l9EpKjMPOLAy8eThYur8PSzvb1M+D14EVvprVGlg54b
Mg0oXeKn6/TtMJfIfhrr5G0Dtyc4IskCAwEAAaOCA2cwggNjMAkGA1UdEwQCMAAw
CwYDVR0PBAQDAgOoMBMGA1UdJQQMMAoGCCsGAQUFBwMBMB0GA1UdDgQWBBSTNrPN
TOalnNq81EnhgHKtGsJXIzAfBgNVHSMEGDAWgBTrQjTQmLCrn/Qbawj3zGQu7w4s
RTCBrAYDVR0RBIGkMIGhghVjb25mZXJlbmNlLmphYmJlci5vcmeCCmphYmJlci5v
cmegIwYIKwYBBQUHCAWgFwwVY29uZmVyZW5jZS5qYWJiZXIub3JnoCMGCCsGAQUF
BwgHoBcWFWNvbmZlcmVuY2UuamFiYmVyLm9yZ6AYBggrBgEFBQcIBaAMDApqYWJi
ZXIub3JnoBgGCCsGAQUFBwgHoAwWCmphYmJlci5vcmcwggFWBgNVHSAEggFNMIIB
STAIBgZngQwBAgEwggE7BgsrBgEEAYG1NwECAzCCASowLgYIKwYBBQUHAgEWImh0
dHA6Ly93d3cuc3RhcnRzc2wuY29tL3BvbGljeS5wZGYwgfcGCCsGAQUFBwICMIHq
MCcWIFN0YXJ0Q29tIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MAMCAQEagb5UaGlz
IGNlcnRpZmljYXRlIHdhcyBpc3N1ZWQgYWNjb3JkaW5nIHRvIHRoZSBDbGFzcyAx
IFZhbGlkYXRpb24gcmVxdWlyZW1lbnRzIG9mIHRoZSBTdGFydENvbSBDQSBwb2xp
Y3ksIHJlbGlhbmNlIG9ubHkgZm9yIHRoZSBpbnRlbmRlZCBwdXJwb3NlIGluIGNv
bXBsaWFuY2Ugb2YgdGhlIHJlbHlpbmcgcGFydHkgb2JsaWdhdGlvbnMuMDUGA1Ud
HwQuMCwwKqAooCaGJGh0dHA6Ly9jcmwuc3RhcnRzc2wuY29tL2NydDEtY3JsLmNy
bDCBjgYIKwYBBQUHAQEEgYEwfzA5BggrBgEFBQcwAYYtaHR0cDovL29jc3Auc3Rh
cnRzc2wuY29tL3N1Yi9jbGFzczEvc2VydmVyL2NhMEIGCCsGAQUFBzAChjZodHRw
Oi8vYWlhLnN0YXJ0c3NsLmNvbS9jZXJ0cy9zdWIuY2xhc3MxLnNlcnZlci5jYS5j
cnQwIwYDVR0SBBwwGoYYaHR0cDovL3d3dy5zdGFydHNzbC5jb20vMA0GCSqGSIb3
DQEBCwUAA4IBAQALksptP+jxlL2T/IXxW3YUM45JEcPXzj8D+Uq5jMBMrJMawMcN
r6ORD4E9NbXlhKFoDcj3xceQx2SD0W6U4mCZWttHCciMRNafv+4PfstI7FonFnbt
WqnC+L8y4Yyqqxn1qAhdyKH55VCZIo5QkXK1K22ZlMyfuv9Pu24iYol7ev3fbdKR
QC5f0xQ74axpeQtvrKwJxtbqYXcQ0q4C41pZ8m+v1AxgzH+TyRLkcOuufGplOArW
/CruSSXe1e5zVtHSaAmFvxEDo9ookiuDapgRCMx+YadVpq38tug7CLlK7tCnKTem
+Zk3h5O51YzCFhkzuYaJkJUgaDTKeHfQNnUo
-----END CERTIFICATE-----
subject=/C=US/CN=conference.jabber.org/emailAddress=stpeter@jabber.org
issuer=/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA
---
No client certificate CA names sent
---
SSL handshake has read 6888 bytes and written 599 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 4D0D34F5F47167E4BF518125D4001DCB9E698A12176805526D50D3B9934CB327
    Session-ID-ctx:     Master-Key: BE006B80DA68AF27F7624F017214EE53D27F8DFBDE1BB6AD4CBD260551311FF2879FFD1161CC29610B77EABB3404EDC6
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - 94 6e 0e b1 5c 41 f3 52-3e ec 16 0f 51 79 6d 10   .n..\A.R>...Qym.
    0010 - c3 b0 99 b9 85 40 94 a8-23 c0 39 a5 7e a3 47 82   .....@..#.9.~.G.
    0020 - dd d3 54 60 10 35 3c 1a-31 f2 44 38 90 6c f0 48   ..T`.5<.1.D8.l.H
    0030 - 8f 1f e5 1f 87 ed 79 93-29 10 3d 55 a6 53 7e 3d   ......y.).=U.S~=
    0040 - 52 04 4a 45 b1 9d c2 79-d6 ac 1a 80 03 4d 37 d2   R.JE...y.....M7.
    0050 - fe db 2e 6f 33 8a ed 30-bc d7 2a 6a 9c fa 8d cb   ...o3..0..*j....
    0060 - 38 db 82 6a b0 82 9b 35-ae 6e 10 81 30 1d e7 2a   8..j...5.n..0..*
    0070 - b7 e8 dc 3f a3 b4 9b 5b-65 6d e2 73 95 8e 03 9b   ...?...[em.s....
    0080 - 35 70 08 43 06 ee d9 49-d1 3d cd b1 a9 5c 33 0e   5p.C...I.=...\3.
    0090 - 04 69 af ef dc 9c 02 d1-26 5c 67 c9 be 44 9e 70   .i......&\g..D.p
    00a0 - fc a7 02 55 fb 5f 20 d8-41 6c 0b ea ef 12 55 14   ...U._ .Al....U.     Start Time: 1455610853
    Timeout   : 300 (sec)
    Verify return code: 19 (self signed certificate in certificate chain)
---
DONE

Michael42 · February 17, 2016, 3:35am

Hi Guus,

Thanks for your quick response. I went ahead an enabled debugging as you suggested. The log entries seem to indicate there is a problem finding the certificate chain. I’ve attached a partial log file listing. In addition, I notice that the Openfire Trust Certificate Store has a few expired certificates including one for “Jabber Software Foundation” which expired on Dec. 2nd, 2011. I’m not sure if this certificate is causing the problem or is unrelated. See below:

I’m assuming that the certificates contained in the Trust Store are part of the Openfire installation. Would the solution be simply to delete all the expired certificates and import a new proper certificate for jabber.org? If so, how do I obtain the proper certificate to import into Openfire? Also, I don’t see any StartCom certificates in the trust store. You mention that you were able to successfully federate with jabber.org. Does that mean that your trust store contents do not match mine? Why is my installation of Openfire missing the required certificates and how can I obtain them to import? And why is Openfire version 3.10.2 able to federate with jabber.org without any issues? By the way, I forgot to mention - I’m running Openfire as a service on a Windows 2008 server, not on a Linux machine.

Thanks again,

Michael
PartialLog.txt.zip (2263 Bytes)

Michael42 · February 18, 2016, 9:30pm

Just to update - it turns out that my Openfire 4.0.1 installation was missing the proper StartCom CA Root certificate in the trust store. As soon as I imported it, bidirectional communications with jabber.org successfully resumed.

It was just odd that the two Openfire 4.0.1 servers which ran on Windows 2008 both had the missing certificate problem. But a third Openfire server which was installed on a Windows 7 Professional workstation had the proper certificate already in its trust store and no issues federating with jabber.org. So I have a few questions:

Where does Openfire 4.0.1 get its trust store and how is it populated? (i.e. Does it pull some certs from the existing Windows environment or is it included in the installation package?)
Why in a fresh installation, are there already a few expired certificates present in the trust store? (Some expired back in 2011)
Why does it work fine on a workstation, but not in a Microsoft AD environment?
As a side note - it would be nice if the trust store contents can be viewed in alphabetical order.

Anyway, a BIG thank you goes out to Guus who pointed me in the right direction and helped me solve this issue. I hope my detailed account of this issue helps someone else as well.

Michael

guus · September 8, 2016, 7:02pm

Openfire ships with various, pre-populated, stores. They can be modified through the admin panel, or using the Java keytool command.
Because we are terrible in maintaining a proper set of certificates in the stores that we ship. :-/
I can’t explain that.
Yeah, those pages would benefit from an overhaul. Column-based sorting would be nice.