Openfire 4.9.2 and 5.0.0 Mtls not working

Openfire Openfire Support
1

Hi,

We have a setup with Openfire 4.7.5 working with mutual auth, when we upgrade the server to version 4.9.2 the clients can’t connect anymore, we also try with the newer version 5.0.0 and the result is the same. No changes were done on the client side and on server only the upgrade of Openfire, if we rollback to version 4.7.5 the clients can connect again.

We did some tests and can confirm that if we change the authentication mechanism we can connect to the versions 4.9.2 and 5.0.0, but when we set back to have the mtls it fails the login.

The login looks to be in a loop logs:

2025.06.23 12:24:27 TRACE [socket_c2s_ssl-worker-10]: org.jivesoftware.openfire.nio.NettyConnection - Sending: <?xml version="1.0" encoding="UTF-8"?><st
ream:stream xmlns:stream="http://etherx.jabber.org/streams" xmlns="jabber:client" from="SERVER_DOMAIN" id="5uielj4zbg" xml:lang="en" vers
ion="1.0"><stream:features><limits xmlns="urn:xmpp:stream-limits:0"><max-bytes>1048576</max-bytes><idle-seconds>360</idle-seconds></limits><mechanisms x
mlns="urn:ietf:params:xml:ns:xmpp-sasl"><mechanism>SCRAM-SHA-1</mechanism><mechanism>EXTERNAL</mechanism></mechanisms><compression xmlns="http://jabber.
org/features/compress"><method>zlib</method></compression><ver xmlns="urn:xmpp:features:rosterver"/><c xmlns="http://jabber.org/protocol/caps" hash="sha
-1" node="https://www.igniterealtime.org/projects/openfire/" ver="adm2s8L3EKNfluhLCGGhCaZKWTY="/><limits xmlns="urn:xmpp:stream-limits:0"><max-bytes>104
8576</max-bytes><idle-seconds>360</idle-seconds></limits></stream:features>
2025.06.23 12:24:27 TRACE [socket_c2s_ssl-worker-10]: org.jivesoftware.openfire.nio.NettyConnectionHandler - Handler on /SERVER_IP:5223--/CLIENT_IP:53275 received: <auth xmlns="urn:ietf:params:xml:ns:xmpp-sasl" mechanism="EXTERNAL">=</auth>
2025.06.23 12:24:27 TRACE [socket_c2s_ssl-worker-10]: org.jivesoftware.openfire.nio.NettyConnection - Sending: <challenge xmlns="urn:ietf:params:xml:ns:xmpp-sasl">=</challenge>
2025.06.23 12:24:27 TRACE [socket_c2s_ssl-worker-10]: org.jivesoftware.openfire.nio.NettyConnectionHandler - Handler on /SERVER_IP:5223--/CLIENT_IP:53275 received: <response xmlns="urn:ietf:params:xml:ns:xmpp-sasl">=</response>
2025.06.23 12:24:27 TRACE [socket_c2s_ssl-worker-10]: org.jivesoftware.openfire.nio.NettyConnection - Sending: <challenge xmlns="urn:ietf:params:xml:ns:xmpp-sasl">=</challenge>
2025.06.23 12:24:27 TRACE [socket_c2s_ssl-worker-10]: org.jivesoftware.openfire.nio.NettyConnectionHandler - Handler on /SERVER_IP:5223--/CLIENT_IP:53275 received: <response xmlns="urn:ietf:params:xml:ns:xmpp-sasl">=</response>
2025.06.23 12:24:27 TRACE [socket_c2s_ssl-worker-10]: org.jivesoftware.openfire.nio.NettyConnection - Sending: <challenge xmlns="urn:ietf:params:xml:ns:xmpp-sasl">=</challenge>
2025.06.23 12:24:27 TRACE [socket_c2s_ssl-worker-10]: org.jivesoftware.openfire.nio.NettyConnectionHandler - Handler on /SERVER_IP:5223--/CLIENT_IP:53275 received: <response xmlns="urn:ietf:params:xml:ns:xmpp-sasl">=</response>
2025.06.23 12:24:27 TRACE [socket_c2s_ssl-worker-10]: org.jivesoftware.openfire.nio.NettyConnection - Sending: <challenge xmlns="urn:ietf:params:xml:ns:xmpp-sasl">=</challenge>
.
. loop until timeout or we cancel login
.

2025.06.23 12:24:31 DEBUG [socket_c2s_ssl-worker-10]: org.jivesoftware.openfire.nio.NettyXMPPDecoder - Closing client connection, as the socket connecti
on was reset: NettyConnection{peer: /CLIENT_IP:53275, state: OPEN, session: LocalClientSession{address=SERVER_DOMAIN/ede379ec-9fcc-4b
2c-9b67-50cd1c6f1bb5, streamID=5uielj4zbg, status=CONNECTED, isEncrypted=true, isDetached=false, serverName='SERVER_DOMAIN', isInitialize
d=false, hasAuthToken=false, peer address='CLIENT_IP', presence='<presence type="unavailable"/>'}, Netty channel handler context name: NettyClientCo
nnectionHandler#0}
java.net.SocketException: Connection reset
        at sun.nio.ch.SocketChannelImpl.throwConnectionReset(SocketChannelImpl.java:401) ~[?:?]
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:434) ~[?:?]
        at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:255) ~[netty-buffer-4.1.118.Final.jar:4.1.118.Final]
        at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1132) ~[netty-buffer-4.1.118.Final.jar:4.1.118.Final]
        at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:356) ~[netty-transport-4.1.118.Final.jar:4.1.118.Final]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:151) ~[netty-transport-4.1.118.Final.jar:4.1.118.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:796) ~[netty-transport-4.1.118.Final.jar:4.1.118.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:732) ~[netty-transport-4.1.118.Final.jar:4.1.118.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:658) ~[netty-transport-4.1.118.Final.jar:4.1.118.Final]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[netty-transport-4.1.118.Final.jar:4.1.118.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998) ~[netty-common-4.1.118.Final.jar:4.1.118.Final]
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.118.Final.jar:4.1.118.Final]
        at java.lang.Thread.run(Thread.java:1583) [?:?]
2025.06.23 12:24:31 TRACE [socket_c2s_ssl-worker-10]: org.jivesoftware.openfire.nio.NettyConnection - Closing NettyConnection{peer: /CLIENT_IP:53275
, state: CLOSED, session: LocalClientSession{address=SERVER_DOMAIN/ede379ec-9fcc-4b2c-9b67-50cd1c6f1bb5, streamID=5uielj4zbg, status=CONN
ECTED, isEncrypted=true, isDetached=false, serverName='SERVER_DOMAIN', isInitialized=false, hasAuthToken=false, peer address='194.39.218.
13', presence='<presence type="unavailable"/>'}, Netty channel handler context name: NettyClientConnectionHandler#0} with optional error: 
<stream:error xmlns:stream="http://etherx.jabber.org/streams">
  <internal-server-error xmlns="urn:ietf:params:xml:ns:xmpp-streams"/>
  <text xmlns="urn:ietf:params:xml:ns:xmpp-streams">An error occurred in XMPP Decoder</text>
</stream:error>
2025.06.23 12:24:31 TRACE [socket_c2s_ssl-worker-10]: org.jivesoftware.openfire.nio.NettyConnection - Flushed any final bytes, closing connection.

Setup:
RHEL9
Java 21.0.7
TLS 1.3/1.2
client certificate trust using elliptic curve certificates (client with user in the CN field)
authentication mechanism EXTERNAL
direct tls and starttls

Any idea what can be the issue?

2

Hi, thanks for sharing this. On first glance, I wonder if this relates to the fix that we applied to Openfire 4.8.0 for issue OF-2514.

For client-to-server SASL EXTERNAL mechanim, Openfire seems to require that the server sends the authorizationIdentity in the initial response (or after the challenge). Your client sends empty responses (=).

As an aside, the SASL EXTERNAL mechanism that Openfire uses for server-to-server doesn’t require this. This appears to be a deliberate difference.

I didn’t have the time yet to dig into this deeper, but can you try with providing the authz-id during the SASL handshake?

3

I’ve now had some time to look at this in more detail. I think this is a bug in Openfire: it interprets the ‘empty initial response’ as ‘no initial response’ and then incorrectly challenges the client.

I have raised a new ticket for this bug: OF-3094

A fix for the problem is also in the works: OF-3094: C2S SASL EXTERNAL must not challenge after receiving empty IR by guusdk · Pull Request #2859 · igniterealtime/Openfire · GitHub

4

That fix has now been merged. Can you test this using a nightly build of Openfire? All nightly builds starting on (and including) June 25, 2025, should have this fix.

5

Hi,

Thanks for the fast reaction. I tested the night version that you sent and it works out! We are able to login using the mtls :slightly_smiling_face:

Any idea when this version will be released?

Once again thanks for the support

6

Ah, thanks for confirming that this works. There’s no release date yet for the next version of Openfire.

7

This issue has been fixed in Openfire 5.0.1 that is available now! See: Openfire 5.0.1 release - our 100th! (maybe)

1 Like