Openfire+Spark - Client X.509/PKI Certificate Support

Hi-

How does the Spark client support Client-side X.509. E.g., what properties do I need to set to get it to recognize a keystore and keystorePassword or prompt for keystore password? (I’m running Spark 2.5.8 against Openfire 3.6.3.)

Server-side SSL Certification is working fine. I have the (self-issued) CA Certificate in both /opt/openfire/resources/security/truststore and /opt/openfire/resources/security/client.truststore; this CA issues both the server and client certificates. I have a user-based keystore in ~/.keystore (contains mykey and the CA certificate).

This: http://www.igniterealtime.org/builds/openfire/docs/latest/documentation/ssl-guid e.html

deals with openfire (server-side) configuration and not client-side configuration, so is not of much help w/r/t Spark.

I have set the various system properties on the server:

xmpp.client.cert.policy

needed

xmpp.client.certificate.accept-selfsigned

false

xmpp.client.certificate.crl

/opt/openfire/resources/security/crl.pem

xmpp.client.certificate.verify

true

xmpp.client.certificate.verify.chain

true

xmpp.client.certificate.verify.root

true

xmpp.client.certificate.verify.validity

true

xmpp.client.tls.policy

required

xmpp.socket.ssl.client.truststore

/opt/openfire/resources/security/client.truststore

Other posts have posed the question, but there’s no complete guide on HOW TO do this.

E.g.,

Would it be possible to put together an end-to-end HOW TO on this? For both Server-side and Client-side configuration?

Thanks,

Kevin

Forgot to note that ‘xmpp.socket.ssl.client.truststore’ was set. - vargok

First off, The 2.5.8 release dosnt have PKI support, I added that support after the release, so its in SVN. When Jive does the next release the code will be there. I think the 2.6.0 beta has everything needed in there. There is a tab in preferences for setting up PKI stuff. Not many people (read: close to 0) are using it, so there might be some bugs lurking in there. I never did much with the keystore format, but there is some basic support for it (I use the PKCS#11 modules)

OK, I pulled down the beta2 of Spark, and still having pretty much the same issue. I’ve set things in the UI for PKI, and verified that it is reading the keystore – wrong value for password shows:

$ /opt/spark/Spark
ls: /opt/spark/lib/windows: No such file or directory
java.io.IOException: Keystore was tampered with, or password was incorrect
at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:768)
at java.security.KeyStore.load(KeyStore.java:1150)
at org.jivesoftware.smack.ServerTrustManager.(ServerTrustManager.java:63)
at org.jivesoftware.smack.XMPPConnection.proceedTLSReceived(XMPPConnection.java:13 67)
at org.jivesoftware.smack.PacketReader.parsePackets(PacketReader.java:313)
at org.jivesoftware.smack.PacketReader.access$000(PacketReader.java:44)
at org.jivesoftware.smack.PacketReader$1.run(PacketReader.java:76)

Whereas the correct value gives no errors. I’ve got the right certificates marked the right way in all the keystores – client side key/trust store uses same file, has the CA certificate (trusted) and the client certificate as ‘mykey’. The server keystore and truststores have the CA certificate (trusted).

However, when I try to connect, I end up with an odd handshake error on the server side:

SocketAcceptorIoProcessor-0.0, fatal error: 80: problem unwrapping net record
java.lang.RuntimeException: Delegated task threw Exception/Error
SocketAcceptorIoProcessor-0.0, SEND TLSv1 ALERT: fatal, description = internal_error
SocketAcceptorIoProcessor-0.0, WRITE: TLSv1 Alert, length = 2
java.lang.RuntimeException: Delegated task threw Exception/Error
at com.sun.net.ssl.internal.ssl.Handshaker.checkThrown(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLEngineImpl.checkTaskThrown(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLEngineImpl.readNetRecord(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLEngineImpl.unwrap(Unknown Source)
at javax.net.ssl.SSLEngine.unwrap(Unknown Source)
at org.apache.mina.filter.support.SSLHandler.unwrap0(SSLHandler.java:658)
at org.apache.mina.filter.support.SSLHandler.unwrapHandshake(SSLHandler.java:614)
at org.apache.mina.filter.support.SSLHandler.handshake(SSLHandler.java:493)
at org.apache.mina.filter.support.SSLHandler.messageReceived(SSLHandler.java:306)
at org.apache.mina.filter.SSLFilter.messageReceived(SSLFilter.java:392)

[…]

Caused by: java.lang.NullPointerException
at com.sun.net.ssl.internal.ssl.HandshakeMessage$CertificateRequest.(Unknown Source)
at com.sun.net.ssl.internal.ssl.ServerHandshaker.clientHello(Unknown Source)
at com.sun.net.ssl.internal.ssl.ServerHandshaker.processMessage(Unknown Source)
at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Unknown Source)
at com.sun.net.ssl.internal.ssl.Handshaker$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.net.ssl.internal.ssl.Handshaker$DelegatedTask.run(Unknown Source)
at org.apache.mina.filter.support.SSLHandler.doTasks(SSLHandler.java:686)
at org.apache.mina.filter.support.SSLHandler.handshake(SSLHandler.java:486)
… 16 more

I added the end-entity client certificate explicitly on the server-side (and restarted), but that changed nothing.

I’ve attached various logging and other information (slightly redacted) to this post. Any help would be most appreciated. I’ve turned on SSL debug, but nothing of value is really provided. The only thing I note is that I never see the client’certificate’s Subject appear anywhere. However, it doesn’t appear to be a trust issue – that’s usually pretty obvious in my experience thus far anyway.
Spark-SSL-Issue.txt (23449 Bytes)

Even setting client certification to Optional, I’m still getting handhake failures:

xmpp.client.cert.policy

wanted

xmpp.client.certificate.verify

false

xmpp.client.certificate.verify.chain

false

xmpp.client.certificate.verify.root

false

xmpp.client.certificate.verify.validity

false

==> error.log <==
2009.04.08 15:29:00 [org.jivesoftware.openfire.nio.ConnectionHandler.exceptionCaught(ConnectionHand ler.java:111)]
java.lang.RuntimeException: Delegated task threw Exception/Error
at com.sun.net.ssl.internal.ssl.Handshaker.checkThrown(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLEngineImpl.checkTaskThrown(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLEngineImpl.readNetRecord(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLEngineImpl.unwrap(Unknown Source)
at javax.net.ssl.SSLEngine.unwrap(Unknown Source)
at org.apache.mina.filter.support.SSLHandler.unwrap0(SSLHandler.java:658)
at org.apache.mina.filter.support.SSLHandler.unwrapHandshake(SSLHandler.java:614)
at org.apache.mina.filter.support.SSLHandler.handshake(SSLHandler.java:493)
at org.apache.mina.filter.support.SSLHandler.messageReceived(SSLHandler.java:306)
at org.apache.mina.filter.SSLFilter.messageReceived(SSLFilter.java:392)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)
at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)
at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)
at org.apache.mina.common.support.AbstractIoFilterChain$HeadFilter.messageReceived (AbstractIoFilterChain.java:499)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)
at org.apache.mina.common.support.AbstractIoFilterChain.fireMessageReceived(Abstra ctIoFilterChain.java:293)
at org.apache.mina.transport.socket.nio.SocketIoProcessor.read(SocketIoProcessor.j ava:228)
at org.apache.mina.transport.socket.nio.SocketIoProcessor.process(SocketIoProcesso r.java:198)
at org.apache.mina.transport.socket.nio.SocketIoProcessor.access$400(SocketIoProce ssor.java:45)
at org.apache.mina.transport.socket.nio.SocketIoProcessor$Worker.run(SocketIoProce ssor.java:485)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.NullPointerException
at com.sun.net.ssl.internal.ssl.HandshakeMessage$CertificateRequest.(Unknown Source)
at com.sun.net.ssl.internal.ssl.ServerHandshaker.clientHello(Unknown Source)
at com.sun.net.ssl.internal.ssl.ServerHandshaker.processMessage(Unknown Source)
at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Unknown Source)
at com.sun.net.ssl.internal.ssl.Handshaker$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.net.ssl.internal.ssl.Handshaker$DelegatedTask.run(Unknown Source)
at org.apache.mina.filter.support.SSLHandler.doTasks(SSLHandler.java:686)
at org.apache.mina.filter.support.SSLHandler.handshake(SSLHandler.java:486)
… 16 more

This was triggered via 3.10 of Smack with a simple connection test client:

XMPPConnection conn1 = new XMPPConnection(server);
conn1.connect();
conn1.login(username, password);
System.out.println(“Logged " + username + “@” + server + " in.”);
Thread.sleep(15 * 1000);
conn1.disconnect();

Disabling client cert totally, allows the handshake to proceed.

I also injected a stack trace into the server code (downloaded source, added a (new Exception()).printStackTrace()), but that doesn’t show anything of any more use (but that’s why the above ConnectionHandler shows line 111 instead of 110):

java.lang.Exception: Debugging SSL/TLS Client-Cert con problems.
at org.jivesoftware.openfire.nio.ConnectionHandler.exceptionCaught(ConnectionHandl er.java:110)
at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.exceptionCaught (AbstractIoFilterChain.java:564)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextExceptionCaught(Ab stractIoFilterChain.java:345)
at org.apache.mina.common.support.AbstractIoFilterChain.access$1000(AbstractIoFilt erChain.java:53)
at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.exceptionCaugh t(AbstractIoFilterChain.java:643)
at org.apache.mina.common.IoFilterAdapter.exceptionCaught(IoFilterAdapter.java:75)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextExceptionCaught(Ab stractIoFilterChain.java:345)
at org.apache.mina.common.support.AbstractIoFilterChain.access$1000(AbstractIoFilt erChain.java:53)
at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.exceptionCaugh t(AbstractIoFilterChain.java:643)
at org.apache.mina.common.IoFilterAdapter.exceptionCaught(IoFilterAdapter.java:75)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextExceptionCaught(Ab stractIoFilterChain.java:345)
at org.apache.mina.common.support.AbstractIoFilterChain.access$1000(AbstractIoFilt erChain.java:53)
at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.exceptionCaugh t(AbstractIoFilterChain.java:643)
at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java :243)
at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(Execut orFilter.java:283)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)
at java.lang.Thread.run(Unknown Source)

I’ve not yet tried the SVN branches.

I’m totally lost on what’s mis-configured. If it matters, i’m using embedded DB.

I beefed up the Smack 3.1.0-based test client a bit, reinstalled openfire completely, properly certificate the server’s certificature (via external CA), but now I’m getting:

==> warn.log <==
2009.04.09 12:39:38 Closing session due to exception: (SOCKET, R: /127.0.0.1:39855, L: /127.0.0.1:5222, S: 0.0.0.0/0.0.0.0:5222)
org.apache.mina.filter.codec.ProtocolDecoderException: java.lang.Exception: Disallowed character (Hexdump: 14 03 01 00 01 01 16 03 01 00 20 B6 39 30 79 68 02 07 8F 6F 4B 29 58 37 C8 87 2B 45 35 9E BC F5 CA 3D 41 9B 9D 66 03 2E 8E BE B8)
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:170)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)
at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)
at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)
at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java :239)
at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(Execut orFilter.java:283)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.Exception: Disallowed character
at org.jivesoftware.openfire.nio.XMLLightweightParser.read(XMLLightweightParser.ja va:211)
at org.jivesoftware.openfire.nio.XMPPDecoder.doDecode(XMPPDecoder.java:32)
at org.apache.mina.filter.codec.CumulativeProtocolDecoder.decode(CumulativeProtoco lDecoder.java:133)
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:163)
… 9 more

And the client complains that:

Logging into tester@localhost
java.net.SocketException: Socket closed
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:99)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at sun.nio.cs.StreamEncoder$CharsetSE.writeBytes(StreamEncoder.java:336)
at sun.nio.cs.StreamEncoder$CharsetSE.implFlushBuffer(StreamEncoder.java:404)
at sun.nio.cs.StreamEncoder$CharsetSE.implFlush(StreamEncoder.java:408)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:152)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:213)
at java.io.BufferedWriter.flush(BufferedWriter.java:236)
at org.jivesoftware.smack.PacketWriter.writePackets(PacketWriter.java:259)
at org.jivesoftware.smack.PacketWriter.access$000(PacketWriter.java:40)
at org.jivesoftware.smack.PacketWriter$1.run(PacketWriter.java:87)
javax.net.ssl.SSLProtocolException: Handshake message sequence violation, 2
at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.j ava:86)
at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:495)
at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:433)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:877)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImp l.java:1089)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:11 16)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:11 00)
at org.jivesoftware.smack.XMPPConnection.proceedTLSReceived(XMPPConnection.java:12 58)
at org.jivesoftware.smack.PacketReader.parsePackets(PacketReader.java:313)
at org.jivesoftware.smack.PacketReader.access$000(PacketReader.java:44)
at org.jivesoftware.smack.PacketReader$1.run(PacketReader.java:76)
Connection is NOT using TLS and therefore is: NOT secure.
java.lang.NullPointerException
at org.jivesoftware.smack.SASLAuthentication.send(SASLAuthentication.java:532)
at org.jivesoftware.smack.sasl.SASLMechanism.authenticate(SASLMechanism.java:130)
at org.jivesoftware.smack.sasl.SASLMechanism.authenticate(SASLMechanism.java:92)
at org.jivesoftware.smack.SASLAuthentication.authenticate(SASLAuthentication.java: 308)
at org.jivesoftware.smack.XMPPConnection.login(XMPPConnection.java:395)
at TestClient.main(TestClient.java:56)
No response from the server.:
at org.jivesoftware.smack.NonSASLAuthentication.authenticate(NonSASLAuthentication .java:74)
at org.jivesoftware.smack.SASLAuthentication.authenticate(SASLAuthentication.java: 345)
at org.jivesoftware.smack.XMPPConnection.login(XMPPConnection.java:395)
at TestClient.main(TestClient.java:56)

The Test Client is:

public static void main(String[] args) {
   
    System.setProperty("javax.net.ssl.keyStore", KEYSTORE_PATH);
    System.setProperty("javax.net.ssl.trustStore", TRUSTSTORE_PATH);
    System.setProperty("javax.net.ssl.keyStorePassword", KEYSTORE_PASSWORD);
    System.setProperty("javax.net.ssl.trustStorePassword", TRUSTSTORE_PASSWORD);

ConnectionConfiguration config = new ConnectionConfiguration(SERVER, 5222);
// config.setCompressionEnabled(true);
config.setCompressionEnabled(false);
config.setSecurityMode(SecurityMode.enabled);
config.setSASLAuthenticationEnabled(true);
// config.setSASLAuthenticationEnabled(false);
config.setKeystorePath(KEYSTORE_PATH);
config.setKeystoreType(“jks”);
config.setTruststorePath(TRUSTSTORE_PATH);
config.setTruststorePassword(TRUSTSTORE_PASSWORD);
config.setTruststoreType(“jks”);

config.setSecurityMode(SecurityMode.required);

SASLAuthentication.supportSASLMechanism(“PLAIN”, 2);
SASLAuthentication.supportSASLMechanism(“DIGEST-MD5”, 1);
SASLAuthentication.supportSASLMechanism(“EXTERNAL”, 0);

System.out.println("Logging into " + USERNAME + “@” + SERVER);
try {
XMPPConnection conn1 = new XMPPConnection(config);
conn1.connect();
String usingTLS = (conn1.isUsingTLS() == true) ? “” : " NOT ";
String secure = (conn1.isSecureConnection() == true) ? “” : " NOT ";
System.out.println("Connection is "
+ usingTLS
+ "using TLS and therefore is: "
+ secure
+ “secure.”);
Thread.sleep(3 * 1000); // Something about timing in the forums
conn1.login(USERNAME, PASSWORD, “smack test client”);
System.out.println(“Logged " + USERNAME + “@” + SERVER + " in.”);
Thread.sleep(15 * 1000); //leave it connected for a bit, then disconnect.
conn1.disconnect();
System.out.println("Disconnected " + USERNAME + “@” + SERVER);
} catch (XMPPException xe) {
// TODO Auto-generated catch block
xe.printStackTrace();
} catch (InterruptedException ie) {
// TODO Auto-generated catch block
ie.printStackTrace();
}
}

Again, everything works as expected against the server when not configured to require Client X.509 Certificates. I even went through the effort of installing jabberd and configuring it under ‘verify-mode: 7’, which, near as I can tell, is the same thing as ‘xmpp.client.cert.policy’=‘needed’.

In this case, jabberd complains that:

Apr 9 13:12:11 jabberd/c2s[29547]: [8] [10.10.10.10, port=40668] connect
Apr 9 13:12:13 jabberd/c2s[29547]: [8] [10.10.10.10, port=40668] error: SSL handshake error (error:140890C7:SSL routines:SSL3_GET_CLIENT_CERTIFICATE:peer did not return a certificate)
Apr 9 13:12:13 jabberd/c2s[29547]: [8] [10.10.10.10, port=40668] disconnect jid=unbound, packets: 0

Is there some other setting that’s required to make sure that the certificate is being sent over the wire? It doesn’t look like it is. (If the handshake debug messages could be beefed up that would be helpful, too.)

Debugging SSL is hard to do, since you cant easily snoop the traffic. A very helpful tool Ive used for this is ssldump (a friend of the tcpdump utility). It will give you details of the SSL handshake (as much as it can). If you have the private keys invovled it can even do decryption for certain types of connections.

Also, this page is very helpful: http://java.sun.com/j2se/1.5.0/docs/guide/security/jsse/ReadDebug.html In essence, set the java commandline option -Djavax.net.debug=all and you will see LOTS of output. But it shows the certificate selection process, etc. Try that with Spark, and see if there is anything useful there. Its possible the server certificate issuer (itself, if self-signed) is not trusted by the client. If the client does not trust the server cert, it will not attempt to continue the SSL handshake. Sadly the error handling/reporting in spark isnt great- I never really perfected it since no one ended up using my changes.

I had been using the SSL debug portion of javax.net.debug, but that hadn’t given me really anything different. It still results, at the end with:

Smack Packet Reader (0), READ: TLSv1 Handshake, length = 1836
Smack Packet Reader (0), handling exception: javax.net.ssl.SSLProtocolException: Handshake message sequence violation, 2
Smack Packet Reader (0), SEND TLSv1 ALERT: fatal, description = unexpected_message
Padded plaintext before ENCRYPTION: len = 18
0000: 02 0A 36 C0 76 94 53 E7 E9 88 ED C2 86 5F 24 19 …6.v.S…_$.
0010: F1 EC …
Smack Packet Reader (0), WRITE: TLSv1 Alert, length = 18
[Raw write]: length = 23
0000: 15 03 01 00 12 4C 7A 7F 67 57 A5 1E DC A2 A8 BE …Lz.gW…
0010: A3 B2 08 9D 48 0F 17 …H…
Smack Packet Reader (0), called closeSocket()
java.net.SocketException: Socket closed
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:99)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:202)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:272)
at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:276)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:122)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:212)
at java.io.BufferedWriter.flush(BufferedWriter.java:236)
at org.jivesoftware.smack.PacketWriter.writePackets(PacketWriter.java:274)
at org.jivesoftware.smack.PacketWriter.access$000(PacketWriter.java:40)
at org.jivesoftware.smack.PacketWriter$1.run(PacketWriter.java:87)

On the Spark side, and:

2009.04.14 07:36:25 Closing session due to exception: (SOCKET, R: /10.10.10.179:48286, L: /10.10.10.179:5222, S: 0.0.0.0/0.0.0.0:5222)
org.apache.mina.filter.codec.ProtocolDecoderException: java.lang.Exception: Disallowed character (Hexdump: 16 03 01 00 20 78 5F 5C 9F E7 90 A5 30 00 BE 6F 33 8C 14 6B 77 D4 FE A1 EB 2C D7 75 A2 A1 A3 CA 41 0E 96 BA 81)
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:170)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)
at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)
at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)
at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java :239)
at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(Execut orFilter.java:283)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.Exception: Disallowed character
at org.jivesoftware.openfire.nio.XMLLightweightParser.read(XMLLightweightParser.ja va:211)
at org.jivesoftware.openfire.nio.XMPPDecoder.doDecode(XMPPDecoder.java:32)
at org.apache.mina.filter.codec.CumulativeProtocolDecoder.decode(CumulativeProtoco lDecoder.java:133)
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:163)
… 9 more

on the openfire side.

I’ve tried JVMs 1.5 and 1.6 to the same result.

How does Spark choose what key to use from the keystore? It just defaults to (only) use the “mykey” entry? Does it attempt to find an alias based on the username of the jabber user? I’ve tried various aliases, but I still don’t see any client certificates being sent across the wire anywhere.

The server certificate is issued by the same CA sa the client certificate, so all the crypto works out. And the non-Client-X509-Required handshake works just fine, so I don’t think it’s a trust issue – the PKIX trust errors are pretty clear (as far as they go).

(I did just try to pull down the Spark SVN source, but the buld script isn’t working for some reason. The source/target for 1.6 were refused by `javac’ for some reason; removing those allowed the compile to proceed, but still results in lots of class resolution issues. There doesn’t appear to be a “how_to_build” for the top level, only the plugins.)

OK, here we go. I finally got ssldump running:

09:15 root ssldump-0.9b3> ./ssldump -i lo
New TCP connection #1: host1(62499) <-> host1(5222)
1 1 1.1957 (1.1957) C>S SSLv2 compatible client hello
Version 3.1
cipher suites
TLS_RSA_WITH_RC4_128_MD5
SSL2_CK_RC4
TLS_RSA_WITH_RC4_128_SHA
Unknown value 0x2f
Unknown value 0x33
Unknown value 0x32
TLS_RSA_WITH_3DES_EDE_CBC_SHA
SSL2_CK_3DES
TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA
TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
TLS_RSA_WITH_DES_CBC_SHA
SSL2_CK_DES
TLS_DHE_RSA_WITH_DES_CBC_SHA
TLS_DHE_DSS_WITH_DES_CBC_SHA
TLS_RSA_EXPORT_WITH_RC4_40_MD5
SSL2_CK_RC4_EXPORT40
TLS_RSA_EXPORT_WITH_DES40_CBC_SHA
TLS_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA
TLS_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA
1 2 1.1986 (0.0028) S>C Handshake
ServerHello
Version 3.1
session_id[32]=
49 e4 8c af d9 98 09 e8 92 15 06 95 62 ab 38 b0
d9 17 4c df 0c de df de 8f a5 ca 00 33 df 2d 27
cipherSuite TLS_RSA_WITH_RC4_128_MD5
compressionMethod NULL
Certificate
CertificateRequest
certificate_types rsa_sign
certificate_types dss_sign
ServerHelloDone
1 3 1.3083 (0.1096) C>S Handshake
Certificate
ClientKeyExchange
1 4 1.3488 (0.0405) C>S ChangeCipherSpec
ERROR: Length mismatch

Length mismatch? Could the client certificate not be “valid” for the right “purpose”?

Certificate purposes:
SSL client : Yes
SSL client CA : No
SSL server : Yes
SSL server CA : No
Netscape SSL server : Yes
Netscape SSL server CA : No
S/MIME signing : Yes
S/MIME signing CA : No
S/MIME encryption : Yes
S/MIME encryption CA : No
CRL signing : Yes
CRL signing CA : No
Any Purpose : Yes
Any Purpose CA : Yes
OCSP helper : Yes
OCSP helper CA : No

Key type/length OK?

Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public Key: (2048 bit)
Modulus (2048 bit):

I’m guessing it all results from something in:

2009.04.14 09:16:32 Closing session due to exception: (SOCKET, R: /10.10.10.179:62501, L: /10.10.10.179:5222, S: 0.0.0.0/0.0.0.0:5222)
org.apache.mina.filter.codec.ProtocolDecoderException: java.lang.Exception: Disallowed character (Hexdump: 15 03 01 00 12 27 1A 48 AF D4 54 8E 76 AC 4E 72 89 1B A0 9C E0 1A D2)
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:170)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)
at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)
at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)
at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java :239)
at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(Execut orFilter.java:283)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.Exception: Disallowed character
at org.jivesoftware.openfire.nio.XMLLightweightParser.read(XMLLightweightParser.ja va:211)
at org.jivesoftware.openfire.nio.XMPPDecoder.doDecode(XMPPDecoder.java:32)
at org.apache.mina.filter.codec.CumulativeProtocolDecoder.decode(CumulativeProtoco lDecoder.java:133)
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:163)
… 9 more

but I can’t figure out where that’s coming from, exactly. And much less so how to resolve it. Presumably the ‘00’ is the invalid character, but…

Turning on full debugging on the server-side shows that it is the “null cert chain” error:

SocketAcceptorIoProcessor-0.0, READ: TLSv1 Handshake, length = 269
*** Certificate chain


SocketAcceptorIoProcessor-0.0, fatal error: 42: null cert chain
javax.net.ssl.SSLHandshakeException: null cert chain
SocketAcceptorIoProcessor-0.0, SEND TLSv1 ALERT: fatal, description = bad_certificate
SocketAcceptorIoProcessor-0.0, WRITE: TLSv1 Alert, length = 2
SocketAcceptorIoProcessor-0.0, fatal: engine already closed. Rethrowing javax.net.ssl.SSLHandshakeException: null cert chain
SocketAcceptorIoProcessor-0.0, called closeOutbound()
SocketAcceptorIoProcessor-0.0, closeOutboundInternal()
[Raw write]: length = 7
0000: 15 03 01 00 02 02 2A …*
SocketAcceptorIoProcessor-0.0, called closeInbound()
SocketAcceptorIoProcessor-0.0, fatal: engine already closed. Rethrowing javax.net.ssl.SSLException: Inbound closed before receiving peer’s close_notify: possible truncation attack?
SocketAcceptorIoProcessor-0.0, called closeOutbound()
SocketAcceptorIoProcessor-0.0, closeOutboundInternal()

But, again, I don’t know how the client is working to access the keystore. There is no place to set a ‘keystore password’ in the client UI the way there is for a truststore password. Obviously setting the default parameters via -Djavax.net.ssl.keyStore and so on aren’t doing much.

Access to the keystore is handled via callback handler, so it would prompt the user when logging in (it was designed for smartcards, so it would say “Enter pin for your smartcard” or something like it).

Then java decides which client cert to use based on the list of CA’s presented from the server. So if the server provides no acceptable CA’s, the client wont know which cert to use.

Then the fact that I’m not seeing any sort of callback UI to enter the password implies that the server isn’t doing the right thing? Even though I have CA Certificate(s) installed in the client.truststore?

Is there anything missing from this:

http://www.igniterealtime.org/community/thread/32759

that needs to be done on the server-side?

Are there any specific bits that need to be set in the CA certificate that might not ordinarily be set in an `OpenSSL’ generated CA Certificate?

I dont think so, but lets try something.

Please run this command:

openssl s_client -prexit -connect yourjabberserverhostname:5223

When this connects, it should do several things:

Display the certificate subject name the server is using

Show the certificate chain the server is using

Show the server certificate

Show the list of acceptable client certificate CAs

Show some info about the specific connection

See if the server is actually sending a list of client CA’s.

Hi Slushpuppie. Thanks for continuing to work with me on this. I’m prettymuch stumped at this point.

But yeah, that’s basically what I’m suggesting – there’s no client-CA list being offered by the server, so the TLS is server-only.

I’m not running 5223, only TLS on 5222, but:

$ openssl s_client -prexit -connect localhost:5222
CONNECTED(00000003)
6550:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:188:

no peer certificate available

No client certificate CA names sent

SSL handshake has read 0 bytes and written 124 bytes

New, (NONE), Cipher is (NONE)
Compression: NONE
Expansion: NONE

I have two CAs listed in the client.truststore, both "trustedCertEntry"s. I’ve tried both leaving the sys prop for client.trustore blank, and explicitly setting it to: /opt/openfire/resources/security/client.truststore.

Running s_client' in-starttls xmpp’ mode yields a successful (non-client) handshake from all appearances/no errors on the server side:

$ openssl s_client -prexit -connect localhost:5222 -starttls xmpp
CONNECTED(00000003)

no peer certificate available

No client certificate CA names sent

SSL handshake has read 455 bytes and written 114 bytes

New, (NONE), Cipher is (NONE)
Compression: NONE
Expansion: NONE

In any event, i’m certainly not getting prompted for a call-back password box, as I’ve seen from Java on other client SSL-enabled apps.


I switched on :5223, and tried to connect with both `s_client’ and spark 2.6.0b2, and while the behavior was slightly different, it didn’t work, and I still get:


No client certificate CA names sent

SSL handshake has read 2645 bytes and written 1446 bytes

New, TLSv1/SSLv3, Cipher is EDH-RSA-DES-CBC3-SHA
Server public key is 2048 bit
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1
Cipher : EDH-RSA-DES-CBC3-SHA
Session-ID: 49EC71B1A0D238E05CEABF61C23F1F9D019CDE93E404EBD90B7BD3C032F289C3
Session-ID-ctx:
Master-Key: 9118A0114F22471F0C83C839E48BEE8DA0BD0379E9863C013BA4CDDB79BB222E7FFE980B7FC9989 6F1420E0B55EFD627
Key-Arg : None
Start Time: 1240232369
Timeout : 300 (sec)
Verify return code: 0 (ok)

However, on :5223, I do get the following error in the server logs, presumably it’s not finding/opening the “client.truststore” properly:

[…]

Caused by: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
at sun.security.validator.PKIXValidator.(Unknown Source)
at sun.security.validator.Validator.getInstance(Unknown Source)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.getValidator(Unknown Source)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkClientTrusted(Unknown Source)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkClientTrusted(Unknown Source)
at com.sun.net.ssl.internal.ssl.ServerHandshaker.clientCertificate(Unknown Source)
at com.sun.net.ssl.internal.ssl.ServerHandshaker.processMessage(Unknown Source)
at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Unknown Source)
at com.sun.net.ssl.internal.ssl.Handshaker$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.net.ssl.internal.ssl.Handshaker$DelegatedTask.run(Unknown Source)
at org.apache.mina.filter.support.SSLHandler.doTasks(SSLHandler.java:686)
at org.apache.mina.filter.support.SSLHandler.handshake(SSLHandler.java:486)
… 16 more

Caused by: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
at java.security.cert.PKIXParameters.setTrustAnchors(Unknown Source)
at java.security.cert.PKIXParameters.(Unknown Source)
at java.security.cert.PKIXBuilderParameters.(Unknown Source)
… 29 more

Thanks,

Kevin

(grr… the system ate my first attempt at this reply – some sort of authentication mix-up.)

The openssl s_client command wont work with starttls on xmpp (it dosnt really understand the protocol) so you will want to have port 5223 (SSL) enabled to test with it. Your output confirms my suspicion, that the server is not reading the list of CA’s needed.

So lets make sure we have it all right:

You have the CA you wish to authenticate clients with in the client.truststore file (imported using the keytool command)

The location of this file is specified in the property xmpp.socket.ssl.client.truststore

The password to the client.truststore file is either “changeit” or specified in xmpp.socket.ssl.client.trustpass

The property xmpp.client.tls.policy is set to either “wanted” or “needed” (anything else sets it to “disabled”)

The property xmpp.client.certificate.crl is set, and points to a file with all the PEM encoded CRL’s of every CA in client.truststore

If the crl file specified, they

  • They must be there

  • They must be valid (not expired)

If the file is not found (or the property is null, etc) then CRLs will not be used. Debug output will say whats going on, just look for the lines prefixed with "ClientTrustManager: "

OCSP is also possible if you have an OCSP responder.

Hi. Yes, those properties have always been set (except CRL; I’ve tried that, though, and it doesn’t have much effect… I’ll try it again):

xmpp.socket.ssl.client.trustpass

changeit

xmpp.socket.ssl.client.truststore

/opt/openfire/resources/security/client.truststore

and

xmpp.client.cert.policy

needed

xmpp.client.certificate.accept-selfsigned

true

xmpp.client.certificate.verify

true

xmpp.client.certificate.verify.chain

true

xmpp.client.certificate.verify.root

true

xmpp.client.certificate.verify.validity

true

xmpp.client.tls.policy

required

just to be sure, I went ahead and generated it again:

xmpp.client.certificate.crl

/opt/openfire/resources/security/CRLs.pem

But still:


No client certificate CA names sent

SSL handshake has read 2645 bytes and written 1446 bytes

I even turned on accept-selfsigned, as you see, just to make it easier.

The truststore is valid and populated:

$ ls -ltr /opt/openfire/resources/security/client.truststore
-rw-r–r-- 1 daemon daemon 1772 Apr 14 10:41 /opt/openfire/resources/security/client.truststore

$ keytool -keystore /opt/openfire/resources/security/client.truststore -list -storepass changeit

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 2 entries

test_ca, Apr 9, 2009, trustedCertEntry,
Certificate fingerprint (MD5): 47:D2:6C:5E:B3:20:4C:8A:2B:62:6F:8C:A8:4D:30:F2
test_ca_2, Apr 14, 2009, trustedCertEntry,
Certificate fingerprint (MD5): 76:63:3B:28:57:0A:DC:77:A0:E2:3A:EF:1C:18:39:3A

have you enabled the debug log? Please look for anything from the ClientTrustManager. Often some of these messages will only happen on the first SSL/TLS session since the server started up- so perhaps shut the server down, clear the logs, start it up, and watch from the beginning.

You mean this in conf/openfire.xml, right?

true

If so, then yes. All I’m getting after normal startup:

2009.04.20 11:17:09 Started server (unencrypted) socket on port: 5269
2009.04.20 11:17:09 Started plain (unencrypted) socket on port: 5222
2009.04.20 11:17:09 Started SSL (encrypted) socket on port: 5223

is:

==> /opt/openfire/logs/error.log <==
2009.04.20 11:18:07 [org.jivesoftware.openfire.nio.ConnectionHandler.exceptionCaught(ConnectionHand ler.java:110)]
java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
at com.sun.net.ssl.internal.ssl.Handshaker.checkThrown(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLEngineImpl.checkTaskThrown(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLEngineImpl.readNetRecord(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLEngineImpl.unwrap(Unknown Source)
at javax.net.ssl.SSLEngine.unwrap(Unknown Source)
at org.apache.mina.filter.support.SSLHandler.unwrap0(SSLHandler.java:658)
at org.apache.mina.filter.support.SSLHandler.unwrapHandshake(SSLHandler.java:614)
at org.apache.mina.filter.support.SSLHandler.handshake(SSLHandler.java:493)
[…]

Caused by: java.lang.Exception: Disallowed character
at org.jivesoftware.openfire.nio.XMLLightweightParser.read(XMLLightweightParser.ja va:211)
at org.jivesoftware.openfire.nio.XMPPDecoder.doDecode(XMPPDecoder.java:32)
at org.apache.mina.filter.codec.CumulativeProtocolDecoder.decode(CumulativeProtoco lDecoder.java:133)
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:163)
… 8 more

from an `s_client’ connect.

There is a separate log file for debug messages. Same directory, named debug.log

I missed the <enabled …/>. However, nothing about ClientTrustManager:

$ grep -i ClientTrustManager *.log
$ grep -i Client *.log
Binary file debug.log matches
error.log: at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkClientTrusted(Unknown Source)
error.log: at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkClientTrusted(Unknown Source)
error.log: at com.sun.net.ssl.internal.ssl.ServerHandshaker.clientCertificate(Unknown Source)
error.log: at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkClientTrusted(Unknown Source)
error.log: at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkClientTrusted(Unknown Source)
error.log: at com.sun.net.ssl.internal.ssl.ServerHandshaker.clientCertificate(Unknown Source)

debug.log got binary data after attempted `s_client’ connect. The line from debug.log that matches is just:

2009.04.20 11:31:11 Set parameter http.useragent = Jakarta Commons-HttpClient/3.1

I do see the null cert chain in the debug log now, though:

[…]

Caused by: javax.net.ssl.SSLHandshakeException: null cert chain
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLEngineImpl.fatal(Unknown Source)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Unknown Source)

[…]

Hi,

I got exactly the same problem. Well I’m using Smack in Version 3.1.0 instead of Spark, but i think this is not of any relevance.

I’ve debuged the TLS Handshake with -Djavax.net.debug=all.

There I was able to see that the Handshake aborts at the very end of the Handshake, they even have the Mastersecret.

I’ve appended the debug log, maybe its helpfull.

Server quits with:

2009.04.21 15:03:26 NIOConnection: startTLS: using c2s
2009.04.21 15:03:26 ClientTrustmanager: crl file not found resources\security\crl.pem
2009.04.21 15:03:26 ConnectionHandler:
javax.net.ssl.SSLHandshakeException: SSL handshake failed.

Caused by: javax.net.ssl.SSLHandshakeException: null cert chain
at com.sun.net.ssl.internal.ssl.Handshaker.checkThrown(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLEngineImpl.checkTaskThrown(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLEngineImpl.writeAppRecord(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLEngineImpl.wrap(Unknown Source)

The crl File is right in place. I don’t know why Openfire doesn’t find it.

regards Daniel
tls-error.txt (48131 Bytes)