OpenFire 3.10.1 Upgrade Breaks Authentication

Hey Guys,

Sorry to complain, but I am having a terrible time trying to upgrade to OpenFire 3.10.1.

I am currently running 3.9.3 and I am an Active Directory domain for authentication. I am connected to my AD LDAP servers using LDAP SSL on port 636. Everything works great.

I also have a real SSL certificate imported into the truststore. The SSL certificate is a real CA signed certificate from StartSSL. This certificate, the private key, and CA trusted root chain is all imported and working great. The admin interface and all XMPP connections are mandatory SSL and it’s wonderful.

Now, when I upgrade, I shutdown the OpenFire daemon, back everything up, unpack the 3.10.1 tar file, restore the conf and resources/security directory, and fire everything up.

This seems to work fine, the SSL interface works fine, and all my settings are there, either from conf or presumably from the MySQL database.

I can hit the admin interface, SSL is up, everything is great.

Now, I fire up an XMPP client. Doesn’t matter if it’s Adium on Mac OS X or Pidgin on Linux. No matter what, the authentication process just gets stuck and doesn’t move. Adium is nice enough to show “75% authenticating” and just stays there for a long time. A tcpdump on the server doesn’t show any SSL TCP 636 connections. It’s like it’s not even trying.

I have tried disabling LDAP SSL and using regular cleartext LDAP on 389. No change. Still hangs at the authentication step.

Then, I can stop the OpenFire daemon again, unpack my backup file, start it back up, and everything is normal again: I can SSL, I can authenticate in milliseconds, and there are no problems getting in. SSL for XMPP, SSL for LDAP, and SSL for the HTTP Admin interface all work perfectly when I revert back to 3.9.3.

Setting the LDAP Debug mode radio button (or whatever it’s labeled) does nothing. I don’t seem to get any additional information written to any of the existing logs, nor does it seem to create any ‘special’ debug logs.

I DON’T have the Asterisk plugin installed. Asterisk is listed under “Available” plugins but it is not installed.

So… seems there’s a problem with the way I’m using OpenFire. Everything is fine on 3.9.3 but I just can’t seem to get things to work on 3.10.1. What do I do now?

Install 3.10.2 and see if it works.

I just gave it a shot. LDAP SSL is broke in 3.10.2. I can’t validate my connection into, setting the port to 636, setting the radio button to SSL and pushing “test”… just returns “Error connecting to the LDAP server. Ensure that the directory server is running at the specified host name and port and that a firewall is not blocking access to the server.” :frowning:

Good news though, the “Authenticating” hang I was getting does appear to be resolved. Time to play whack-a-mole and figure out why I can’t LDAPS any more…

Please look over the various openfire log files and see if there are any clues to LDAPS connection failures.

Thanks. I found this getting logged to nohup.out whenever I attempt to swap to LDAP SSL and push the Test button. Does it mean anything to you? Not sure what to do with this. I suck at reading Java exceptions. I’m guessing something is suddenly broke with how it’s validating the LDAP SSL Certificate chain, but I don’t really understand what exactly has changed between 3.10.1 and 3.10.2. This part actually used to work on 3.10.1, even though the actual XMPP authentication piece used to hang up.

javax.naming.CommunicationException: simple bind failed: ldap.muxier.com:636 [Root exception is javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target]

at com.sun.jndi.ldap.LdapClient.authenticate(Unknown Source)

at com.sun.jndi.ldap.LdapCtx.connect(Unknown Source)

at com.sun.jndi.ldap.LdapCtx.(Unknown Source)

at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(Unknown Source)

at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(Unknown Source)

at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(Unknown Source)

at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(Unknown Source)

at javax.naming.spi.NamingManager.getInitialContext(Unknown Source)

at javax.naming.InitialContext.getDefaultInitCtx(Unknown Source)

at javax.naming.InitialContext.init(Unknown Source)

at javax.naming.ldap.InitialLdapContext.(Unknown Source)

at org.jivesoftware.util.JiveInitialLdapContext.(JiveInitialLdapContext.java :43)

at org.jivesoftware.openfire.ldap.LdapManager.getContext(LdapManager.java:548)

at org.jivesoftware.openfire.ldap.LdapManager.getContext(LdapManager.java:462)

at org.jivesoftware.openfire.admin.setup.setup_002dldap_002dserver_005ftest_jsp._j spService(setup_002dldap_002dserver_005ftest_jsp.java:67)

at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)

at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:808)

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.ja va:1669)

at com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:39)

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.ja va:1652)

at org.jivesoftware.util.LocaleFilter.doFilter(LocaleFilter.java:74)

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.ja va:1652)

at org.jivesoftware.util.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingF ilter.java:50)

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.ja va:1652)

at org.jivesoftware.admin.PluginFilter.doFilter(PluginFilter.java:78)

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.ja va:1652)

at org.jivesoftware.admin.AuthCheckFilter.doFilter(AuthCheckFilter.java:159)

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.ja va:1652)

at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)

at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)

at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:22 3)

at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:11 27)

at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)

at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185 )

at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:106 1)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)

at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandler Collection.java:215)

at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.jav a:110)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)

at org.eclipse.jetty.server.Server.handle(Server.java:497)

at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)

at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)

at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)

at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635 )

at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)

at java.lang.Thread.run(Unknown Source)

Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

at sun.security.ssl.Alerts.getSSLException(Unknown Source)

at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)

at sun.security.ssl.Handshaker.fatalSE(Unknown Source)

at sun.security.ssl.Handshaker.fatalSE(Unknown Source)

at sun.security.ssl.ClientHandshaker.serverCertificate(Unknown Source)

at sun.security.ssl.ClientHandshaker.processMessage(Unknown Source)

at sun.security.ssl.Handshaker.processLoop(Unknown Source)

at sun.security.ssl.Handshaker.process_record(Unknown Source)

at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)

at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)

at sun.security.ssl.SSLSocketImpl.writeRecord(Unknown Source)

at sun.security.ssl.AppOutputStream.write(Unknown Source)

at java.io.BufferedOutputStream.flushBuffer(Unknown Source)

at java.io.BufferedOutputStream.flush(Unknown Source)

at com.sun.jndi.ldap.Connection.writeRequest(Unknown Source)

at com.sun.jndi.ldap.Connection.writeRequest(Unknown Source)

at com.sun.jndi.ldap.LdapClient.ldapBind(Unknown Source)

… 48 more

Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

at sun.security.validator.PKIXValidator.doBuild(Unknown Source)

at sun.security.validator.PKIXValidator.engineValidate(Unknown Source)

at sun.security.validator.Validator.validate(Unknown Source)

at sun.security.ssl.X509TrustManagerImpl.validate(Unknown Source)

at sun.security.ssl.X509TrustManagerImpl.checkTrusted(Unknown Source)

at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(Unknown Source)

… 61 more

Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(Unknown Source)

at java.security.cert.CertPathBuilder.build(Unknown Source)

… 67 more

I tried (to no avail) to import the SSL certificate for the LDAPS system into client.truststore, keystore, and truststore. Nothing. Still can’t get it to use LDAPS.

I have a real cert on the Active Directory system as well, from the same CA (StartSSL) … so if the HTTP SSL and XMPP SSL works then it should definitely trust and use the LDAPS from a client perspective… I would think?

I don’t know what gives. Very frustrating though.

I’m using ldaps without any problem.

With prev version, ldaps connection would use any certificate (included untrusted/self signed/etc), at the cost of not being able to use connection pools. That behavior changed. You’re on the right track. Check to see if you have a root certificate from you CA in your java store. Thats likely the issue, and you’ll need to import your root CA into the JRE that Openfire is using. Here is an example of the command

“C:\Program Files (x86)\Java\jre1.8.0_45\bin\keytool” -importcert -keystore “C:\Program Files (x86)\Java\jre1.8.0_45\lib\security\cacerts” -storepass changeit -file ROOTCA.cer -noprompt

Weird, because this used to ‘just work’. I have a StartSSL free signed cert, with the chain, running in Active Directory. I can use ldapsearch from the server where OpenFire is running and it validates using the local OpenSSL install and that’s fine. The chain validates. Not sure why 3.9.3 was fine without modifying the system’s cacerts, but 3.10.2 requires the chain to be added… does that make sense?

I tried doing what you said and initially it failed. I later realized that OpenFire has to be bounced in order to pick up the change, as it appears that the JVM is somehow caching the contents of the store and doesn’t pick up changes made after it was started.

Here’s what did work:

[blentz@muxrhim01 ~]$ sudo service openfire stop

Shutting down openfire: [ OK ]

[blentz@muxrhim01 ~]$ sudo cp /usr/java/default/lib/security/cacerts /usr/java/default/lib/security/cacerts.orig

[blentz@muxrhim01 ~]$ sudo /usr/java/default/bin/keytool -keystore /usr/java/default/lib/security/cacerts -importcert -file ldap.muxier.com.cachain.crt -storepass changeit -noprompt -alias ldap.muxier.com.cachain.crt

Certificate was added to keystore

[blentz@muxrhim01 ~]$ sudo service openfire start

Starting openfire:

[blentz@muxrhim01 ~]$

Thank you for your help (both of you). Much appreciated! I don’t have to revert back to 3.9.3 again!

Ben,

This was my fault

The previous code used for connected to ldap over ssl used a custom ssl socket factory. The custom socket would pretty much accept any ssl cert, trusted or not, expired, self signed, etc. It really didn’t care!

When the custom socket was used, this prevented connection pooling over SSL from taking place, which killed ldap performance of ssl.

The quick fix (for me anyway) was to remove the use of the custom socket. Sorry for the inconvenience

Absolutely no need to apologize! I sincerely appreciate the help and I am glad that the mystery is solved!

As a responsible end-user, I did sift through the changelog file but I didn’t see anything that jumped out at me as though it indicated a change to LDAP SSL handling. I probably missed it.

I am happy! Thank you for your support and thank you for making such excellent software. OpenFire is awesome!

I agree…Openfire is awesome, but thank the devs…they do all the work. the ldap thing was my first contribution to code…and outside of vbs scripts/bat files/easy stuff, I’ve never got into coding…so that was my very first bit of java. I’m just glad youre up and running!