Openfire has the ssl certificate blues

Openfire Openfire Dev
1

the purpose of this post is two-fold:

  • document for system admins the necessary certificate work-around

  • provide feedback to developers about openfire’s certificate handling

this specifically pertains to openfire servers that provide the xmpp service at a domain (example.com), with one or more public services using subdomains (conference.example.com).

work-around

(first because i doubt system admins are interested in the java source code details and just want to “get it done”):

either

  1. have no “othername” items and have the wildcard be the last CN item in the “Subject” (ie keytool: -dname “CN=.example.com, CN=example.com, C=US"; openssl: -subj "/C=US/CN=example.com/CN=.example.com”)

or

  1. put the wildcard name in an “othername” item and have only one “othername” item (see http://wiki.jabber.org/index.php/XMPP_Server_Certificates)

afaict xmpp federation certs are incompatible with openfire when trying to use encryption with subdomains because the othername only contains the bare domain and the wildcards are always listed before the bare domain for CN items (and DNS items but openfire doesn’t look at them; see output of “openssl s_client -connect jabber.org:5223 | openssl x509 -noout -text”)

problem

now i present a few different overviews of the relevant code and discuss the specifics of the code.

(i apologize in advance for my liberal use of , but it was the only way i found to preserve indentation.)

call chain

ServerTrustMananger.checkServerTrusted()
`-- CertificateManager.getPeerIdentities()
    `-- CertificateManager.getSubjectAlternativeNames()

pseudo code

enter checkServerTrusted()
    enter getPeerIdentities()
        enter getSubjectAlternativeNames()
            for item in "X509v3 Subject Alternative Name"
                if item is of type "othername"
                    decode item to name
                    add name to list
        exit getSubjectAlternativeNames()
        if list is empty
            grab name from last CN in Subject
            add name to list
    exit getPeerIdentities()
    if only one name in list and it is a wildcard
        if name is not equal to remote server name
            throw exception
    else if list does not contain remote server name
        throw exception
exit checkServerTrusted()

priority

once a name is acquired from one of the following locations, in the order listed, no other locations are searched

  1. “othername” items in “X509v3 Subject Alternative Name”

  2. last “CN” in “Subject”

misbehavior

(based on my (mis)interpretation of xep-0178, section 3 “server-to-server recommendation”)

  • locations are exclusive (first location that contains at least one name is only location used), but should be complimentary, not supplementary

  • othername items are allowed to be wildcards, but othername items should only contain “bare domain”

  • the wildcard name “*.example.com” matches all subdomains of example.com (agreed), but also the bare domain example.com (disagree, but admittedly controversial)

  • “DNS” items in “X509v3 Subject Alternative Name” are not considered, but should be

  • “CN” items in “Subject” other than last “CN” are not considered, but should be

openfire should:

1. gather a list of all candidates from:
  a. othername items
  b. dns items
  c. CN items
2. iterate through the list of candidates
  a. if the candidate name is a wildcard, then remove leading "*" and do an endsWith match against host name
  b. otherwise do an equality/== match with host name

thanks for openfire!