While trying to diagnose problems I’ve been having with s2s tls, I came across the SubjectAlternativeName invalid type error and had a look at the code. I noticed that while looking for otherName attributes, it does no checking to see if an otherName is a XMPP otherName but just blindly assumes that it is. There was also some horrible looking code to unwrap the TaggedObject layers around the UTF8String in the XMPP otherName. I went ahead and added code to check the otherName objectId and only attempt to use those that match the XMPP otherName OID. I also cleaned up the unwrapping code so that it is more readable. Patch attached below as otherName.patch.
I haven’t seen any discussion on the board about the error message that is returned for the invalid otherName types, but it seems like the error message serves no real purpose other than to possibly confuse people. In my specific case, my certificate has a DNS otherName on it as well as the XMPP otherName, which is a valid combination and it shown on a jabber.org wiki page. The fact that OpenFire logs even a debug message complaining about a non-applicable type seems wrong to me. OpenFire should just silently ignore the types it doesn’t know/care about and move on. I’ve attached a second patch below, invalidType.patch, that will remove the debug log for the invalid type and change the comment to say that other types are not applicable and will be silently ignored.
I think the otherName patch needs to be applied because it cleans the code up and it also restricts exactly what type of otherName attributes that OpenFire attempts to use. The invalidType patch I would like to see applied, but that is more open to debate as someone else might see a use for that log line that I haven’t envisioned.
These patched were generated against the head of the Subversion trunk, but should apply cleanly to the release branch as well.