With reference to to the site: https://www.cloudflare.com/dns/dnssec/how-dnssec-works/
The test is based on aTalk, ported to use smack v4.3.0 and mindDNS v0.3.2. The DNSSEC validation is carried out using the babai.ru, and the same observation is reconfirmed on another domain in .ru zone.
Following describes the actual validation process tracing, until the problem is encountered:
1. smack and miniDns perform the RRSIG signatures validation using ZSK and KSK for domain (babai.ru) in zone (.ru) without any problem.
2. The problem occur when DnssClient is about to perform DS record validation with the root servers while generating the query. At which point, the variables and contents are as follow:
a. rrsig.signerName = "."
b. rrsig.typeCovered="DS"
c. q="ru. IN DS"
I am assuming that rrsig.singerName "'." is referring to the root server.
3a. While executing DnssecClient.verifySignedRecords(q, rrsig, records);
3b. it performs DnssecClient.queryDnssec(rrsig.signerName, TYPE.DNSKEY);
4a. The exception occurs when trying to perform rawAce = MiniDnsIdna.toASCII(name); where name = "."
4b. at idnaTransformator.toASCII(string); i.e. string="."
5. The error message "Invalid input to ASCII: ." was thrown by DefaultIdnaTransformator class.
Look like it fails to handle dot "." transformation.
I did not goes into full understanding of the miniDNS source, I apologize if my comments above do deviate from the actual signature verification flow.
Trying to be absolute sure, I did a recompilation of miniDns-core library with some source changes. However it seems AS or aTalk gradle setting configuration are causing problem. The generation of apk although without problem, it appear to have the multidex packed incorrectly with classes not related to smack or mindDns when running on android; aTalk runs but the user login process cannot be performed - still try to figure out why.
Note: On smack 4.2.4 with miniDNS 0.2.4, it throws a different error message i.e. “Validation of 2 DNSKEY records failed: Signature is invalid”. Look similar unable to validate the DNSKEY with the root server?