Currently it is possible to perform DNS validation, also with the certbot LetsEncrypt client in manual mode. Automation is possible as well (see below).
Manual plugin
You can either perform a manual verification - with the manual plugin.
certbot -d bristol3.pki.enigmabridge.com --manual --preferred-challenges dns certonly
Certbot will then provide you instructions to manually update a TXT record for the domain in order to proceed with the validation.
Please deploy a DNS TXT record under the name
_acme-challenge.bristol3.pki.enigmabridge.com with the following value:
667drNmQL3vX6bu8YZlgy0wKNBlCny8yrjF1lSaUndc
Once this is deployed,
Press ENTER to continue
Once you have updated the DNS record, press Enter, certbot will continue and if the LetsEncrypt CA verifies the challenge, the certificate is issued as normally.
You may also use a command with more options to minimize interactivity and answering certbot questions. Note that the manual plugin does not yet support non-interactive mode.
certbot --text --agree-tos --email you@example.com -d bristol3.pki.enigmabridge.com --manual --preferred-challenges dns --expand --renew-by-default --manual-public-ip-logging-ok certonly
Renewal does not work with the manual plugin as it runs in non-interactive mode. More info in the official certbot documentation.
Update: manual hooks
In the new certbot version you can use hooks, e.g., --manual-auth-hook
, --manual-cleanup-hook
. The hooks are external scripts executed by certbot to perform the task.
Information is passed in environment variables - e.g., domain to validate, challenge token. Vars: CERTBOT_DOMAIN
, CERTBOT_VALIDATION
, CERTBOT_TOKEN
.
certbot certonly --manual --preferred-challenges=dns --manual-auth-hook /path/to/dns/authenticator.sh --manual-cleanup-hook /path/to/dns/cleanup.sh -d secure.example.com
You can write your own handler or use already existing ones. There are many available, e.g., for Cloudflare DNS.
More info on official certbot hooks documentation.
Automation, Renewal, Scripting
If you would like to automate DNS challenge validation it is not currently possible with vanilla certbot. Update: some automation is possible with the certbot hooks.
We thus created a simple plugin that supports scripting with DNS automation. It's available as certbot-external-auth.
pip install certbot-external-auth
It supports the DNS, HTTP, TLS-SNI validation methods. You can either use it in handler mode or in JSON output mode.
Handler mode
In handler mode, the certbot + plugin calls external hooks (a program, shell script, Python, ...) to perform the validation and installation. In practice you write a simple handler/shell script which gets the input arguments - domain, token and makes the change in DNS. When the handler finishes, certbot proceeds with validation as usual.
This gives you extra flexibility, renewal is also possible.
Handler mode is also compatible with Dehydrated DNS hooks (former letsencrypt.sh). There are already many DNS hooks for common providers (e.g., CloudFlare, GoDaddy, AWS). In the repository there is a README with extensive examples and example handlers.
Example with Dehydrated DNS hook:
certbot \
--text --agree-tos --email you@example.com \
--expand --renew-by-default \
--configurator certbot-external-auth:out \
--certbot-external-auth:out-public-ip-logging-ok \
-d "bristol3.pki.enigmabridge.com" \
--preferred-challenges dns \
--certbot-external-auth:out-handler ./dehydrated-example.sh \
--certbot-external-auth:out-dehydrated-dns \
run
JSON mode
Another plugin mode is JSON mode. It produces one JSON object per line. This enables a more complicated integration - e.g., when Ansible or some deployment manager is calling certbot. Communication is performed via STDOUT and STDIN. Cerbot produces JSON objects with data to perform the validation, for example:
certbot \
--text --agree-tos --email you@example.com \
--expand --renew-by-default \
--configurator certbot-external-auth:out \
--certbot-external-auth:out-public-ip-logging-ok \
-d "bristol3.pki.enigmabridge.com" \
--preferred-challenges dns \
certonly 2>/dev/null
{"cmd": "perform_challenge", "type": "dns-01", "domain": "bs3.pki.enigmabridge.com", "token": "3gJ87yANDpmuuKVL2ktfQ0_qURQ3mN0IfqgbTU_AGS4", "validation": "ejEDZXYEeYHUxqBAiX4csh8GKkeVX7utK6BBOBshZ1Y", "txt_domain": "_acme-challenge.bs3.pki.enigmabridge.com", "key_auth": "3gJ87yANDpmuuKVL2ktfQ0_qURQ3mN0IfqgbTU_AGS4.tRQM98JsABZRm5-NiotcgD212RAUPPbyeDP30Ob_7-0"}
Once DNS is updated, the caller sends the new-line character to STDIN of certbot to signal it can continue with validation.
This enables automation and certificate management from the central management server. For installation you can deploy certificates over SSH.
For more info please refer to the readme and examples on certbot-external-auth GitHub.
EDIT: There is also a new blog post describing the DNS validation problem and the plugin usage.
EDIT: We currently work on Ansible 2-step validation, will be soon off.
Two certificates are considered the same as long as the Subject Name and the Public Key of the certificates match.
Therefore, all you need to do is to re-use the keys and ensure that the Subject Name in the new certificate is the same as the old. After that, you can change any of the other fields and/or extensions and the resulting certificate will be considered the same.
For example, create your OpenSSL configuration file:
[ req ]
prompt = no
string_mask = default
distinguished_name = x509_distinguished_name
x509_extensions = x509_ext
[ x509_distinguished_name ]
# Note that the following are in 'reverse order' to what you'd expect to see.
# Adjust for your setup:
countryName = za
organizationName = My Company
organizationalUnitName = Security
commonName = My Root CA
[ x509_ext ]
basicConstraints = critical,CA:true
keyUsage = critical, keyCertSign, cRLSign
subjectKeyIdentifier = hash
crlDistributionPoints = URI:http://security.mycompany.co.za/root.crl
and save it as e.g. rootca.cnf
. Ensure that the elements of the [req_distinguished_name]
are identical to the ones in your original Root CA certificate (this is the identical Subject Name part).
Next, run:
openssl req -new -x509 -key rootca.key -out MyNewCA.pem -config rootca.cnf
where rootca.key
is the private key used in the original certificate (this is the identical Public/Private Key part).
This creates MyNewCA.pem
, which you can check with:
$ openssl x509 -noout -text -in MyNewCA.pem
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 17564687072266118846 (0xf3c24dd49d5166be)
Signature Algorithm: sha256WithRSAEncryption
Issuer: C=za, O=My Company, OU=Security, CN=My Root CA
Validity
Not Before: Jul 15 05:05:54 2017 GMT
Not After : Aug 14 05:05:54 2017 GMT
Subject: C=za, O=My Company, OU=Security, CN=My Root CA
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:c8:3d:32:8a:56:31:f6:27:1a:ce:9e:b2:1d:fb:
ce:9f:ce:5b:42:25:aa:fe:8b:f4:34:32:ac:b3:02:
50:71:f8:c3:43:0c:c7:2c:9f:fe:48:1b:c6:c0:e7:
d6:44:a9:e7:d7:a0:7e:58:f4:b6:38:61:7e:d0:af:
0f:56:21:e7:49:7a:66:13:f5:7a:fe:3d:ab:65:f8:
01:eb:52:a7:3b:ae:a0:cf:50:57:b9:e0:09:0b:1f:
90:14:fb:18:56:1d:57:99:a9:76:a2:63:d1:c2:d3:
a3:f4:3a:ff:91:0d:ee:8d:44:13:58:00:09:93:da:
e8:6a:fd:64:5f:c3:42:8e:2a:49:6e:0d:73:b7:b9:
d4:6c:c6:ce:30:c5:82:24:a5:00:37:17:a0:1d:f1:
c9:e9:e3:18:48:22:4f:33:96:a7:3c:a9:31:f1:3f:
14:40:6a:74:e8:78:82:45:04:d4:4b:56:0b:cd:be:
48:8d:03:fb:39:70:0b:91:99:70:06:bd:5e:8b:f2:
d1:f4:6f:fc:ce:e7:f8:3c:0a:20:ea:35:b8:5f:2f:
ee:8d:ff:d3:93:85:6b:fb:71:db:1b:e6:e9:1d:a7:
f8:e4:ae:f4:71:fe:35:a7:89:24:af:69:a4:34:3b:
14:66:05:02:5e:2a:1d:ac:e0:d2:48:6c:13:4e:75:
58:93
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Basic Constraints: critical
CA:TRUE
X509v3 Key Usage: critical
Certificate Sign, CRL Sign
X509v3 Subject Key Identifier:
3B:45:93:3A:2A:BC:39:29:36:13:6C:BD:B6:B4:31:C7:E7:BB:32:73
X509v3 CRL Distribution Points:
Full Name:
URI:http://security.mycompany.co.za/root.crl
Signature Algorithm: sha256WithRSAEncryption
4d:96:d4:03:4f:e3:7c:26:be:59:f8:23:87:60:f7:4c:d3:a1:
1c:77:a1:14:e3:e7:da:c8:2a:a3:1b:06:2a:4d:55:1c:83:26:
73:46:0d:8a:e4:b7:d1:1e:38:cc:78:90:00:01:b3:8e:f9:3c:
62:be:04:09:90:4e:22:87:b1:aa:bd:f9:73:bd:a7:76:ad:d5:
ae:2d:7a:1c:1e:1a:67:c8:57:4c:f9:6d:8b:62:d6:e5:ea:e0:
40:5c:12:28:7e:ea:f0:0c:d6:cd:f4:1d:d5:56:09:ad:43:b4:
eb:8c:68:ce:56:a2:a8:ae:a4:d5:35:bb:58:b8:ed:82:82:b5:
ef:cb:e2:6d:76:61:ed:ee:a5:1f:68:95:07:ed:5b:f0:65:92:
d2:dc:1d:c6:fa:7f:e0:c9:38:c2:c6:6f:03:38:e7:3a:b0:24:
06:e0:bc:07:dd:e7:a0:dc:74:09:e5:37:7b:66:e1:6f:47:4c:
71:ff:02:48:7f:d4:4f:ce:cb:91:e9:ee:cd:b6:f1:0a:03:19:
3e:19:05:7d:8f:48:e7:f1:cc:07:37:3d:91:3c:6f:54:71:3c:
a2:6c:55:c3:03:c1:7f:eb:9e:70:f1:8f:a1:fb:62:33:8b:86:
2c:79:bc:76:e2:01:9a:68:df:af:40:a1:b2:9c:f6:a1:e1:6e:
2a:dd:1a:d6
Use this new certificate in place of the original.
You can change other fields and extensions, such as the validity period of the certificate, but bear in mind that you shouldn't really have any constraints (other than basicConstraints = critical,CA:true
) on the Root CA certificate.
After further consideration, your issue may simply be down to the fact that your replacement Root CA certificate doesn't have the basicConstraint
and possibly the keyUsage
extensions. It might be worth trying adding those two extensions to your ext.conf
first and testing the resulting new Root CA certificate using the -x509toreq
method you posted.
Best Answer
The basic infrastructure, that would make this possible, exists and is called DNS-Based Authentication of Named Entities (DANE) and specified in RFC6698. It works by means of a
TLSA
resource record, that specifies the certificate or its public key of the end entity or one of its CAs in the chain (There are actually four different types, see the RFC for details).Adoption
DANE has however not seen widespread adoption yet. VeriSign monitors DNSSEC and DANE adoption and tracks its growth over time:
For comparison, according to VeriSign, there exists about 2.7 million DNS zones, so that means that a bit more than 1% of all zones have at least one TLSA record.
I can't give any authoritative answer, why DANE, but here are my speculations:
DANE suffers from the same problem as Certificate Revocation Lists (CRLs) and the Online Certificate Status Protocol (OCSP). In order to verify the validity of a presented certificate, a third party must be contacted. Hanno Böck gives a good overview, why this is a big problem in practice. It boils down to the issue of what to do, when you can't reach the third party. Browser vendors opted to soft-fail (aka permit) in this case, which made the whole thing rather pointless and Chrome ultimately decided to disable OCSP in 2012.
DNSSEC
Arguably DNS offers much better availability than the CRL and OCSP servers of CAs, but it still makes offline verification impossible. In addition DANE, should only be used in conjunction with DNSSEC. As normal DNS operates over unauthenticated UDP, it is quite prone to forgery, MITM attacks, etc. The adoption of DNSSEC is much better than the adoption of DANE, but is still far from ubiquitous.
And with DNSSEC we run again into to the soft-fail problem. To the best of my knowledge no major server/client operating systems provides a validating DNSSEC resolver by default.
Then there is also the issue of revocation. DNSSEC has no revocation mechanism and relies on short lived keys instead.
Software Support
All participating software must implement DANE support.
In theory, you might think, that this would be the job of crypto libraries and application developers wouldn't have to do much, but the fact is, cryptographic libraries typically only provide primitives and applications have to do a lot of configuration and setup themselves (and there unfortunately many ways to get things wrong).
I'm not aware, that any major web server (e.g. Apache or nginx) for example implemented DANE or has plans to do it. Web servers are of particular importance here, because more and more stuff is build on web technologies and therefore they are often the first, where things get implemented.
When we look at CRL, OCSP, and OCSP Stapling as comparison, we might be able to infer how the DANE adoption story will go. Only some of the applications, which use OpenSSL, libnss, GnuTLS, etc. support these features. It took a while for major software like Apache or nginx to support it and again referring back to Hanno Böck's article, they got it wrong and their implementation is flawed. Other major software projects, like Postfix or Dovecot don't support OCSP and have very limited CRL functionality, basically pointing to a file in the file system (which isn't necessarily re-read regulary, so you would have to reload your server manually etc). Keep in mind that these are projects, which frequently use TLS. Then you can start looking at things, where TLS is much less common, like PostgreSQL/MySQL for example and maybe they offer CRLs at best.
So I've not even modern web servers implement it and most other software hasn't even got around to implement OCSP and CRL, good luck with your 5 year enterprise application or appliance.
Potential Applications
So where could you actually use DANE? As of now, not on general Internet. If you control the server and the client, maybe its an option, but in this case, you can often resort to Public-Key Pinning.
In the mail space, DANE is getting some traction, because SMTP did not have any kind of authenticated transport encryption for the longest time. SMTP servers did sometimes use TLS between each other, but did not verify that the names in the certificates actually matched, they are now starting to check this through DANE.