CertBot – Unable to Use LetsEncrypt When HTTP to HTTPS Redirect is Setup

certbothttpslets-encryptnginx

I am trying to configure CertBot and it only works when I serve my site over http. Usually I have an https redirect and I don't want to have to change the site config each time I need to use certbot. I tried to serve only /.well-known/ over http but it is still failing any ideas how to resolve this?

I am trying to copy this idea but not working –> NGINX redirect everything except letsencrypt to https

Eg: This Works:

server {
    listen 80;
    listen [::]:80;
    server_name example.com www.example.com;


location / {

        proxy_pass              http://localhost:8575/;
        include                 /etc/nginx/conf.d/proxy.conf;
    }
}

This does not: (Note that the current configured SSL Certs are not correct, but needed for NGinX to start)

server {

   listen 80;
   listen [::]:80;
   server_name www.example.com example.com;

    location /.well-known/acme-challenge/ {
        proxy_pass              http://localhost:8575/;
        include                 /etc/nginx/conf.d/proxy.conf;
    }

location / {
       return 301 https://$server_name$request_uri;
    }

}

server {
        listen 443 ssl;
        listen        [::]:443;
        server_name www.example.com example.com;

#        ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
#        ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
        ssl_certificate /etc/ssl/crt/crt.crt;
        ssl_certificate_key /etc/ssl/crt/key.key;

location / {

        proxy_pass              http://localhost:8575/;
        include                 /etc/nginx/conf.d/proxy.conf;
    }
}

Error Log:

certbot    | Saving debug log to /var/log/letsencrypt/letsencrypt.log
certbot    | Plugins selected: Authenticator webroot, Installer None
certbot    | Registering without email!
certbot    | Obtaining a new certificate
certbot    | Performing the following challenges:
certbot    | http-01 challenge for www.example.com
certbot    | http-01 challenge for example.com
certbot    | Using the webroot path /var/www/html for all unmatched domains.
certbot    | Waiting for verification...
certbot    | Challenge failed for domain www.example.com
certbot    | Challenge failed for domain example.com
certbot    | http-01 challenge for www.example.com
certbot    | http-01 challenge for example.com
certbot    | Cleaning up challenges
certbot    | IMPORTANT NOTES:
certbot    |  - The following errors were reported by the server:
certbot    |
certbot    |    Domain: www.example.com
certbot    |    Type:   unauthorized
certbot    |    Detail: Invalid response from
certbot    |    http://www.example.com/.well-known/acme-challenge/WyVEA5g6BWVDPpYUhEJ0bG5iH6daF1rZpFd0vuTXOa0
certbot    |    [50.117.156.123]: "        <!DOCTYPE html><html lang=\"en-US\">\r\n
certbot    |    \t<head>\n\n\t\t        <meta charset=\"UTF-8\">\r\n        <meta
certbot    |    name=\"viewport\" con"
certbot    |
certbot    |    Domain: example.com
certbot    |    Type:   unauthorized
certbot    |    Detail: Invalid response from
certbot    |    https://www.example.com/x61_h9wxFY2Ye8-16GllyMq_dfsXbsEB1lYOjeq4LjU
certbot    |    [50.117.156.123]: "        <!DOCTYPE html><html lang=\"en-US\">\r\n
certbot    |    \t<head>\n\n\t\t        <meta charset=\"UTF-8\">\r\n        <meta
certbot    |    name=\"viewport\" con"
certbot    |
certbot    |    To fix these errors, please make sure that your domain name was
certbot    |    entered correctly and the DNS A/AAAA record(s) for that domain
certbot    |    contain(s) the right IP address.
certbot    |  - Your account credentials have been saved in your Certbot
certbot    |    configuration directory at /etc/letsencrypt. You should make a
certbot    |    secure backup of this folder now. This configuration directory will
certbot    |    also contain certificates and private keys obtained by Certbot so
certbot    |    making regular backups of this folder is ideal.
certbot    | Some challenges have failed.
certbot exited with code 1

Best Answer

With HTTP-01 challenge it's OK to redirect to HTTPS first and serve the challenge over TLS. However, the challenge always starts with a plain HTTP connection using port 80, and you can only redirect to HTTPS on port 443.

Our implementation of the HTTP-01 challenge follows redirects, up to 10 redirects deep. It only accepts redirects to “http:” or “https:”, and only to ports 80 or 443. It does not accept redirects to IP addresses. When redirected to an HTTPS URL, it does not validate certificates (since this challenge is intended to bootstrap valid certificates, it may encounter self-signed or expired certificates along the way).

The HTTP-01 challenge can only be done on port 80. Allowing clients to specify arbitrary ports would make the challenge less secure, and so it is not allowed by the ACME standard.

Therefore, this kind of Nginx configuration should work, as well:

server {
    listen 80;
    server_name www.example.com example.com;

    location / {
        return 301 https://$server_name$request_uri;
    }
}

server {
    listen 443 ssl;
    server_name www.example.com example.com;

    location /.well-known/acme-challenge/ {
        # HTTP-01 challenge
    }

    location / {
        # Your web application
    }
}

In your case this means the following could be either in the HTTP or the HTTPS server block.

location /.well-known/acme-challenge/ {
    proxy_pass   http://localhost:8575/.well-known/acme-challenge/;
    include      /etc/nginx/conf.d/proxy.conf;
}

You were able to replace the /.well-known/acme-challenge/ with $request_uri because:

When variables are used in proxy_pass:

location /name/ {
    proxy_pass http://127.0.0.1$request_uri;
}

In this case, if URI is specified in the directive, it is passed to the server as is, replacing the original request URI.

Also, if the / has the same root, you don't need a separate location at all.

Related Topic