Certbot fails to implement redirection to https for Apache2 web site

apache-2.4certbot

The question
I want to understand how to implement redirection from http to https, using Apache2 with certification from Certbot.

The details
On a fresh server running Ubuntu 16.04, I've installed a "Perfect Server" setup (ISPConfig, Postfix, Dovecot, and family) in order to run a small email server.

I've also used Certbot to install SSL Certificates. Before using Certbot, I manually added ServerName my.domain.com to four files that I found in the /etc/apache2/sites-available/ directory…

  • 00-default.conf
  • apps.vhost
  • default-ssl.conf
  • ispconfig.vhost

… and also to this file:

  • /etc/apache2/apache2.conf

My understanding is that this gives Certbot information about what domain names exist on the server, and ensures that the challenges will be met. (Perhaps this was overkill? Perhaps it was enough to edit only `apache2.conf?)

I then ran:

certbot --apache -d my.domain.com

During the installation, I selected the Redirect option:

Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
-------------------------------------------------------------------------------
1: No redirect - Make no further changes to the webserver configuration.
2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
new sites, or if you're confident your site works on HTTPS. You can undo this
change by editing your web server's configuration.
-------------------------------------------------------------------------------
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2

Here's the promise that Certbot made:

Redirecting vhost in /etc/apache2/sites-enabled/000-apps.vhost to
ssl vhost in /etc/apache2/sites-enabled/000-ispconfig.vhost

However, there are two issues:

  1. No redirection was proposed for the default site on port :80
  2. The files Certbot was talking about have never existed, as you can see here:

# ls -al /etc/apache2/sites-available/
total 36
drwxr-xr-x 2 root root 4096 Nov  9 17:29 .
drwxr-xr-x 9 root root 4096 Nov  9 16:44 ..
-rw-r--r-- 1 root root 1336 Nov  9 14:35 000-default.conf
-rw-r--r-- 1 root root 1340 Nov  9 16:44 apps.vhost
-rw-r--r-- 1 root root 1200 Nov  9 16:33 apps.vhost.save
-rw-r--r-- 1 root root 6387 Nov  9 15:08 default-ssl.conf
-rw-r--r-- 1 root root 1929 Nov  9 12:36 ispconfig.conf
-rw-r--r-- 1 root root 3349 Nov  9 16:44 ispconfig.vhost

Now, when I connect to http://my.domain.com, I see the Apache2 Ubuntu Default Page, with no redirection. If I connect to https://my.domain.com, then all browsers tell me that there is a problem. Firefox is most explicit:

Secure Connection Failed

An error occurred during a connection to my.domain.com.
SSL received a record that exceeded the maximum permissible length.
Error code: SSL_ERROR_RX_RECORD_TOO_LONG

The page you are trying to view cannot be shown because the
authenticity of the received data could not be verified.

However, when I visit the ISPConfig page at https://http://my.domain.com:8080/, everything works smoothly. Browsers are happy to show me the green "secure" sign at the right of the address bar.

Visiting http://http://my.domain.com:8080/ is polite, but not so successful:

Bad Request

Your browser sent a request that this server could not understand.
Reason: You're speaking plain HTTP to an SSL-enabled server port.
Instead use the HTTPS scheme to access this URL, please.

I note that Certbot has made changes to the file at /etc/apache/sites-available/ispconfig.vhost. In the extract below, the lines marked --- have been removed, and the lines +++ have been added.

# SSL Configuration
  SSLEngine On
  SSLProtocol All -SSLv3
  --- SSLCertificateFile /usr/local/ispconfig/interface/ssl/ispserver.crt
  --- SSLCertificateKeyFile /usr/local/ispconfig/interface/ssl/ispserver.key
  #SSLCACertificateFile /usr/local/ispconfig/interface/ssl/ispserver.bundle

  SSLCipherSuite ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-S$
  SSLHonorCipherOrder On

  <IfModule mod_headers.c>
  Header always add Strict-Transport-Security "max-age=15768000"
  RequestHeader unset Proxy early
  </IfModule>

  SSLUseStapling On
  SSLStaplingResponderTimeout 5
  SSLStaplingReturnResponderErrors Off
  +++ SSLCertificateFile /etc/letsencrypt/live/my.domain.com/fullchain.pem
  +++ SSLCertificateKeyFile /etc/letsencrypt/live/my.domain.com/privkey.pem
  +++ Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>

(Certbot has also added a directory at /etc/letsencrypt/ containing all the certification goodness.)

My goal
I am seeking to understand what changes I need to make to the files in /etc/apache2/sites-available so that http://my.domain.com is redirected silently to https://my.domain.com, both for the main site and also for the ISPConfig pages on port :8080.

In other words, I want to know how to do to real files what Certbot said it would do to files that existed only in its imagination.

Best Answer

The main issue here is duplicated server entries in the configuration. Because of that, it's likely that only some of your server configuration options are being loaded by Apache.

certbot --apache (run without the certonly option) reads the web server configuration, and attempts to automatically update the configuration of the site it's obtained a certificate for. If there are multiple entries for the site in the web server configuration, it may update a configuration that isn't being read - I think that's what's happening here.

There are several different approaches to organising web server config files, and it's easy to get them mixed up. Which one are you aiming to use? This Server Fault question might help.

EDIT: re-read about certbot redirects in question - removing the lines added by certbot and running certbot apache -d example.com again after the config is organised should allow certbot to automatically add redirect options.

Related Topic