Ssl – SMTP Problems due to CN vs SNI on the Certificate

opensslsmtpsslssl-certificate

I have a scenario where I'm using a shared mail server (cPanel) for SMTP on multiple domains for several clients. I hit an issue where Ruby's OpenSSL library complains that the hostname "smtp.domain2.tld" does not match the server certificate.

In this case I think the issue is that the main CN for the certificate is "domain2.tld" and while "mail.domain2.tld" is an accepted DNS SAN, it is not the main CN and if no SNI is requested it fails…

For example: if I connect with openssl without a "servername" directive, the certs I get back are for the server itself are server.main-domain.tld:

$ openssl s_client -connect mail.domain2.tld:587 -starttls smtp -showcerts | grep CN=

depth=2 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Certification Authority
verify error:num=20:unable to get local issuer certificate
verify return:0
250 HELP
0 s:/OU=Domain Control Validated/OU=PositiveSSL/CN=server.main-domain.tld
i:/C=US/ST=TX/L=Houston/O=cPanel, Inc./CN=cPanel, Inc. Certification Authority
1 s:/C=US/ST=TX/L=Houston/O=cPanel, Inc./CN=cPanel, Inc. Certification Authority
i:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
2 s:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
subject=/OU=Domain Control Validated/OU=PositiveSSL/CN=server.main-domain.tld
issuer=/C=US/ST=TX/L=Houston/O=cPanel, Inc./CN=cPanel, Inc. Certification Authority

Note that the certificate is for server.main-domain.tld instead of domain2.tld or mail.domain2.tld.

However, if we specify the SNI name with -servername:

$ openssl s_client -connect server.main-domain.tld:587 -servername mail.domain2.tld -starttls smtp -showcerts | grep CN=

depth=2 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Certification Authority
verify error:num=20:unable to get local issuer certificate
verify return:0
250 HELP
0 s:/CN=domain2.tld
i:/C=US/ST=TX/L=Houston/O=cPanel, Inc./CN=cPanel, Inc. Certification Authority
1 s:/C=US/ST=TX/L=Houston/O=cPanel, Inc./CN=cPanel, Inc. Certification Authority
i:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
2 s:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
subject=/CN=domain2.tld
issuer=/C=US/ST=TX/L=Houston/O=cPanel, Inc./CN=cPanel, Inc. Certification Authority

It appears that some tests (such as swaks) is not forcing an SNI lookup (and thus it sends out emails via SMTP just fine), but perhaps Ruby's OpenSSL library handles this a bit differently…

What should I do to set up the certificates or server properly so as to not create this issue?

Best Answer

It appears that some tests (such as swaks) is not forcing an SNI lookup ...

There is no "SNI lookup". The client either uses SNI in the TLS handshake to specify a target hostname via the server_name extension or not. But in general the use of SNI with SMTP is not really common and support is mixed. This includes mail servers which usually don't support multiple certificates on the same IP address and also mail clients where some use SNI and others don't.

It is better to not rely on SNI for SMTP, at least at the moment. This means either have one certificate covering all possible names via subject alternative names or have different IP addresses for the different certificates.

Related Topic