What problems arise when we use a self-signed certificate for the SMTP protocol

smtpssl-certificate

What problems arise when we use a self-signed certificate for the SMTP protocol,
that is, when a SMTP server uses a self-signed certificate?

As long as the user accepts the exception warning due to self-signed certificate (which can be the case in the Thunderbird mail client), I would think there is no problem.

Can anyone tell me what other problems may arise?

Best Answer

On server to server communications there's no problem, because SMTP servers accepts any certificate and even falls back to unencrypted connection. In that sense, there's no public key infrastructure for MX servers. Against downgrade attacks there's DNS-Based Authentication of Named Entities (DANE) (see RFC 7672 on SMTP Security via Opportunistic DANE TLS), utilizing public keys published in DNS over public keys signed by a CA. Self-signed certificates are ok for that.

For client to server communications it's at least more convenient to have a trusted CA signed certificate. If users learn to accept an exception, it's way more likely they do so even when there's a MitM attack in place. Moreover, e.g. Android mail client only allows requiring a trusted certificate or accepting any, making the MitM attack even easier. That's why I would recommend using a CA signed certificate for client communications.

It's even possible to get a free Let's Encrypt certificate for your SMTP/IMAP server as it's not limited to use the same certificate for multiple protocols. This pretty much removes any need for self-signed certificates with publicly-referenced mail servers. (Example configuration for Postfix.)

Related Solutions

Self Signed Certificate – Active Directory – Make it trustable to all users

Yes, it is quite possible to push out certificate trusts to users. This is done through Group Policy. You can find it under Users -> Windows Settings - > Security Settings -> Public Key Policies. From there you can manage which certificates and certificate-authorities are to be trusted. The same hive exists on the Computer side of the GPO as well.

You'll probably find your AD CA in there, if you have one. These are quite useful for enterprise certificate-authorities.

MySQL Connection Issue – Can’t Connect Using Self-Signed SSL

Yes, you are correct that if you don't specify --ssl-ca then the client does not check the server certificate at all. Since it works without that option the most likely reason for the failure is that the client doesn't trust the server certificate.

If you are using self-signed client and server certificates then the ca.cert file should include both these files. That way the client will trust the server certificate and the server will trust the client certificate.

For example:
Generate the server key and certificate:

$ openssl req -x509 -newkey rsa:1024 \
         -keyout server-key-enc.pem -out server-cert.pem \
         -subj '/DC=com/DC=example/CN=server' -passout pass:qwerty

$ openssl rsa -in server-key-enc.pem -out server-key.pem \
         -passin pass:qwerty -passout pass:

Generate the client key and certificate:

$ openssl req -x509 -newkey rsa:1024 \
         -keyout client-key-enc.pem -out client-cert.pem \
         -subj '/DC=com/DC=example/CN=client' -passout pass:qwerty

$ openssl rsa -in client-key-enc.pem -out client-key.pem \
         -passin pass:qwerty -passout pass:

Combine the client and server certificates into the CA certificates file:

$ cat server-cert.pem client-cert.pem > ca.pem

Best Answer

Related Solutions

Self Signed Certificate – Active Directory – Make it trustable to all users

MySQL Connection Issue – Can’t Connect Using Self-Signed SSL

Related Topic