Sendmail messages rejected from Microsoft when using TLS

opensslsendmail

I'm going to preface this with a statement that I am no expert in sendmail. I rarely use it, but in this situation I have to for my client.

Background:

A client of mine has a mail server running Debian 7.7, Sendmail 8.14.4, and OpenSSL 1.0.1e. It generates emails (stuff like account reset) and sends them to customers. I'll call it "Server A".

Messages sent from Server A to any mail server at the domain *.outlook.com (which strangely doesn't include @outlook.com messages, those are handled by hotmail) were being deferred due to a failed TLS handshake. This is only a problem for servers owned by Microsoft; everything else is working. The error in the log looked something like:

sendmail[22213]: ruleset=tls_server, arg1=SOFTWARE, relay=mail.protection.outlook.com, reject=403 4.7.0 TLS handshake

sendmail[22213]: s2L9EakQ022098: to=<blah@microsoft.com>, delay=00:00:55, xdelay=00:00:31, mailer=esmtp, pri=181653, relay=mail.protection.outlook.com. [145.123.225.25], dsn=4.0.0, stat=Deferred

Note: These logs are made up because my client doesn't want things copied and pasted less they accidentally leak sensitive data. The information is there, I just typed it by hand so ignore typos, etc.

That was all I could get from the logs, even with the logs on 15 (Beyond that, the system started to become disk bound).

Problem:

A tcpdump of the handshake showed that Server A was connecting to the outlook.com, EHLO went sucessfully, Server A initiated STARTTLS, outlook.com then responded with its certificate chain. All of this went normally.

Then after Server A received the microsoft certificate chain, Server A sent its own client certificate. Outlook.com then dropped the connection.

Our client certificates are self-signed certificates that we use for intraorganizational verification and shouldn't go out to outlook.com.

From my limited knowledge, my best guess is that outlook.com is requesting the client certificate for verification (or at least sendmail thinks its requesting the certificate), which sendmail on Server A is obliging. Outlook.com is then dropping the connection, either because the certificate is invalid or it is not what it was expecting.

Two questions:

  1. Why is sendmail sending the client certificate to outlook.com as part of STARTTLS?

  2. Is there a way that I can prevent sendmail from sending client certificates to servers outside our domain, even if the server requests it? From the experiments I've done, outlook.com will accept the message if I simply ignore the certificate request and send the message.

Before anyone suggests it, I have already created a access map that prevents TLS from connections with some IP addresses associated with Microsoft. This isn't a good permanent solution because I would prefer using TLS, and the list of IP address has to be manually maintained, which is a bother.

Best Answer

Welcome to Serverfault! :-)

Have you explicitly configured SendMail to use TLS? If not, out of the box, SendMail will still try to perform opportunistic TLS operations with a zero-byte client certificate. I assume this is what you're seeing? For more information on this (oddity?) check out @MadHatter's excellent response here: https://serverfault.com/a/514180/21875

So, with that said, I'll respond to your questions in the order they were asked:

  1. Why is sendmail sending the client certificate to outlook.com as part of STARTTLS?

When two MTAs are attempting to establish a secure tunnel for STARTTLS, it's normal for them to negotiate security information and exchange public certificates so they can securely encrypt information to one another. If sendmail didn't provide a client certificate to outlook.com, the outlook.com mail servers wouldn't have a way to encrypt responses/acknowledgements back to sendmail.

  1. Is there a way that I can prevent sendmail from sending client certificates to servers outside our domain, even if the server requests it? From the experiments I've done, outlook.com will accept the message if I simply ignore the certificate request and send the message.

You have a few options:

Option #1: Implement a per-server / per-domain exception rule. This can be done by adding Try_TLS:<broken.server> NO to your access table. It sounds like you're already doing this but it's important to note that the "broken.server" part can be an IP, MX record (microsoft-com.mail.protection.outlook.com), or partial MX record (outlook.com).

Option #2: Implement a bypass rule for all domains. This can be done by adding Try_TLS: NO to your access table (no need to specify IPs).

Technically, you could also hack the sendmail.mc file to prevent sendmail from checking for TLS capability at all. I wouldn't recommend doing this though since it requires delicate changes.

Related Topic