Exchange Hybrid O365 – Fix Mail Flow Pending Issue

exchange-2010exchangeonline

Beating my head against the wall on this one.

Used the Hybrid Configuration Wizard to setup a 2010-O365 hybrid setup today, but have had mail pending for hours now trying to figure out what the hold up is.

Error for all of them are:

Reason: [{LED=450 4.7.320 Certificate validation failed [Message=SubjectMismatch] [LastAttemptedServerName=mail.<companyname>.com] [LastAttemptedIP=40.97.231.242:25] [BN3NAM04FT016.eop-NAM04.prod.protection.outlook.com]};{MSG=SubjectMismatch};{FQDN=mail.<companyname>.com};{IP=40.97.231.242};{LRT=1/27/2019 6:51:39 AM}]. OutboundProxyTargetIP: 40.97.231.242. OutboundProxyTargetHostName: mail.<companyname>.com

I have:

  • Enable-ExchangeCertificate -thumbprint <hash> -services:SMTP on the Exchange server, then restarted the transport hub (No use of get-receiveconnector due to Exchange 2010's version of this cmdlet not having a property called -TlsCertificateName; Only 2013+)
  • Tried turning off "Always use Transport Layer Security (TLS) to secure the connection (recommended)" (and the corresponding setting on the receive connector on the Exchange 2010 side)
  • Tried "Any digital certificate, including self-signed certificates" instead of "Issued by a trusted certificate authority (CA): mail.< companyname >.com"" (and the corresponding setting on the receive connector on the Exchange 2010 side)
  • Tried turning on "Enable Domain Security (mutual auth tls)"

What is and is not working in terms of mail flow is:

Mail to O365 mailbox from external - Working
Mail from O365 mailbox to external - Working
Mail to on-prem mailbox from O365 mailbox - NOT
Mail from on-prem mailbox to O365 mailbox - Working
Mail to on-prem mailbox from external - NOT
Mail from on-prem mailbox to extenal - Working

There is a GoDaddy cert that expires in 7 months for *.< companyname >.com and a Federation cert showing under Server Config in Exchange Management Console

I have run out of possible solutions via Google, so am hoping that someone in this community can help me get inbound mail flowing again today.

Best Answer

I certainly hope this helps someone else having the same issue to not have to spend such a long time fixing this silly one!

When one uses the O365 Hybrid configuration wizard AND has a wildcard 3rd party cert intending to be used for that O365 - on-prem exchange communication, make darned sure to put the subject of the cert in (including the wildcard, which it seems is treated literally, and not as a wildcard, when matching the cert subject presented by the on-prem server).

Putting the exact subject of the cert in the step of the hybrid configuration wizard pictured in 1, seems to put that setting in the last text box in picture 2 (which is reached by logging into O365 Exchange admin -> mail flow -> connectors -> Pick connector -> Next through to this screen.

Picture 1: Hybrid Config tool TLS config

Picture 2: O365 Connector TLS config