Nginx – SMTP load balancing with remote host’s IP passed through to SMTP servers

haproxyload balancingnginxsmtp

I'm attempting to build a load-balanced SMTP cluster. The mail servers already exist and run Exim 4. Initially, I looked at using Nginx to do the load-balancing, however on the test system all the mail servers see the inbound connections as coming from the load-balancer IP rather than the actual remote sender IP, and after extensive Googling there doesn't appear to be any way round this. As that effectively turns the mail cluster into an open relay it's clearly a non-starter, which is a pity as Nginx works beautifully otherwise.

So I'm looking to use HAProxy instead, as I gather from further Googling that it has the ability to pass the connections with their original source IP intact so the system relay-allow lists and ACLs will operate correctly.

However having set HAProxy up as per several on-line examples, I either get "SMTP synchronisation error" (and a 500 series error so mail will bounce), and the connection immediately dropping, or just the connection dropping with no SMTP message at all.

Here is the haproxy.conf that's in use:

global

    log         127.0.0.1 local2

    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        nobody
    group       nobody
    daemon

    stats socket /var/lib/haproxy/stats

defaults
    log                     global
    option                  redispatch
    retries                 3
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout check           10s
    maxconn                 3000

listen smtp
    bind 0.0.0.0:25
    mode tcp
    no option http-server-close
    balance roundrobin
#    option smtpchk HELO smtp-in.example.com
    server smtp01 10.0.0.141:25 send-proxy check
    server smtp02 10.0.0.143:25 send-proxy check

Despite the presence of the send-proxy command, which I gather is how you tell haproxy to pass through the source IP, the Exim logs look like this:

016-12-26 07:06:48 SMTP protocol synchronization error (input sent without waiting for greeting): rejected connection from H=[10.0.0.150] input="PROXY TCP4 10.0.0.150 10.0.0.143 40334 25\r\nHELO smtp-in.example.comr\n"

In this case .150 is the load balancer and .143 is the Exim SMTP server.

Questions:

  1. Is it in fact possible to get Nginx to present the SMTP connections to the mail servers with the source IP being the real remote source IP instead of the load-balancer?

  2. Alternatively, is this possible in HAProxy, and how is it done?

In this case the current production system runs LVS, however that relies on both the load balancer and the loopback interfaces on all the mail servers sharing the same IP address. The new load balancer will be OpenSUSE 42.2, and amongst other things if that detects an IP already in use on the network it appears to helpfully removes it from itself to avoid a conflict. So LVS is out in the new build.

Other solutions to the problem which I'm considering include separating inbound and outbound SMTP traffic entirely (currently it all runs through the same load balancer), installing a simple relay (qmail possibly) on the load-balancer IP address, configured to only allow recognised ranges as per standard relay practice, and using simple DNS round-robin on the MX records to send inbound SMTP direct to the mail servers. But a load-balanced solution would be more elegant.

Best Answer

I used the custom tcp check thus

backend smtp-back
    mode tcp
    no option http-server-close
    balance roundrobin
    tcp-check expect string 220
    tcp-check send HELO\ test\r\n
    tcp-check expect string 250
    tcp-check send QUIT\r\n
    tcp-check expect string 221
    server mail01  mail01:25 check inter 60s rise 3 fall 3 backup
    server mail02  mail02:25 check inter 60s rise 3 fall 3 backup