HAProxy server check option causes 503

apache-2.4haproxy

First time posting so please be genial 🙂

I'm having trouble with the server check option for haproxy. I'm running HA-Proxy version 1.5.2 with two RHEL 7.2 backend web servers. There is currently no configuration on the backend servers, httpd is simply turned on and the welcome page is available from each server directly. This config works great until I add the check option to the server line. Once I do that I get 503 when targeting the vIP but its fine targeting the backend servers directly.

One thing of note, this haproxy is acting as an LB for may services and the server check option is working fine in other places. I've done a fair amount of googling but haven't been able to find exactly what the server check option is expecting on the web server side. I'm expecting the issue to be with the backend but I don't know where it could be.

The server names have changed to protect the innocent. Seems like a pretty straight forward config, any thoughts?

global
log 127.0.0.1 local2

chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon

## turn on stats unix socket
stats socket /var/lib/haproxy/stats level admin

defaults
    mode http
    log global
    option httplog
    option dontlognull
    option http-server-close
    option httpchk
    option forwardfor except 127.0.0.0/8
    option redispatch
    retries 3
    timeout http-request 10s
    timeout queue 1m
    timeout connect 10s
    timeout client 1m
    timeout server 1m
    timeout http-keep-alive 10s
    timeout check 10s
    maxconn 3000

listen stats *:1936
    stats enable
    stats uri /

frontend server-check-fail-fe
    bind xx.xx.xx.xx:80
    default_backend server-check-fail-be

backend server-check-fail-be
    balance roundrobin
    server server1 server1.fqdn.com:80 check
    server server2 server2.fqdn.com:80 check

Best Answer

In the option httpchk haproxy documentation you can find what the check do and what you can configure in detail.

Normally you can see the "health check" request in your backend web server logs.

You can also try to debug the http traffic with "tcpdump" to see what happens when no other information on logs can help.