HAProxy Only Sending Traffic To One Node

haproxymariadb

I have an HAProxy with IP 10.60.61.20 and three MariaDB nodes at IP .21, .22, .23, all of course in the same subnet. I have the MariaDB cluster sync'd and they can read/write to eachother, but when I setup the HAProxy, it is only sending the traffic to 10.60.61.21 (the first node). Even after I turn off mysql on the first node, the HAProxy says that there has been a connection error and it doesn't send traffic to the other two nodes. Has anyone seen this before? I just build 4 Ubuntu 12.04 servers, 3 are for the nodes and the other is the HAProxy. Here is my HAProxy .conf file:

global
    log 127.0.0.1 local0 notice
    user haproxy
    group haproxy

defaults
    log global
    retries 2
    timeout connect 3000
    timeout server 5000
    timeout client 5000

listen mysql-cluster
    bind 0.0.0.0:3306
    mode tcp
    option mysql-check user haproxy_check
    balance roundrobin
    server mariadb1 10.60.61.22:3306 check
    server mariadb2 10.60.61.21:3306 check
    server mariadb3 10.60.61.23:3306 check

Any help would be appreciated.

Best Answer

I would guess your problem is on the application side.

Since this is mode TCP, it is going to balance TCP connections among the various servers and not sql transactions. This means if your application doesn't use a pool of connections but just maintains a single open connection, that connection is going to remain open against the one server. If you application doesn't retry on failed connections, HAProxy won't magically move the failed TCP connection to another server.

I think a simple test would be to run something like the following command against the haproxy IP over and over again (however, don't run it within a single console session):

mysql -u monitor -p -e "show variables like 'server_id'"
Related Topic