Nginx – Haproxy balance source hash-type consistent only routing to single backend

haproxyload balancingnginx

Folks,

I have a vm setup with nginx and haproxy to act as a load balancer for a webapp that needs to maintain session stickyness to a single backend once a client has made a connection.

The nginx config is pretty standard, but posted below.

upstream haproxy {
    server 127.0.0.1:5000;
    }

server {
    listen 192.168.1.10:443 ssl;
server_name foo.bar.com;

## Compression
gzip              on;
gzip_buffers      16 8k;
gzip_comp_level   4;
gzip_http_version 1.0;
gzip_min_length   1280;
gzip_types        text/plain text/css application/x-javascript text/xml application/xml application/xml+rss application/javascript text/javascript image/x-icon image/bmp;
gzip_vary         on;

tcp_nodelay on;
tcp_nopush on;
sendfile off;

access_log  /var/log/nginx/foo.bar.com_access.log  main;
error_log  /var/log/nginx/foo.bar.com_error.log  warn;

location / {
       #proxy_read_timeout 30; #to allow for large reports
       proxy_connect_timeout   10;
       proxy_send_timeout      75;
       proxy_read_timeout      180;
       proxy_buffering off;
       #proxy_buffer_size   128k;
       #proxy_buffers   4 256k;
       #proxy_busy_buffers_size   256k;
       proxy_set_header Host $host;
       proxy_set_header X-Real-IP $remote_addr;
       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
       proxy_pass http://haproxy;
 }


    error_page 502 502 = /maintenance.html;
    location = /maintenance.html {
    root /www/;
 }
}

Here is the haproxy config

backend webapp
    balance source
        hash-type consistent 
        mode http
        server arwen.bar.com 192.168.1.50:8080 check port 8080
    server beren.bar.com 192.168.1.53:8080 check port 8080
    server boromir.bar.com 192.168.1.55:8080 check port 8080
    server isildur.bar.com 192.168.1.62:8080 check port 8080

Yet when I look at the stats, I only see a single backend server is handling all of the incoming connection, which is counter to the purpose of load balancing. I thought I had haproxy setup properly from the docs, but what am I missing?

haproxy stats page

Best Answer

First of all, if that's your entire Nginx config file, why are you bothering to proxy twice? Why not just give HAProxy the "public" IP and have it do all the work?

In any event, since you're proxying to HAProxy from NGginx, all the connections are coming from the same place (Nginx).

If you want to proxy based on the client's IP, then you've got to tell HAProxy to balance based on either the X-Real-IP or X-Forwarded-For headers that you're setting in Nginx.

Your new HAProxy config would look like this if you used X-Real-IP:

backend webapp
  balance hdr(X-Real-IP)
  hash-type consistent 
  mode http

  server arwen.bar.com 192.168.1.50:8080 check port 8080
  server beren.bar.com 192.168.1.53:8080 check port 8080
  server boromir.bar.com 192.168.1.55:8080 check port 8080
  server isildur.bar.com 192.168.1.62:8080 check port 8080
Related Topic