Haproxy: retain existing sessions under high load, serve ‘503’ to new arrivals

haproxyPROXY

Trying to do what it says in the title: retain existing sessions under high load, and serve 503-message to newly arrived visitors.

Problem: it works, but the sessions don't last beyond about 90 seconds.

Current results have me wondering if there's a time-out setting I'm missing.

Purpose

I'm trying to get haproxy to:

  • send requests for new sessions to backend-001 when the total number of sessions on the frontend is below a certain threshold.
  • serve a 503 error to new sessions when the total number of sessions on the frontend is above that threshold
  • allow requests for existing sessions even if the number of sessions exceeds the threshold

This way, visitors that are in the middle of filling in a multi-step form won't be surprised by a 503-error, and new visitors can be told to "please come back later because we're really busy right now".

Setup

The setup is as follows:

            {visitors}
                ↓ 
            [haproxy]
                ↓ 
[rails app on unicorn served by nginx]   (right now just one 
                                            backend: 'backend-001')

current approach

To achieve the above, I'm using the configuration below.

This one is for testing, with a very low limit (10 connections on the front-end (fe_conn gt 10)), to make testing easier.

To put the server under some load, I'm using httperf as follows:

httperf –hog –server staging.machine.tld –uri /do_some_things –wsess=500,10,30 –rate 2

global
    daemon
    maxconn 10000

defaults
    mode        http
    timeout connect 6s
    timeout client  60s
    timeout server  60s
    balance roundrobin
    option http-server-close

frontend http-in
    bind [PUBLIC_IP]:80

    default_backend backend-001

    acl too_many fe_conn gt 10
    use_backend b_too_many if too_many

backend backend-001
    fullconn 10
    appsession _session_id len 128 timeout 7200s

    cookie SERVERID insert maxidle 7200s
    server Server1 127.0.10.1:80 cookie backend-001 check

backend b_too_many
    errorfile 503 /var/www/50x.html

problem

As mentioned above, the problem is: it almost works, but the sessions don't last beyond about 90 seconds.

If you keep clicking about, you get to keep your session even when there are 10 sessions busy.

Trying to open a page on the server with a different browser instance gets you the 503-error.

So, it looks like I'm almost there. Does anyone have an idea what might be causing the short session times?

And particularly how I can fix it 🙂

(edit: removed 'weight 1 maxconn 10' from the 'server'-line, not relevant and might confuse)
(edit the 2nd: corrected '10 sessions on the front-end' to '10 connections on the front-end')

Best Answer

Unfortunately, you appear to be completely confusing connections with application-level sessions. A user visiting the site might have a cookie which makes you think he owns a connection while it's not necessarily the case. He might open as many connections as needed to fetch objects and navigate pages.

The 90 seconds you're observing surely is the browser's keep alive timeout for idle connections.

It is possible to achieve what you want but it's a bit more complex than that, as you have to also consider the presence of the persistence cookie in the request to figure out whether the visitor is a new one or not.

Also in general it's more efficient to rely on the average per-server connection count than on the frontend connection count. The reason is that when a server dies, you need to readjust this number. The most efficient way to do it is to set up a server maxconn value to enable queuing, and to use avg_queue so that the limit applies to the average number of queued requests on servers. This allows you to correctly handle known visitors while softly moving new users to another backend when the load increases due to existing visitors.

Related Topic