I'm struggling to find a solution to a specific load balancing requirement. It seems that various combinations of HAProxy, Nginx, Varnish and Apache's mod_proxy_balancer can do some but not all of what we need. My question is whether anyone can suggest a suitable configuration that would (perhaps including software that we may not have considered, but it will need to be FOSS).
The key requirement is, unfortunately, sticky sessions. We have a large, complex Tomcat app that somewhat abuses the HTTP Session, and it utilises frameworks that do the same. Converting to a stateless design or replicated session have both been investigated and discounted. Additionally, the stickiness MUST be based on a cookie and not an IP hash as we have large spikes from corporate clients where many thousands of users can all appear to be hitting the site at the same time from the same IP address.
Second most important requirement is a dynamic backend. This application runs in a cloud (not AWS) and app servers come and go from time to time based on automatic or manual scaling.
Thirdly, we need to be able to mark servers as "quiescing" so that they will continue to honour existing sessions but will not be handed new ones. When all current sessions end, the server can be turned off or upgraded/restarted.
There are some additional complications/requirements but it's meeting the three above that seems to be presenting the most difficulty.
From research so far, I think the following statements are true:
- Nginx/Varnish can't meet the sticky session requirement (based on cookies) but HAProxy and Apache can
- HAProxy can't do dynamic backend modification without a restart (might be ok if it's quick though). Varnish can with some VCL changes and a HUP, as could Nginx with some creative config file management I think. Apache can possibly handle this via a managed interface to mod_proxy_balancer..?
- I'm not sure if/how any of them can deal with the third requirement
Would really appreciate any pointers or insights.
Best Answer
With HAProxy you can add the whole range of possible backend servers and then turn them on and off (hard or soft) using the admin pages/socket calls.
How does that not fit your requirements?