I'm trying to research the best way to load balance long running TCP connections for the following scenario:
We have multiple servers behind a redundant set of firewalls and clients establish long running (usually 10-15 hours) TCP connections to our backend servers.
Right now, "load balancing" is handled via a client side round-robin approach to go through a list of IP addresses which are all homed at our firewalls and NAT'd accordingly to the backend servers.
I'd like to get away from this approach and have only one public IP and have a separate load balancer that can check the health/load of the servers and distribute the incoming client connection requests accordingly.
One problem here is that every client, establishes 3 socket connections on 3 different ports and I'd prefer if those connections were "sticky", so all those 3 connection requests are sent to the same backend server.
I've been looking at e.g. HAProxy but I'm not really sure if it's really suited for my scenario. We have a relatively low connection count (~300 clients * 3 socket connections for each). Usually we see ~15KB/s continuous data transfer volume for each socket.
Any input on this is greatly appreciated!
Thanks,
Tom
Best Answer
Assuming that all your back-end machines are "active" and able to respond to requests all you really need is a load balancer front-end.
A good load balancer will be able to keep track of the number of connections going to each host and dynamically distribute new connections to avoid swamping one of the back-end systems (by "good" I mean "expensive", like Cisco Content Switches/Content Switch Service Modules). Price goes hand in hand with features here: Content switches are pretty high up on the solutions tier.
I've got no experience with HAProxy, but it sounds like it can do least-connection load balancing like content switches, so this would probably be a good choice (and at a much more attractive price point). I'm not sure if HAProxy can do source-tracking (send all connections from the same IP to the same back-end) though.
A few steps down down the pf firewall (or the pfsense customized distribution) can do load balancing (random or round-robin, I don't believe they can do "weighted least connections" as a balancing option like the content switches). Source tracking is implemented in pf, though you may have to play with how long that information is retained to avoid problems with connections getting moved from one server to another.
If you're already using pf/pfsense as your firewall this is a no-cost option: We use this in my current deployment with good results, but our connections aren't as long-lived as yours.