Haproxy is currently balancing the load between 2 servers, A an B. Clients are connecting to these servers via persistent TCP connections.
Question: We want to upgrade from server A to a more powerful server C. How can we add a 3rd server C and remove A without disrupting/disconnecting the client? If iptables is used to redirect connections from A to C, will it still work if we shut down A at some point, or is A required to stay online to continue redirecting existing clients to C?
Best Answer
For the first question: how to add a third server. Changing the haproxy configuration to add C will require a restart of the load balancer. But, http://www.mgoff.in/2010/04/18/haproxy-reloading-your-config-with-minimal-service-impact/ says that a combination of -st and -sf command line arguments will minimize the disruption by allowing the haproxy instances to hand off the ports they listen to. Implementing this may require that you change your haproxy init.d script.
For the second: If you use a firewall to block A then traffic bound for A will go to B and C or just to C, depending on the haproxy configuration. This may disrupt a bit of customer traffic if the sessions have stickiness. If the application's sessions are stateless then the changes will be invisible to clients. If you shut down A after blocking its traffic then your customers will not notice.