Jeff, I disagree, load balancing does not imply redundancy, it's quite the opposite in fact. The more servers you have, the more likely you'll have a failure at a given instant. That's why redundancy IS mandatory when doing load balancing, but unfortunately there are a lot of solutions which only provide load balancing without performing any health check, resulting in a less reliable service.
DNS roundrobin is excellent to increase capacity, by distributing the load across multiple points (potentially geographically distributed). But it does not provide fail-over. You must first describe what type of failure you are trying to cover. A server failure must be covered locally using a standard IP address takeover mechanism (VRRP, CARP, ...). A switch failure is covered by resilient links on the server to two switches. A WAN link failure can be covered by a multi-link setup between you and your provider, using either a routing protocol or a layer2 solution (eg: multi-link PPP). A site failure should be covered by BGP : your IP addresses are replicated over multiple sites and you announce them to the net only where they are available.
From your question, it seems that you only need to provide a server fail-over solution, which is the easiest solution since it does not involve any hardware nor contract with any ISP. You just have to setup the appropriate software on your server for that, and it's by far the cheapest and most reliable solution.
You asked "what if an haproxy machine fails ?". It's the same. All people I know who use haproxy for load balancing and high availability have two machines and run either ucarp, keepalived or heartbeat on them to ensure that one of them is always available.
Hoping this helps!
Yes you can very well hash any header of your choice. Just use "balance hdr(header_name)". I must say I had never thought about doing it that way. In some environments, it can indeed be quite useful. You can also hash on a URL parameter or a POST parameter if that can help.
You can't define your hashing method however. You can select if the hash is map-based or consistent but that's all.
Best Answer
Squid is not smart enough for this. As the squid documentation says:
IPVS and haproxy have several algorithms available like weighted round-robin, least connections, and so on. But they don't have a built in mechanism to do this by CPU load on the real servers.
On one IPVS installation that I manage, we have a cronjob that SSHes into each server and grabs the load from /proc/loadavg. Based on the load, it calculates an appropriate weight. Using the ipvsadm command, the weight for that real server is then tweaked. Works great and causes almost equal loads on each server.
Keep in mind that load isn't the only thing to look at. Average response time may be more accurate representation of user experience.