Azure – Internal Failover Load Balancing in Azure

There's two main parts to this,
- load balancing and failover across multiple datacenters
- load balancing and failover across multiple servers within a datacenter

There are three main ways to do multi-datacenter: BGP/"anycast", GSLB/DNS, or using an origin-failover mechanism within a CDN. none are simple, easy, or cheap.

Once the users' traffic arrives at a particular datacenter from there you want a loadbalancer component. There are tons of options here that can be broadly categorized as appliance-vs-software and layer4-vs-layer7. Judging from the details you did give I'm gonna bet your needs are pretty simple and your budget pretty small so lets just skip straight to nginx for this part. Within nginx you can configure it to serve your static content and to loadbalance your dynamic content across as many backend servers as you like running your python application.

Good luck, its a long road you've started down.

Jeff, I disagree, load balancing does not imply redundancy, it's quite the opposite in fact. The more servers you have, the more likely you'll have a failure at a given instant. That's why redundancy IS mandatory when doing load balancing, but unfortunately there are a lot of solutions which only provide load balancing without performing any health check, resulting in a less reliable service.

DNS roundrobin is excellent to increase capacity, by distributing the load across multiple points (potentially geographically distributed). But it does not provide fail-over. You must first describe what type of failure you are trying to cover. A server failure must be covered locally using a standard IP address takeover mechanism (VRRP, CARP, ...). A switch failure is covered by resilient links on the server to two switches. A WAN link failure can be covered by a multi-link setup between you and your provider, using either a routing protocol or a layer2 solution (eg: multi-link PPP). A site failure should be covered by BGP : your IP addresses are replicated over multiple sites and you announce them to the net only where they are available.

From your question, it seems that you only need to provide a server fail-over solution, which is the easiest solution since it does not involve any hardware nor contract with any ISP. You just have to setup the appropriate software on your server for that, and it's by far the cheapest and most reliable solution.

You asked "what if an haproxy machine fails ?". It's the same. All people I know who use haproxy for load balancing and high availability have two machines and run either ucarp, keepalived or heartbeat on them to ensure that one of them is always available.

Hoping this helps!