Load Balancing a UDP server

load balancingnetworking

I have a udp server, it is a central part in my business process. in order to handle the loads I'm expecting in the production environment I'll probably need 2 or 3 instances of the server. The server is almost entirely stateless, it mostly collects data, and the layer above it knows how to handle the minimal amount of stale data that can arise from the the multiple server instances.

My question is, how can I implement load balancing between the servers? I would prefer to distribute the requests as evenly as possible between the servers. I would also would like to have some fidelity, I mean if client X was routed to server y, then I want all of X's subsequent requests to go to server Y, as long as it is sensible and not overloads Y.

By the way it is a .NET system… what would you recommend?

the state is internal within the servers, not a transaction of some sort. the state is some data the servers aggregate from the data they receive, and is quaryable with a simple WCF WebService.
The application is based on UDP, and though i dont agree with the decision, its "Above my pay grade"

I'm currently trying out MS's NLB, it works ok, it does the fidelity thing out of the box, but it generates noise on the entire network…

Also no DNS…
Oh and it's a completely costume protocol.

Best Answer

I have a udp server, [...] server is almost entirely stateless [..] have some fidelity, I mean if client X was routed to server y, then I want all of X's subsequent requests to go to server Y, as long as it is sensible and not overloads Y.

So, you're using an undisclosed application protocol that keeps some application state, and runs on top of UDP? You're kind of going in a difficult direction. UDP is not a reliable data transport, that's the whole point of it -- for reliable data transport see its popular friend TCP. The only way you can get your 'fidelity' is by having a load balancing proxy which understands your application layer protocol, and which knows what your current application state is and can act accordingly.

I see 3 approaches that come close to providing what you seek:

  • Statically spread incoming connections out over 3 IP addresses, based on source (end user) IP address. This way a given user will always be directed to the same server. Most professional firewalls can do this for you. You may have to make the 3 servers highly available yourself, as most firewalls won't do backend health checks for you.

  • Use DNS, and use DNS Round Robin, as already suggested by Matt Simmons.

  • Use Windows's built-in Network Load Balancing (NLB). Honestly, I don't know how the failover scenario would play out with NLB and your semi-stateful UDP based service -- you would have to investigate that yourself, based on how your application handles state. On the plus side, NLB is very easy to set up, free of charge with the Windows license, mature, and well performing.

Related Topic