I am planning on setting up streaming server.
Streaming is done kind of like YT does it; there will be a large number of audio content and each user can listen to it right on the website. In order to accommodate a large number of users concurrently, I will be using a small cluster of servers.
Since I will be using NGINX, it will be responsible for allocating load on each server equally (fair load balancing since each server will have identical specs). Each server is connected at 1 Gbps.
But since I am on a tight budget, I can't have too many servers so I am planning on having a dynamic throttle (each song will have a known constant bitrate). Eg. If 1 user connects, whole bandwidth is available to him. Another connects and the bandwidth is split in half and so on and so for. So that every user is given an equal amount of bandwidth. (By bandwidth, I mean speed at which user streams music to his computer eg. 300kb/s)
Throttling should continue until max threshold value is reached (1. At that point NGINX should be redirecting requests to other server or in case everything has reached its cap, return 503 error message.
Solution needs to be server based as I will be renting dedicated boxes from datacenter and won't have access to network equipment (firewalling will be done via server too). I have looked at what NGINX, squidproxy and HAPROXY offers.
NGINX and HAPROXY seems to limit only the number of requests from user, not the speed at which they download. Squidproxy seems to limit speed however it is statically defined eg. I can limit each user at 640kb/s but if there is only 1 user currently connected, the rest of pipe is left to idle.
I kept looking for solution for 2 weeks already but can't find what I am looking for. Perhaps, I don't know the proper terminology too.
Best Answer
The solution is to use Traffic shaping (specifically minimum bandwidth guarantee) and effective load balancing - hwever this is beyond rocket science - I know a lot of people who get paid a lot of money as network experts who don't understand traffic shaping properly. You're not going to learn how to do it from reading some answers here and some online tutorials.
You can afford multiple GBps internet conenctivity but you can't afford a lot of servers? I think one of us is confused about what you are trying to say here.
I suspect the big problem here is premature optimization.