The application we have in mind is a relay server. The aim is to accept large numbers of incoming socket connections (at least thousands) that will stay open for lengthy periods (hours or maybe days). They will exchange modest quantities of data and they need reasonably low latency.
The technical design is straightfoward and we have a test implementation. Our preference is to use Windows hosting and .NET, because that's the technology we know. However, this kind of usage is well outside what we are familiar with.
The question is whether there are specific limits or constraints to be aware of that are inherent in or common to software that does this, and that we should allow for in our design and/or test for before a roll-out.
I found this question (Handling large amounts of sockets) and this link (http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-1), which tend to suggest that our solution should work.
—
Commenters have suggested opening and closing ports, or using some kind of protocol, without suggesting how. The problem is that at any time a message might be relayed to an individual known destination with the expectation that it be received quickly (say 1 second at most, preferably sooner). The destination is (in general) behind a firewall with unknown properties, so cannot (in general) run a server or accept incoming connections. It can only make outgoing connections. We think it needs a persistent outgoing connection in order to receive packets at any time without notice. Alternative suggestions would be of interest, although strictly off topic for this question.
Other commenters have suggested there are OS limits, but not specified any. Assuming that this is some version of Windows server and some (perhaps large) amount of memory, are the limits likely to be a problem? Roughly what will they be? Is Windows a bad idea?
Best Answer
I worked on a relay server for stock market data in C# on a Windows Server. There was no way I could get thousands of simultaneous connections relayed by one machine. The specs for the relay were very simple 1 connection to stock market data provider, and unlimited outbound connects to SilverLight clients.
There are two basic approaches that I investigated.
Neither approach could exceed the performance limits of the CPU, and each approach had serious limitations and restrictions.
Using a thread pool.
Windows sucks at multi-thread handling. Once I hit around 250 threads things just started to go down hill. It isn't a memory or system resource problem. It's a quantity problem. While Windows has no problem managing 250 threads. It's another story asking Windows to keep those 250 threads busy relaying data. As performance lags a data backlog starts to happen.
Using a worker thread.
You can't use a worker thread to iterate sockets if those sockets are blocking. Each time the thread hits a socket that has to timeout all other sockets are left waiting. If you switch to asycn socket operations, then a huge backlog of callbacks are generated to quickly and everything breaks.
For me the results were.
100 clients everything is stable. 250 clients everything is working but limit reached. 1000 clients never achieved.
Conclusion.
C# on Windows is not the right tool for a socket relay server. Not for client connections ranging in the thousands.
The only alternative is to not use HTTP sockets, and switch to a broadcasting protocol like UDP or TCP. For me this was not an option as no data was allowed to be dropped. Most broadcasting protocols assume packet loss is acceptable.
Finally.
If you are able to create a C# relay that can handle thousands of clients. Please come back and let me know how you did it.