C# – Multiple API calls potentially spawning multiple Websockets – Threading vs Async

apiasynccmultithreadingwebsockets

I am designing a program with the flow as outlined below. Note, these are all network calls, there is no system I/O (hard drive).

Initially, multiple independent API calls need to happen – they don't rely on the others at all. The response from these will be processed and if they meet certain criteria, they should spawn a websocket connection and stop their API calling until the socket is closed. For example, if there are 4 API calls happening and one meets the requirements, it stops and connects to a websocket. Now there should be 3 API calls happening as well as a websocket receiving data. There should never be more websockets than the number of initial API calls.

Now, the websocket(s) are also processing the recieved data looking for specific requirements. If any one of the sockets meets the requirement, a SECOND websocket is then connected to. At this point, all outstanding API calls/websockets should close/stop their processing. Only this new websocket should be running looking for its final set of requirements. Once this final set is met, the whole process should restart (meaning the API calls start up again).

Let me walk you through a scenario: 4 API calls start. After 3 seconds one meets the requirements. There are now 3 API calls STILL happening, and 1 websocket running. After 2 seconds, the websocket fails its requirements. There are now 4 API calls happening again. Soon, 2 API calls both meet requirements. Now there are 2 API calls and 2 websockets listening. After another 3 seconds, one of the websockets meets its requirements. All websockets and API calls should stop, and a final websocket is opened. After this final one either succeeds or fails, the initial 4 API calls should be restarted and the process repeats all over again.

My question: Would this benefit from some form of threading (instead of only using async) and if so, what method would you use? Each API call on it's own thread that spawns its following websockets on it too? Maybe all the API calls on single thread but then spawn off websockets on their own thread? Maybe the whole program can be run in a single thread with good performance? The operations are time sensitive so the faster I can determine requirements being met the better! Thanks in advance!

Best Answer

If the underlying API does querying to a relational database, then partitioned data parallelism may be useful:

Nodes used based on degree of parallelism needed by query

For BI and analytics queries for which larger amounts of data are likely to be processed, the query execution architecture should be able to parallelize at multiple levels. The first level is partitioned parallelism, so that multiple processes for an operation such as join or aggregation are executed in parallel.

The API can be setup in a number of ways. For example:

For each executable SQL statement in a context, the first run-time services call always tries to obtain a latch. If it is successful, it continues processing. If not (because an SQL statement in another thread of the same context already has the latch), the call is blocked on a signaling semaphore until that semaphore is posted, at which point the call gets the latch and continues processing. The latch is held until the SQL statement has completed processing, at which time it is released by the last run-time services call that was generated for that particular SQL statement. The net result is that each SQL statement within a context is executed as an atomic unit, even though other threads may also be trying to execute SQL statements at the same time. This action ensures that internal data structures are not altered by different threads at the same time. APIs also use the latch used by run-time services; therefore, APIs have the same restrictions as run-time services routines within each context. Contexts may be exchanged between threads in a process, but not exchanged between processes. One use of multiple contexts is to provide support for concurrent transactions.

References

Related Topic