I have developed a windows service with Message Broker (RabbitMQ)
The purpose of the service is to receive data over TCP/UDP connection. Parse the incoming data which is in HEX format and convert them to objects and save the object to the database. Also the service performs some calculations and checks to send out alerts/notifications.
The number of devices is about 3000 – 3500.
Service is developed in C#, Rabbit MQ, Entity Framework and SuperSocket.NET.
Also I have implemented caching for data reading from database.
So this is an overview of the architecture.
Part A –
1 - Our service receives the data over TCP/UDP.
2 - Parse the HEX string to objects and saves them to the database.
3 - Push the object to the Message Queue for further processing.
Part B –
1 - Receives the object from Message Queue
2 - Performs a heavy calculation and saves to database
3 - Calculate events and send out email and sms alerts/notifications (dependent on Part B --> step 2)
Now the problem is that because Part A is processing the data so fast, the Queue gets filled up with 200,000 records over 2 hours.
I have tried creating multiple queues in Part B to process the records, but still the incoming rate of messages is alot higher then the outgoing.
I need some guidance on what to do, as I have ran out of ideas on how to improve the service, as this is the 2nd time I have redone the entire service.
Best Answer
Split Part B in multiple services.
The B service does the heavy computation.
Once the heavy computation is done, push it onto another queue and have two services that write the computed data to the DB (C) and that send out alerts (D).
This way, you can run multiple instances of the heavy computation service. You could even scale the number of heavy-computation instances dynamically based upon the message queue sizes.
e.g. A B B B B C D.
And if that's still too slow, you can even run the B's on dedicated servers with more power....