After some consideration I ended up seeing that Service Fabric Actors is probably the best way to start. Now, I'm wondering if I'm approaching this the right way.
The structure
Below, is the structure i came up with.
Summary
I have a Stateless Service (DeviceCalculatorManager) which is responsible for mapping the received events to the correct Actor Service (DeviceCalculatorActor). Then, the write operations are taken care of by a different actor, the DeviceWriterActor which takes care of the write operations in either SQL Database, Queues or any other more intensive write operation.
Questions
-
Assuming each actor is independent from other Actors, Is it required to perform transactions over StateManager? I don't think so because its always guaranteed 1 Actor has only 1 Thread running or am I missing something here? The Async methods don't release the lock of the Thread, is this correct?
-
I wanted to use some sort of ordered queue to store pending events to process for each stateful service. Like the DeviceCalculatorActor would receive events from DeviceCalculatorManager and store them in a queue before it begins processing it. Whats the best way to implement this? I think this is important in cases when Services want to shutdown and process the events in a later date.
Note: The DeviceCalculatorManager is a stateless service because I'm not storing any information that is relevant about the used Actors. Since Service Fabric maintains its own Actor's table for management. If you see I should approach this service differently I guess this counts as the 3rd question.
Thank you in advance.
Best Answer
There are quite a few approaches you can take depending on the problem you're trying to solve. I didn't quite understand the motivation behind having a separate Write Actor and a Calculator Actor. The important thing you need to keep in mind is that each Actor is turn-based. If you want to delegate the responsibility of saving the Actor's state to a 3rd party system that is OK but you don't really need to. You could simply write to a Stateful Service. This service can be queried directly using a 3rd party REST API etc if needed.
Please note that Actors already manage their state and keep 3 copies (configurable) already.
If you feel like you want to persist this state on a durable medium (say SQL Server, Azure Table Storage or Redis), you could backup the state at regular intervals.
It's kinda hard to tell the exact motivation behind your solution but I have an example that I implemented (using Akka) which I'm now porting over to Service Fabric because of the overhead it eliminates.
In Akka, I was using Akka Persistence on MongoDB but this comes with performance overheads of restarting, restoring state, managing snapshots etc. With Service Fabric, this overhead goes away. You simply don't have to worry about the State (within reason as long as you don't corrupt it i.e. Use cancellation tokens always).
This Shopping Cart Actor checks inventory when someone wants to buy a product (with Akka, I was checking in memory inventory persisted on mongo store).
Now in Service Fabric, my Shopping Cart Actor calls an Inventory Actor for a given product I'm purchasing to check if inventory is available.
This Inventory Actor internally calls a stateful service which maintains a reliable dictionary of inventory. I can easily update the inventory without threading issues.
At some point, I would like to get a Product's inventory summary (e.g. how many in stock, how many sold etc). To simplify this, I also update this Inventory in Redis (not sure if this is needed, but given that Redis is extremely fast, I think there might be benefits to keeping a copy of the inventory for quick reads). All writes happen using the Inventory Actor which in turn calls the Stateful Service.
The biggest benefit of having an inventory actor is that it will always be turn based. I could've stored the Inventory directly in the Inventory Actor's state, but I chose against it because I may want to eventually open up the inventory for reads across different systems.
Stateful services allow you to do that. Turn based Inventory reads wouldn't have worked very well because you are pretty much waiting for your turn just to read the inventory summary, so delegating that to a Stateful service (and Redis) helps.
Hopefully this gives you some direction. Using this Architecture in Akka I was able to scale significantly more than ever imagined. With Service Fabric, this will become even easier as I don't have to worry about Cluster Management, Actor location (simple partition keys), state management etc.
Hopefully this gives you some insights.