We're building two microservices that use CQRS and Event Sourcing: an Order Service and an Inventory Service. When an order is made, the inventory of that item must be decreased by the amount ordered. An order can also be canceled. In that case, the inventory must be increased again.
You may be getting yourself tripped up because you are conflating manipulation of the order with reservation of the inventory.
Hint: creating an order doesn't deplete your stock; shipping product depletes stock.
Sometimes the patterns don't fit because they are trying to tell you that the model is broken.
Send a message with only the OrderId but have the Inventory Service listen to all necessary messages of the Order Service and build its own read model of the orders.
I want to call this out explicitly as a bad idea -- the inventory service should not be trying to build its own read model of orders; that couples the inventory service to the internals of the order model, and means that you can't easily change the order model without also re-deploying the inventory service.
But... it's perfectly reasonable for the inventory service to have its own model for "reserved inventory" that it builds up because of the messages that it receives from the order service.
In other words, during cancellation the inventory service doesn't care what the order items are, it just cares what stock was reserved for that order. It can look in its own store for that....
How do I tackle the issue where an Event Handler needs more information?
One thing to keep in mind: by the time an event reaches an event handler, the event is stale.
If the Inventory Service receives an event, and needs additional information to act upon it, then it queries the authority for the information it needs. So if the inventory service needs to know the state of the order because the Order was cancelled, then it sends a query to the order service asking for the information that it needs.
Note the separation - you establish an api contract that allows the inventory service to ask for what it needs, and a schema for the messages exchanged. The two services can independently change in any way that respects that API.
So you might imagine an exchange where the inventory service gets a thin notification that an order was cancelled, and then queries the order service to get a fatter representation of the order at that time.
That query doesn't necessarily need to be synchronous -- the result of the query is just another input message to the inventory service.
Does a microservices architecture mean I will need lots of read models (aka endpoints) per client?
Write only databases aren't very interesting, so you are going to need endpoints that let you get the data back out. If you are careful about your api designs, you won't necessarily need more read models than you would in a monolith. But you will sometimes need to support more than one read model for a given capability, so that legacy and upgraded clients can both access the information that the need.
Deciding if order is important is up to each microservice. In your case, it definitely is. Your example above is actually easy to solve since if the OrderService gets an UpdateProduct event when the product doesn’t exist, it can put the UpdateProduct event back on the queue assuming there’s a Create event coming soon. However, two UpdateProduct events arriving out of order could cause problems so we still have to solve the issue.
There are ways of ensuring order in the publisher and subscriber but they are clumsy. If your Update events contain the entire new state of the product, you can add a timestamp and refuse to update your cache if the incoming timestamp is older than the persisted one. If your Update events only send updated data, you can put an incrementing version number on your aggregate and if an incoming event is more than 1 greater than your existing version, requeue the event assuming the intervening event is coming soon. As you can see, these are clumsy. You’re much better off ensuring order in the queue.
You may consider Azure Event Hub instead of Azure Event Grid. The Event Hub will guarantee order in a single partition, so wisely partitioning your app can give you the order guarantee you need. The Event Hub acts more like Kafka than a traditional Queue in that it provides a queue plus persistence of events for a small number of days. This can be advantageous if a system goes down and needs to recover.
Best Answer
In my experience, most questions about transactionability and microservices are caused by the following two reasons:
The transactional data is placed in different microservices: This is wrong by definition. Data that should be modified transactionaly belongs to the same service. Grouping the right data in the right service is very tricky and you need to put a lot of thought into it. If you just create microservices based on entities (Orders, Users, Inventory, Products, etc), then you will find the same problem over and over, plus you'll end up with N services depending on N other services to perform their purpose. Therefore: put all data that changes transactionally together.
The operation is not transactional in the first place: You are trying to solve with technology something that should be solved by business rules. This is your scenario in my opinion. Orders and Inventory are not transactional. You obviously don't want people buying products out of stock, but you have to accept that there will be always a slight chance that this happens (what if your inventory numbers are correct, but someone at the warehouse drops a box and breaks the last item?). Businesses have been dealing with these sort of problems far longer than transactions existed. Therefore, talk to business and ask what you should do when an order comes in and you are out of stock. If proper stock management is in place, you'll probably know already when new items will be available, so you can email the customer with the new delivery date and offer a refund if she doesn't accept (as an example).
As a side note, your sentence "an order microservice which then emits events to deduct inventory from the inventory microservice" sounds wrong. It might be just the sentence wording, but just in case, I want to point out that the wording suggests that what you do is emitting a Command, not an Event. A command is imperative, you tell someone to do something (DeductInventoryCommand). An Event is something that has already happened (OrderPlacedEvent).
Ideally, you don't want one microservice telling another one what to do, as this effectively breaks the autonomy of the service that receives the command. Instead, develop your services to be able to perform their business capability based on the information they "hear" from other microservices, but autonomously. A Logistics service can listen to the placed orders (and cancelled orders) and take care of all the logistics process without being told what to do. Billing should be able to do its job without being told what to do either, and so on.