How to Implement a Process Manager in Event Sourcing

cqrsdomain-driven-designevent-sourcing

I'm working on a small example application to learn the concepts of CQRS and event sourcing. I have a Basket aggregate and a Product aggregate which should work independently.

Here's some pseudo code to show the implementation

Basket { BasketId; OrderLines; Address; }

// basket events
BasketCreated { BasketId; }
ItemAdded { BasketId; ProductId; Quantity }
AddItemSucceeded { BasketId; ProductId; Quantity }
AddItemRevoked { BasketId; ProductId; Quantity }
ItemRemoved { BasketId; ProductId; Quantity }
CheckedOut { BasketId; Address }

Product { ProductId; Name; Price; }

// product events
ProductReserved { ProductId; Quantity }
ProductReservationFailed { ProductId; Quantity }
ProductReservationCancelled { ProductId; Quantity; }

Commands are pretty similar to the events, using the imperative name and not past tense.

Right now these work just fine independently. I issue a command AddItem, and it creates a ItemAdded event on the Basket aggregate which does what it needs to do with the state of the 'Basket'. Similarly, for product the command and events work just fine.

I'd now like to combine this into a process which would go something like this (in terms of commands and events that happen):

The process manager would do the following:

on BasketCreated: CreateShoppingProcess
on ItemAdded: ReserveProduct
on ProductReserved: SucceedAddingItem // does nothing, but needs to be there so that the basket knows it can check out
on ProductReservationFailed: RevokeAddItem
on RemoveItem: CancelProductReservation
on Checkout: CreateOrder // create an order and so on...

The questions that I couldn't find definitive answers to are:

Do I need to persist the process manager? It seems like I do, but I'm not sure
If I do, I need to save the events for the process manager. However, the events that It's listening to are tied to the aggregates. Do I add the process id to those? Do I have separate events just for the process manager? How to do this and keep as DRY as possible
How do I know what basket the ProductReserved events are for? Is it OK to have a BasketId on those too, or is that leaking info?
How do I keep a relationship between events, how do I know which ItemAdded produced which ProductReserved event? Do I pass along an EventId? This seems odd…
Should I implement the Basket as a process manager instead of a simple aggregate?

After some more research I came to this:
A Saga is something that keeps its own events and listens to events from the outside. Basically, it's an Aggregate that can also react to events happening outside it's own little world.

A Process Manager works with the events from the outside and sends out commands. It's history can be rebuilt from the events that have happened on the Aggregates which share a common identifier like a correlationId.

Best Answer

Review what Rinat Abdullin wrote about evolving business process. In particular, notice his recommendation for developing a business process in a fast changing environment -- a process manager is "just" an automated replacement for a human being staring at a screen.

My own mental model of a process manager is that it is an event sourced projection that you can query for a list of pending commands.

Do I need to persist the process manager? It seems like I do, but I'm not sure

It's a read model. You can rebuild the process manager from the history of events each time you need it; or you can treat it like a snapshot that you update.

If I do, I need to save the events for the process manager.

No - the process manager is a manager. It doesn't do anything useful on its own; instead it tells aggregates to do work (ie, make changes to the book of record).

How do I know what basket the ProductReserved events are for? Is it OK to have a BasketId on those too, or is that leaking info

Sure. Note: in most "real" shopping domains, you wouldn't insist on reserving inventory before processing an order; it adds unnecessary contention to the business. It's more likely that your business would want to accept the order, then apologize in the rare case that the order can't be fulfilled in the required time.

How do I keep a relationship between events, how do I know which ItemAdded produced which ProductReserved event?

Messages have meta data - in particular, you can include a causationIdentifier (so you can identify which commands produced which events) and a correlationIdentifier, to generally track the conversation.

For instance, the process manager writes its own id as the correlationId in the command; the events produced by a copy the correlation id of the command, and your process manager subscribes to all events that have its own correlationId.

Should I implement the Basket as a process manager instead of a simple aggregate?

My recommendation is no. But Udi Dahan has a different opinion that you should review; which is that CQRS only makes sense if your aggregates are sagas -- Udi used saga in the place where process manager has become the dominant spelling.

should process managers retrieve aggregates?

Not really? Process managers are primarily concerned with orchestration, not domain state. An instance of a process will have "state", in the form of a history of all of the events that they have observed -- the correct thing to do in response to event Z depends on whether or not we have seen events X and Y. So you may need to be able to store and load a representation of that state (which could be something flat, or could be the history of observed events).

(I say "not really" because aggregate is defined vaguely enough that it's not completely wrong to claim that list of observed events is an "aggregate". The differences are more semantic than implementation -- we load process state and then decide what messages to send to the parts of the system responsible for domain state. There's a bit of hand waving going on here.)

So the PM does not need to use one type of state management over another because it is only responsible for doing stuff live and never during replays?

Not quite - state management isn't a do-er, it's a keeper tracker of-er. In circumstances where the process manger shouldn't emit any signals, you give it inert connections to the world. In other words, dispatch(command) is a no-op.

Related Solutions

CQRS – Calling External Services from Sagas/Process Manager

Short answer: call the external services from the Saga but invert the dependency by using an Interface

Unlike Aggregates, which must be pure (no dependency to anything that touches the infrastructure, nor abstract neither concrete), Sagas are domain models that may call external services. But because they are also from the Domain layer, they may not depend on the Infrastructure. You manage to do that inverting the dependency, by defining an Interface in the Domain layer with an implementation in the Infrastructure. In this way, the Domain owns the interface and not the Infrastructure.

In other words, you must use an Anti-corruption layer when communicating with external models, and that is that Interface for.

Btw, in this way you increase the Saga's testability, for free.

1-) One solution is to make these calls and depending on the service call, transition to another state by self publishing an event. Though I don't like the idea Process manager publishes events.

Me neither, only Aggregates should generate domain events.

2-) I can wrap my service calls behind another interface and that service call itself can raise the event. Though I don't like this idea since an event should be persisted before publishing.

This feels weird.

DDD: Event handlers and aggregates in functional programming

Is there a reason to have additional event handler functions which get the command data and call functions on the aggregate namespace?

Yes. Part of the motivation for a "domain model" is to have all of the code responsible for ensuring the consistency of the data in "one" place. Evans describes solutions in the context of a three tiered architecture (application, domain model, persistence), and was discouraging the anti pattern of leaking the consistency checks into the application layer.

Consistency, here, means that we don't blindly change the data as described by the command, but instead make additional changes, if necessary, to ensure that the overall consistency is maintained.

In other words, domain models are typically associated with services, in the sense described by Udi Dahan. If we weren't interested in ensuring consistency of the commands, we would remove the domain model utterly and deal with the database directly.

So a signature like

f: CommandData -> Events

typically isn't adequate, because in the general case we need to understand the current state to allow the domain model to calculate its own changes.

Let's consider a domain model of a game of tic-tac-toe. We can think of the game state as a representation of which parts of the grid are occupied by symbols, whose turn it is to play, whether the victory condition has been met.

If we get a "Play an X in the center position" command, what events do we emit? And the answer is "that depends"; we can't know what events to emit unless we already know "is it X's turn to play?", "is the center position available?" The answers to these questions depends on the state of the game, which is to say the events that have already happened. We need to know the current state of the game to map "Play an X in the center position" to the correct behavior.

Thus, we need a signature that is analogous to

g: History -> CommandData -> Events

with both the history of the aggregate and the command data being used to compute the new events.

See also: A Functional Foundation for CQRS/ES, by Mathias Verraes

Best Answer

Related Solutions

CQRS – Calling External Services from Sagas/Process Manager

DDD: Event handlers and aggregates in functional programming

Related Topic