Event Sourcing – Persistence Models Explained

event-programmingevent-sourcingpersistence

I am intrigued by the Event Sourcing pattern but am struggling to design an event sourcing model and put it in concrete form.

First off, I want to make sure that I understand the main use cases of this pattern. As I understand it, one uses event sourcing is to capture state changes in your app at different points in time so that you can:

  • Replay events to rewind your app to a previous state; or
  • Replay events in another environment to recreate production at some point in time and troubleshoot it

So first off, if I have misunderstood the main use case(s) for event sourcing, or if I am missing any major ones, please begin by correcting me!


Assuming I'm more or less on track, my real mental roadblock lies with the event persistence model.

  • How many databases are there in an event sourcing system? Are there two databases (one for storing main entities, and another for storing the events)? Or is there just one database where all entities are the persisted events themselves?
  • How do you capture events (and persist them!) for things that are totally outside the app's control? For example: the state of some 3rd party API that your app integrates with, or a message broker (and the state of all its messages/queues) that your app integrates with, runtime command-line arguments passed into your app, etc.?
  • How does the replay mechanism actually work? So you have some entities. You have events describing state changes to those entities. You can read those events (and/or the entities they represent) out of the database, but then you need something to actually write those events somewhere else. So does this "replayer" just treat each subsequent event as an update to an existing entity?

My best attempt (thus far):

The entity(ies):

// Groovy pseudo-code
class Order {
    Long id
    Long userId
    Long paymentMethodId    // Perhaps an ID to a 3rd party system that
                            // is PCI compliant, etc.

    // Constructors, getters/setters, etc.
}

// Each 'Order' has 1+ 'OrderLineItems'
class OrderLineItem {
    Long id
    Long orderId
    Long productId    // 'Product' is yet another entity, etc.
    Integer quantity

    // Constructors, getters/setters, etc.
}

The event:

abstract class BaseEvent<ENTITY> {
    Date occurredOn
    Date recordedOn
    ENTITY entity

    BaseEvent(ENTITY entity) {
        super()

        this.occurredOn = new Date() // now
        this.entity = entity
    }
}

class OrderEvent extends BaseEvent<Order> {
    // Perhaps some other metadata about order events

    OrderEvent(Order order) {
        super(order)
    }
}

OK so this is a good step in the right direction, but still doesn't help me figure out how the "replay mechanism" will be able to load OrderEvents out of a database, and actually "replay them" to put another database into a particular state representing a single point in time.

Best Answer

I want to make sure that I understand the main use cases of this pattern.

Another nice use case described by Fowler is distributed storage. A cluster of systems with in-memory databases are kept up to date with each other through a stream of events.

How many databases are there in an event sourcing system? Are there two databases (one for storing main entities, and another for storing the events)? Or is there just one database where all entities are the persisted events themselves?

Main entities and events are separate objects and both are persisted, either in the same or in different databases.

How do you capture events (and persist them!) for things that are totally outside the app's control?

You'll need to wrap any external systems with a gateway. The gateway must be able deal with any replay processing that the Event Sourcing system is doing. See sections 'External Updates/Queries/Interactions' of Fowler's description of this pattern.

How does the replay mechanism actually work?

Take Fowler's most basic example:

enter image description here

Each event object has a 'process' method to perform the required update. A replay works like this:

for each Shipping Event e, in ascending order of e.occurred:
    e.process();