Microservices Architecture – Understanding Canonical Schema

Apologies in advance for relying on the @ArseniMourzenko comment, but once I started to read the Wikipedia I immediately understood what Canonical Schema means.

Here OP's comment that focuses on the real doubt

I believe even in microservice architecture, the request and response have to comply to some data model.

Some data model yes, but seems that the article is referring to a "shared" or "common" data models between 2 or more services.

The Canonical Schema is a pattern meant to save services from in runtime data transformations. It also saves you from duplicating code. But you are then coupling your service to an external data model too.(See diagrams at the Wikipedia's page linked above)

It's a sort of common "language" between services.

So looks like the article is emphasising on the total independence of the MS from the "ecosystem" where it lives at.

Take for example the mention it does to ESB.

They also very much avoid using ESBs and instead implement ESB-like functionality in the microservices themselves.

ESB usually demands an enterprise data model (messages) which is going to be common for everyone attached to the bus.

So, back to the article, seems that the author is pointing to the fact that MS rejects to be attached to any external system (and their constraints).

After doing a bit more research, I stumbled upon this article from which I've pulled some quotes out that I think is helpful for what I want to accomplish (and for any future readers). This offers a way to adopt a reactive programming model over an imperative programming model.

Event-sourcing

The idea here is to represent every application’s state transition in a form of an immutable event. Events are then stored in a log or journal form as they occur (also referred to as ‘event store’). They can also be queried and stored indefinitely, aiming to represent how the application’s state, as a whole, evolved over time.

What this helps accomplish is that if a microservice goes down yet other events pertinent to it are being published and events are consumed by, say, other instances of that microservice, when that microservice comes back up, it can refer to this event store to retrieve all the events that it missed during the period it went down.

Apache Kafka as Event Broker

Consider the use of Apache Kafka which can store and dispatch thousands of events per second and has built-in replication and fault-tolerance mechanisms. It has a persistent store of events which can be stored on disk indefinitely and consumed at any time (but not removed) from the Topic (Kafka's fancy queue) were delivered to.

The events are then assigned offsets that univocally identify them within the Topic — Kafka can manage the offsets itself, easily providing “at most once” or “at least once” delivery semantics, but they can also be negotiated when an event consumer joins a Topic, allowing microservices to start consuming events from any arbitrary place in time — usually from where the consumer left off. If the last consumed event offset is transactionally persisted in the services’s local storage when the usecases ‘successfully complete’, that offset can easily be used to achieve an “exactly once” event delivery semantics.

In fact, when consumers identify themselves to Kafka, Kafka will record which messages were delivered to which consumer so that it doesn't serve it up again.

Sagas

For more complex usecases where the communication among different services is indeed necessary, the responsibility of finishing the usecase must be well recognized — the usecase is decentralized and only finishes when all the services involved acknowledge their task as successfully completed, otherwise the whole usecase must fail and corrective measures must be triggered to rollback any invalid local state.

This is when saga comes into play. A saga is a sequence of local transactions. Each local transaction updates the database and publishes a message or event to trigger the next local transaction in the saga. If a local transaction fails because it violates a business rule then the saga executes a series of compensating transactions that undo the changes that were made by the preceding local transactions. Read this for more info.