Microservices – Managing Database Access

cdatabaseentity-frameworkmicroservicesweb-api

We currently have a work stream where we are splitting out a Monolith into microservices and there has been a debate as to how we will access and persist data.

To give you a good idea of how most of our apps are being set up, we have an app built strictly for the UI, a Web API it talks to and then another Web API that, that API talks to that manage access to the database.

While it definitely seems appealing to have a Web API that we use to manages access to the database, we still have to build out endpoints specific to each app and, for the most part, the Web API our UI App talks to just acts as a proxy.

Also, there is no real design pattern set forth, do we just use our Web API as a proxy, or do we set up multiple endpoints in the data layer API and consume all of those to create our own DTOs?

This is a C# project and we typically use Entity Framework, so to me, it seems like using a Database First approach to generate our Entities in our Web API makes a lot more sense. That way we aren't managing migrations in multiple projects.

It seems too late in the game to separate our database into multiple databases, which is something I would prefer.

Best Answer

In general, the process to migrate a monolith to a microservice architecture involves carefully extracting parts from the monolith so they can live as stand-alone applications. For example, let's consider a hotel booking system. You have a rental subsystem to allocate rooms, a profile subsystem to persist data about your customers, a billing subsystem, and so on1. This is our starting point.

The monolith most likely evolved organically for a months or years, and all subsystems are not as decoupled as you would wish. From a microservice perspective, this is bad. Our first step will be to take one of those subsystems (usually, the least risky) and to re-architecture it inside the monolith so that it is completely decoupled. In our example, this could possibly be the profile subsystem.

Data access to the customer data is spread everywhere in our code. Our first task is to isolate this subsystem. We'll move the code to a separate library which will expose an interface through which the other subsystems will request data about customers. This is very important: no other subsystem can ask the database directly about customer data. They have to ask the service for this information.

Once this step is done2, the subsystem is ready to be moved to its own service to live on his own. In what's left of the monolith, now, instead of using the library to access customer data, we will call the new stand-alone profile service3. We have our first microservice4.

We accomplished two very important things:

  1. The microservice has high cohesion. Everything related to customer data is now isolated in its own microservice. All changes to the way customer data is persisted, and how customer business rules are applied are scoped to a single service;
  2. There is low coupling between the new microservice and the rest of the code. When we change how we persist customer data, the rental subsystem is completely unaffected. Which is a good thing: it shouldn't care about this, it only cares about rentals.

Back to your use case, it seems to me you are working your way backward. By building "microservices" in front of the monolith to proxy calls, you have created two important problems.

  1. You reduced the cohesion of your system. New features will most likely require changes in the proxy service to alter the API, then changes to the monolith to support this new API.
  2. You increased the coupling between the two services, because you now have to change both of them for every feature.

You're not gaining the benefits of a microservice architecture, but you still increased the complexity of the system as a whole. Your monolith is still just a big, it's still costly to run your entire test suites and involve QA for every bug fix or feature and you cannot make progress as fast as you would like.

1. In domain-driven design, these subsystems would certainly be bounded contexts.
2. Even though you could extract the subsystem to a separate service right away, it's probably a good idea to wait until its public interface has stabilized through a few features. Once it's extracted to a new microservice, it may be costly to fix mistakes due to placing boundaries incorrectly.
3. This means that even though you may actually use the same instance of a DBMS for all your services, the database of the service A should not be coupled to the database of the service B. For instance, if you had a customer table and a rental table and had a foreign key customer.id in rental, you will have to delete this foreign key and lose this referential integrity enforced by the DBMS. You'll have to ensure consistency in another way. It's the price to pay in a microservice architecture.
4. Obviously, I omitted a lot of infrastructure complexity from this answer. Doing microservices isn't a low hanging fruit: make sure it's actually appropriate to your use case!

Related Topic