Microservices Development – How to Ensure Services Do Not Share Data

It should depend on the way the data is stored and used.

Storage:

• Some architectures will have a separate database for accounts, and another one for user information. This makes particularly sense in a context where you have to store a lot of personal information: not just the list of e-mails and first and last names, but also phone numbers, addresses, name of the person to contact in case of emergency, pet's name and age... you name it. Polluting the authentication database with this information may not be a good idea: separating both will make it possible to easily upgrade the first without affecting the second.

• Others would combine both. If you store only sparse information about the users, it may not make sense to have a dedicated service just to serve three to four fields. Without impacting performance too much, the authentication service may include person's information in the response.

Usage:

• Sometimes, the only thing the applications would care is to know if the user is legitimate, as well as sticking an ID to the user (especially for audit purposes). For instance, if your service is hosting videos uploaded by the user, in most cases the only thing you care about is the user's ID—to know which videos belong to the user and to log that the user accessed those videos, used that much traffic, and should later pay that much for the service usage. When and if you actually need personal information (such as sending a permalink to the video to the user by e-mail), you can easily query a different web service for that.

• In other cases, it may make little sense to just authenticate a user without knowing at least her name.

Sometimes, you'll need to use a mixed approach. You will have two web services, one for authentication, and another one containing personal information, and you'll add a third service which will call the other two to provide aggregated info. Without knowing the internals of Google infrastructure, I would imagine that this is how Google's OAuth service works.

In a case of a web application showing, among other content, the names of users, there are three possible approaches we've identified through the comments and in our chat discussion:

• Send HTML template to the user, letting the browser to do one AJAX request per person to fill the info such as the pseudonym or the first and last name.

  This approach has the benefits of being straightforward in AngularJS, and allows aggressive client-side caching, and doesn't require any additional server-side programming (AJAX calls lead directly to the web service).

  However, there is a huge drawback: scalability. It may work well if you have, say, half a dozen persons on a page. On the other hand, if you expect (and according to the chat discussion, you do) dozens of persons, this means dozens of AJAX requests. Browsers do (or rather did, in 2009) a bad job at doing a lot of requests in parallel, so this could quickly become an issue. Additionally, clients may be using a crappy PC with a crappy internet connection, or may even be visiting your website from a six-years old smartphone from Himalayas.

• Send HTML template to the user, letting the browser to request in a single AJAX request the data about all the persons which figure on the page.

  The benefit is to do one request instead of a few dozens.

  The drawbacks, however, are that caching is very difficult to impossible to implement, and that you'll end up with very long and weird URIs such as http://example.com/persons/2,7,14,...,541.

• Do the query to the web service server-side, i.e. from the web application itself, and aggregate the results within the HTML.

  I would personally chose this approach. Making dozens of small calls to the web service within the same data center wouldn't matter performance-wise. Additionally, you can do them in parallel (map-reduce), asynchronously, while performing some other actions needed to render the HTML page.

  The only drawback I can see here is that the person's data won't be cached on client side (i.e. the HTML content would be considered dynamic). I can hardly imagine that being an actual bottleneck.

A few notes:

I don't know how to do that because request another service synchronously breaks microservices creating a bottleneck.

Then you call it asynchronously, and handle failure (with timeouts among others); this is actually what microservices architecture is all about—to make services call each other, while avoiding bottlenecks. Building Microservices book by Sam Newman explains those aspects in detail in chapter 11: Microservices at scale, and introduces the concept of circuit breakers which prevent one faulty or slow service from affecting the performance of all the services and applications which rely on it.

One benefit of well-designed microservices architecture is that the overall system can still work if one or several services are unavailable. For instance, if the comments service used on an e-commerce website stops working, this shouldn't prevent you from selling products, which includes showing product pages—without users' comments, obviously—as well as providing the cart and the shipment and paying features.

Note that if the called service is too slow, you may need to scale it. In order to scale it horizontally, you need to call it through a reverse proxy.

I have a Video model with comments and each comment has been made by an user. So the "Video service" return the comments with a user_id associated and I want to show the user name instead of the user_id on web views for all comments.

This is exactly the case where you can query a web service from another one. If you have a web service which handles the comments, one approach is to return just the IDs of the authors in the response; another approach is to aggregate this data with the information from person's service to have not just the IDs of the authors, but also, for instance, their first and last names, as well as the URI of their avatar.

In the case of a web application, you have to provide aggregated data. Giving simply the IDs of the authors and expecting the browser to do AJAX requests to retrieve the first and last names of every user is crazy: simply provide this information directly in order to have decent client-side performance and to avoid wasting client's bandwidth.

And it is a back practice to share a database between microservices, isn't it?

The mainstream opinion is that yes, sharing the database between multiple services is a bad idea. Although one may disagree with this opinion, in your specific case, I don't see any reason to share the database.

Microservices should be relatively independent of each other. During developer testing, you should be able to only start the one you are working on and perhaps two or three others, usually related to storage, messaging, or authentication. If you can't, you should work on improving your architecture.

If your full application won't fit properly on a developer machine, then provide a separate cluster for developer integration testing. A lot of organizations simply budget extra time on their production cloud provider, or you can build an in-house cluster using something like OpenStack.

This cluster can be over-scheduled because every developer doesn't need it 100% of the time. I estimate I do about 85% of my microservice development just with unit tests, about 10% just with the single-microservice environment on my own machine, and the remaining 5% on our shared cluster.

You probably also want to invest in better monitoring and orchestration, so you know exactly why your microservices are crashing and can have them automatically restart when they do. Perhaps something like Prometheus and Kubernetes. Kubernetes also refuses to schedule microservices in the first place if they will require too many resources than the available nodes can handle.