auditloggingmicroservices
What is the best practice for logging user-related data in microservices architecture?
In example: I would like to audit/persist to database information on who deleted a resource.
I have user info on the first (client-facing) service, but I don't pass it further.
I wonder if there is a smart way of doing it – like injecting some generic filter to decorate all requests with user-related headers?
It should depend on the way the data is stored and used.
Storage:
Some architectures will have a separate database for accounts, and another one for user information. This makes particularly sense in a context where you have to store a lot of personal information: not just the list of e-mails and first and last names, but also phone numbers, addresses, name of the person to contact in case of emergency, pet's name and age... you name it. Polluting the authentication database with this information may not be a good idea: separating both will make it possible to easily upgrade the first without affecting the second.
Others would combine both. If you store only sparse information about the users, it may not make sense to have a dedicated service just to serve three to four fields. Without impacting performance too much, the authentication service may include person's information in the response.
Usage:
Sometimes, the only thing the applications would care is to know if the user is legitimate, as well as sticking an ID to the user (especially for audit purposes). For instance, if your service is hosting videos uploaded by the user, in most cases the only thing you care about is the user's ID—to know which videos belong to the user and to log that the user accessed those videos, used that much traffic, and should later pay that much for the service usage. When and if you actually need personal information (such as sending a permalink to the video to the user by e-mail), you can easily query a different web service for that.
In other cases, it may make little sense to just authenticate a user without knowing at least her name.
Sometimes, you'll need to use a mixed approach. You will have two web services, one for authentication, and another one containing personal information, and you'll add a third service which will call the other two to provide aggregated info. Without knowing the internals of Google infrastructure, I would imagine that this is how Google's OAuth service works.
In a case of a web application showing, among other content, the names of users, there are three possible approaches we've identified through the comments and in our chat discussion:
Send HTML template to the user, letting the browser to do one AJAX request per person to fill the info such as the pseudonym or the first and last name.
This approach has the benefits of being straightforward in AngularJS, and allows aggressive client-side caching, and doesn't require any additional server-side programming (AJAX calls lead directly to the web service).
However, there is a huge drawback: scalability. It may work well if you have, say, half a dozen persons on a page. On the other hand, if you expect (and according to the chat discussion, you do) dozens of persons, this means dozens of AJAX requests. Browsers do (or rather did, in 2009) a bad job at doing a lot of requests in parallel, so this could quickly become an issue. Additionally, clients may be using a crappy PC with a crappy internet connection, or may even be visiting your website from a six-years old smartphone from Himalayas.
Send HTML template to the user, letting the browser to request in a single AJAX request the data about all the persons which figure on the page.
The benefit is to do one request instead of a few dozens.
The drawbacks, however, are that caching is very difficult to impossible to implement, and that you'll end up with very long and weird URIs such as http://example.com/persons/2,7,14,...,541.
Do the query to the web service server-side, i.e. from the web application itself, and aggregate the results within the HTML.
I would personally chose this approach. Making dozens of small calls to the web service within the same data center wouldn't matter performance-wise. Additionally, you can do them in parallel (map-reduce), asynchronously, while performing some other actions needed to render the HTML page.
The only drawback I can see here is that the person's data won't be cached on client side (i.e. the HTML content would be considered dynamic). I can hardly imagine that being an actual bottleneck.
A few notes:
I don't know how to do that because request another service synchronously breaks microservices creating a bottleneck.
Then you call it asynchronously, and handle failure (with timeouts among others); this is actually what microservices architecture is all about—to make services call each other, while avoiding bottlenecks. Building Microservices book by Sam Newman explains those aspects in detail in chapter 11: Microservices at scale, and introduces the concept of circuit breakers which prevent one faulty or slow service from affecting the performance of all the services and applications which rely on it.
One benefit of well-designed microservices architecture is that the overall system can still work if one or several services are unavailable. For instance, if the comments service used on an e-commerce website stops working, this shouldn't prevent you from selling products, which includes showing product pages—without users' comments, obviously—as well as providing the cart and the shipment and paying features.
Note that if the called service is too slow, you may need to scale it. In order to scale it horizontally, you need to call it through a reverse proxy.
I have a Video model with comments and each comment has been made by an user. So the "Video service" return the comments with a
user_idassociated and I want to show the user name instead of theuser_idon web views for all comments.
This is exactly the case where you can query a web service from another one. If you have a web service which handles the comments, one approach is to return just the IDs of the authors in the response; another approach is to aggregate this data with the information from person's service to have not just the IDs of the authors, but also, for instance, their first and last names, as well as the URI of their avatar.
In the case of a web application, you have to provide aggregated data. Giving simply the IDs of the authors and expecting the browser to do AJAX requests to retrieve the first and last names of every user is crazy: simply provide this information directly in order to have decent client-side performance and to avoid wasting client's bandwidth.
And it is a back practice to share a database between microservices, isn't it?
The mainstream opinion is that yes, sharing the database between multiple services is a bad idea. Although one may disagree with this opinion, in your specific case, I don't see any reason to share the database.
I typically have a service that is dedicated to serve the needs of each frontend app. This service does any backend lifting on behalf of the frontend. This has a couple of pros:
First, get list of user ID's matching the user's first name. Pass those IDs to the business service to get the list of business objects there by achieving filtering.
I recommend this approach. I would have the "frontend service" make a request to the user service to get all users by name, extract the ids and make a request to the business service to get business objects by user ids. You should implement endpoints in the services to facilitate these two requests. If you're making n requests, you've done it wrong.
Best Answer
Let me assume your microservices communicate via HTTP. We hit the same question when designing your microservice approach so let me share that:
First thing we recognized was, that there is no way around handing over some kind of user information with every cross-service request in the backend. Now, how this user information in structured is not that relevant. We kept it very simple. Since we use token authentication we just hand over this exact token from service to service. (Your problem is very similar to this one)
And yes, that approach requires every service to re-enumerate user information for this given token from some kind of
token-service. Since this happens once and only once (make sure of that when reviewing code!!) per request, we decided we'd take this road and optimize the hell out of ourtoken-service(we'll publish it as opensource soon!). But it also enables some interesting possibilities like services that can use a token of their own to impersonate a user and perform actions on it's behalf, while the service receiving this impersonation token can query the information from thetoken-servicein exactly the same way it would without impersonation!To summarize: We hand over the access token in all cross-service calls to keep the overhead in the request to an absolute minimum, adds only a constant number of additional cross-service requests!