Microservices Communication – Safely Distinguishing Internal Calls

It is a matter of separation of concerns. In one case, a dedicated identity provider (IDP) does the authentication and provides an access token, which is verified by all web applications before a user (or an application) can execute an operation. In the other case, every web application has to do that authentication (even if they share the same code base).

The implications are significant esp. for consuming code:

• If there is one identity provider, a user (or the consuming code) will typically authenticate once, and the access token can then be supplied to as many web applications as needed. This enables single sign on (SSO) to be implemented across multiple domains (consider the case where a user logs in once to Facebook and then don't have to supply credentials for every site that uses FB authentication.)
• Since there is one IDP, all the user experience around logins is much easier to manage. For example, a user goes to a web application, it redirects to the IDP to login. The cases such as user forgetting user name, or password, or has some pre-requisite missing on their machine can all be handled by this common service. In the case of each web application doing this, somehow the authentication user experiences have to be exposed through each web application, which is doable, but more difficult to manage.
• There're already well-established protocols that you can use for such scenarios (e.g. OAUTH), which means that you can use an external IDP (now or in the future) that supports the same protocol(s) without changing your application much.
• Since the specifications for such protocols are standardized, there is higher chance that one would get security right since important parameters are forced into the protocol (although there're still vulnerabilities that one has to manage, e.g. OAUTH is prone to CSRF attacks as an example).

The benefit of each web application doing authentication is:

• Initial implementation is usually lower cost, because one does not have to deal with creating access tokens and passing them around and validating them. Once credentials are passed to a web application, it can make an internal call to a common library (or service) to validate the credentials. Usually, web development frameworks already provide some features to accomplish these so the cost is often very low.
• If all your apps are on one domain (and you can use cookies for single sign on), it is a far cleaner user experience (but as soon as you have two or more domains, the user experience becomes difficult esp. when there is no single sign on.)

Amazon, Microsoft, Facebook, Google, etc., all use a dedicated service because the pros are far more important for their users and consuming applications than the cost of implementing. In addition, they all have dedicated teams that work on authentication features such as automated password resets, multi-factor authentication, API access, as well as implementing newer protocols and improving existing one, esp. in terms of security.

[Disclaimer: I work at Microsoft in Active Directory and Security group. Opinions are my own, and I have made my best effort to make this response technology agnostic since it is a design question.]

"The idea is that the request comes to Gateway, the Gateway looks to authorization header, it validates user, create JWT token and then sends request with info about user to another microservice internally. The microservice is reachable only internally, therefore it trust the JWT token."

This is wrong. the JWT token should be known to the client and sent with every request. All services should validate the token and not just trust it.

Your API gateway should have no logic. Its just network infrastructure