The best thing to do is to ensure that all requests are from an authenticated source. This could mean a user request, but also a 'service' user that corresponds to each microservice. If a hacker gains access to your tier2, and you have an un-authenticated API, they can wreak havoc.
Each microservice should have its own distinct user for use for unsolicited requests, but generally every API call should be passing along the user token you received from the original request.
No, Service Fabric doesn't provide any messaging mechanism to communicate with reliable services. In practice, reliable services (stateful and stateless) are basically processes running doing nothing until you tell them to do something. You are responsible to create the mechanism to allow a service from receiving requests to do work (with remoting, REST, messaging...). Service Fabric only provides you the basis of how to implement this mechanism which is the ICommunicationListener.
If you want asynchronous communication between services, you could use, for example, Azure Service Bus and use a ready made communication listener.
Now, using messaging doesn't fully solve your problem of having long running processes, as starting a long running process can be done synchronously as well (you could call your service with REST for example, and the service could return immediately with an OK and start the long running process so that the client is not synchronously waiting). The main problem with long running processes is normally how to communicate back to the client that the process has completed. Note that being stateless services, you cannot poll the service to get the status of the process as the request could go to a different instance of the service (and it would go against the design rules of stateless services anyway).
If your scenario is two microservices talking to each other, you have several options to communicate back the results. If you are ok introducing a circular dependency, then you could simply do a REST call from A to B to start the process and a REST call from B to A to communicate the completion.
If you use messaging, you could create a mechanism to allow Replies, without introducing a circular dependency, for example, by sending the Reply Queue as part of the original message (or message properties in ASB). This way your service B doesn't have to know anything about the caller.
I hope it makes sense.
Best Answer
Let's start with the fundamental question:
Short answer: yes, but...
I just want to clarify a couple concepts. An API gateway is there to redirect requests to one of the potentially multiple instances of a microservice you have running. The gateway handles the load balancing, etc.
If you want direct connect between microservices, you still need the loadbalancing. It all depends on your infrastructure how you handle it. For example, the way you accomplish that with Kubernetes, Spring Cloud infrastructure, or Consul will be slightly different.
The reality is that any one of your microservices can go into a partial failure, where it ceases to respond to new requests and is essentially "stuck". It can be for many reasons like memory pressure and garbage collection gone into hyperdrive, an endless loop that didn't get caught in testing, etc.
The bottom line is:
The sooner you can send a meaningful response to your caller, the more responsive your application will feel. There may be further processing that needs to be done, but the user doesn't need to sit and wait for everything to be completed as long as they can trust that it will be completed.
As a general rule of thumb: