Decoupling microservices with gRPC

apache-kafkamicroservices

I'm setting up a microservices architecture, and am confused about how gRPC can loosely-couple services (compared to a pub-sub message service like Kafka). Doesn't the request go directly to the server, and not through a pub/sub system? While gRPC supports asynchronous requests, should I still use pub/sub as a buffer between services to scale them independently?

Best Answer

gRPC and queues / brokers are not mutually exclusive. We could choose one, both or none. We could also use an SMTP server as a queue, or a plain text file.

In other words, they complement each other because they satisfy different purposes.

Decoupling

Think in a pipeline where stages are Services.

[Service A] < ----- > [Servicie B]

Arrows denote cardinality and dependency.

Both services are client and server at the same time. A is client and server of B. And vice-versa. Regardless the protocol, the IPC here involves services being aware of each other.

Now, set a queue or broker in-between

[Process A]  <----- > [Queue|Broker] <------> [Process B]

Arrows denote cardinality or dependency.

Queues and brokers break the cardinality between services, hence the dependency. They no longer need to be aware of each other. We could drop any of the services or change them so that the other remains unaware of these thingse.

Adopting features

Queues and brokers are usually designed to be performant/reliable/consistent/scalable/resilient I/O and we can provide our architecture with these features by implementing one or another. For example, gRPC is a transport mechanism for request/response and (non-persistent) streaming use cases. If we can not afford missing messages, we probably will have to look for alternatives or complementary architectural components. Queues and brokers might do the job. Persistence is not the only feature to provide our IPC with. To mention a few

  • Message consistency and persistence
  • Catching and batching
  • Error handling
  • Failover
  • Retry
  • Scalability
  • Modularity or compartmentality (having dedicated brokers or queues)
  • Compatibility with several protocols out-of-the-box
  • Traffic traceability
  • Traffic monitoring
  • Tooling

Related links

I found this blog post which dives a little more into the subject.

Worth a mention the Nginx blog post about Microservice and IPC implementations.

Related Topic