We're implementing a service that exposes data related to a particular part of a business. It will pull data in from different sources, do some ETL, and store it in Redis. It will expose this data via ReST endpoints (and possibly GraphQL).
For scalability, Redis will be replicated from London to US. The service that resides in the US will not need to do any ETL as it'll just use the existing data that's in Redis.
Because this service has two 'modes' (read and write in London, and just read in the US), we're debating whether we should split it into two services; one that writes the data, and one that reads the data.
Option one:
Have two services, potentially in different repos, where one does the ETL and writing of data, and one that just reads that data.
The problems that we can see with this are that we'll have two lots of everything (repos, TeamCity projects, Octopus deploys etc.). We could also end up with versioning issues as they have the potential to evolve at different speeds.
Option two:
One service that can either be started as 'read-write' or 'readonly'; in London, we'll start it in 'read-write' mode, and in the US, we'll start it in 'readonly' mode.
This solves the issue of multiple repos, multiple TeamCity projects etc.
The problems we can see with this approach is the added complexity of starting it in a particular 'mode'. Also, we'll be deploying code (the ETL components) that are essentially 'switched off' in readonly mode.
We've been reading various articles online. Some say that databases should never be shared between 'Microservices' and some saying that it's absolutely fine.
Up to this point, I've used apostrophes around the word Microservices. I've done this because so far in this question, I've been using the word to describe 'technical functionality', rather than the more well defined:
The microservice approach to division is different, splitting up into
services organized around business capability
I like that description of a Microservice and I think it gives weight to my preferred option, which is option two.
But I'm no architecture expert, so I'd thought I'd ask around to see what other peoples opinions are.
What option would you choose, or would you do something entirely different?
Best Answer
The reason why it's generally preferable to separate out writes and reads is because the problems that arise from them are different, and depending on whether your application is read or write heavy you can scale both the services independently. Managing latencies during writes(which are generally higher) and connection count to your databases is much easier, a specialized service handling writes to avoid conflicts is super useful.
'Microservices' or not, the goal should be to ensure that you have enough separation between your services to allow them to scale independently and fail in isolation.
Sure, by separating out you have to keep versioning in mind, but there are so many serialising frameworks that handle versioning really really well (like Protobuf, thrift).
Separating out repos - of all the issues we developers have to deal with, the last thing should be multiple teams working on the same repo leading to a lot of conflicts.