I think I get microservices now
I used to hate the idea of splitting my monoliths into microservices. The more I read about them, the more they seemed like unnecessary complexity. The apps I worked on worked just fine as-is for the scale they operated at. So why change? As I continued working on similar apps, my feelings only cemented even further. "Just scale in other ways!" I thought, "you don't need microservices!"
And, for those apps, I'm still right. However, I was wrong to be so vehemently against them. Sometimes they are a valid solution. It only took being confronted with a problem that they might solve to change my mind.
At my current job, we have a primary/replica database cluster. The idea being that we can reduce load of the primary for reads. Those get offloaded to one of the replicas. A simplified version would look something like this:
Writes (and reads used for writes) go to the primary, and all other reads go to one of the replicas. The app communicates with the primary directly, but goes through a proxy to the replicas. The proxy will load balance between the replicas and direct traffic away from replicas that are behind by too much.
For the most part, replication lag isn't visible to users, but there are times it is. There are tweaks that we can make to the proxy to reduce perceived replication lag: lower the threshold to remove replicas and take them out quicker. However, the trade-off is that can increase load of the primary. Are there ways to solve this without microservices? Absolutely, but I can see the appeal.
If we split up parts of our monolith into microservices, we can also give them their own database clusters. Both the service and their databases can be scaled independently from one another. Put another way, high load from one area of the system won't affect another part. Maybe that would look something like this:
I only drew one database per service, but that could represent multiple configurations. Maybe there is only one database server. Or, there could be a primary/replica cluster like before. Again, they can be scaled independently of each other.
Of course, it isn't all upside: this introduces a TON of complexity and takes a lot of work. What are the right boundaries to split out into separate services? How will we change our infrastructure to accommodate this? I can't say whether it's the right decision for us, but it sounds less like the terrible idea I once thought it was.