You Aren’t Gonna Need Microservices
Do you really need a cluster of pods, service mesh, and stream-processing platform right now?
You Aren’t Gonna Need It (YAGNI) is one of the most important principles in software development.
Perfection is achieved not when there is nothing more to add, but when there is nothing more to take away. ~ Antoine de Saint-Exupery
You can see YAGNI as a special case of the Keep It Simple Stupid (KISS) principle. YAGNI applies to a higher level as it tells us to cut off any unnecessary part while KISS advises making the rest as simple as possible.
Everything should be made as simple as possible, but no simpler. ~ Albert Einstein
Violation of YAGNI is sometimes called overengineering. A feature for every possible case, functions/methods with a lot of input parameters, multiple if-else branches, rich and detailed interfaces, all those could be a smell of overengineering. It’s the opposite of the famous UNIX-mantra: Do one thing well.
To avoid this trap, always ask yourself a question: “Is it really needed right now?” If the answer is “Well, maybe” then probably not.
Often, significant architectural decisions must be made to implement a feature. This is a problem because the earlier a decision is made the less information we have and the more likely a wrong decision is to be made. Therefore, it is a good practice to defer difficult decisions until the last possible moment. Ignoring YAGNI leads to a violation of this practice.
The main point of YAGNI is to pay the costs first when the feature is actually needed, and not right now, just in case. YAGNI violations make development always expensive because the complexity behind each feature must be paid, even though the feature is not necessary (yet). In other words, accidental complexity rises. And complexity is expensive. Complexity is no fun.
The same reasoning can be applied to microservices. But, don’t take me wrong: there are plenty of highly useful benefits that the microservices style can bring you and a lot of great principles you can apply even to your monolith. However, the actual question remains the same: Do you really need them right now? Do you really need to scale every service up to a thousand instances? Do you need a Kubernetes cluster, Kafka stream-processing platform, and Istio service mesh? Once again: maybe, but probably not right now.
Fallacies of microservices
Let's take a look at some typical fallacies I've observed so far among teams adopting microservices:
Microservices help you find boundaries
This is probably the most dangerous misunderstanding of microservices and a very expensive mistake you can make. Remember: Microservices are about technology, boundaries are about business. No technology can help you define your boundaries, only proper understanding of your business does.
Microservices are just one option for how to physically separate components once boundaries are correctly defined. At the same time, they make the boundaries hard to change if they are found to be wrong, which often happens, especially in the early phases of product development.
Try to define your service boundaries without any concerns of technology, build a team around them, implement a few product iterations. As you gain some confidence, you can turn them into microservices, if it will solve an actual problem.
Microservices make your system modular
Microservices don't make your system modular out of the box. Modularity means logical separation of business concepts. Microservices make this separation visible and difficult to cross over. While this is good, it does not come without drawbacks such as resistance to refactoring the structure of the modules.
Anyway, there are much cheaper options such as using packages/namespaces, modules/artifacts, etc. Those solutions provide a good level of physical separation along with ease to change.
Microservices make your system performant
The possibility to scale every single service independently is one of the most emphasized advantages of microservices. While this is true, it comes with costs. Running every service as a separate process can't be done without supporting infrastructure. Things like service mesh, service discovery, or load balancer come to mind first.
Cloud providers offer a working solution for you, but one has to bear in mind that remote calls are always more expensive than inter-process communication. Do you really need to scale all your services? Maybe one or two most loaded would be enough. Vertical scaling or scaling the whole system are options that usually just work and that are much cheaper than microservices.
Another aspect is good modularity. When service boundaries are incorrectly defined and multiple remote calls must be done to process a request, the overall performance is likely to decrease when turning such a system into microservices.
Microservices are cheaper to operate
A typical example of this fallacy is Serverless computing. While there definitely are use-cases where the serverless approach can be beneficial and cheap to develop and operate, the amount of such use-cases is pretty limited. What's more, it's easy to get an impression that going serverless means just writing code and taking no care of how it's being operated. This is usually not true. For example, monitoring is still an important and hard problem no one else will solve for you.
Serverless is an extreme case. An application running in the cloud in the same way as previously in your own server warehouse is likely to be much more expensive. This is very true, especially for so-called Lift&Shift solutions.
One idea behind microservices is to move a piece of complexity from development into operations, where it is easier (and cheaper) to tackle. Yes, by doing microservices right, you might save some money on development but your operation costs will probably increase rapidly. This is not necessarily a bad thing as you do save some bucks at the end of the day, it's just something to be aware of.
Microservices are easier to develop
Because a lot of complexity is moved to operations, great Ops skills are required inside the teams when taking the microservices-oriented approach. As the need for communication between Devs and Ops becomes crucible, there's no room for traditional separation by technical responsibilities. All members of the team must work together on a whole business feature, preferably colocated in the same room.
Microservices are a practical approach to Agile development and require a real DevOps mindset, where the whole team is responsible for the product from the first sketches to running the production environment.
If "DevOps" in your organization is done by that guy sitting in the last office on the second floor, then you shouldn't even think about microservices.
Microservices are small
Very popular topic and a big misconception. "Micro" does mean small or even smaller, right? Small is good, small is easy. Easy to understand, easy to develop, easy to operate… Well, no. "Micro" is not about size, it is about responsibility. Anecdotes such as two pizza teams or code that fits on my screen might work in some organizations, however, they are no general rules.
A service should be minimal in its responsibility and API. A service should implement just a single business capability. Nothing more, nothing less. The contract must be simple and clear. No unnecessary pollution, no implementation details.
The implementation can be big, even huge at the beginning. The bounded context (from Domain-driven development) defines the biggest possible service. That is the boundary to start with. Other services will emerge by themselves as the domain becomes clearer.
Summary
Always ask yourself: "Do I really need this right now? Is this the simplest thing that can possibly work?"
Don't go microservices just because they look cool and everybody is talking about them - they are no silver bullet. Doing so would be a violation of the YAGNI principle on the architecture level. That is, don't do it.
All in all, you aren't gonna need it!