Hello, Habr! I present to you the author's translation of the article .
In an era when the IT world is gradually transitioning to microservices and tools like Kubernetes, one problem becomes increasingly apparent. This problem is of microservices versions. Still, the IT community believes that the current situation is significantly better than of the previous generation of technologies. Nevertheless, managing versions of microservices is a very complex issue. One piece of evidence for this can be found in articles like .
If you are reading this text and don’t yet understand the problem, allow me to explain. Suppose your product consists of 10 microservices. Now, let's assume that one new version comes out for each of these microservices. Just one version — I hope we can all agree that this is quite a trivial and insignificant fact. Now, however, let’s look at our product again. With only one new version for each component, we now have 2^10 — or 1024 permutations of how to assemble our product.
If there is still confusion, let me break down the math. So we have 10 microservices, each receiving one update. This means we have 2 possible versions for each microservice (either the old or the new one). Now, for each of the product components, we can use either of these two versions. Mathematically, this is the same as if we had a binary number of 10 digits. For example, let's say that 1 represents the new version and 0 represents the old version — then one possible permutation can be represented as 1001000000 — where the 1st and 4th components are updated, and all the others are not. From mathematics, we know that a binary number of 10 digits can have 2^10 or 1024 values. Thus, we have confirmed the scale of the number we are dealing with.
Let's continue the reasoning further — what happens if we have 100 microservices and each has 10 possible versions? The situation becomes quite unpleasant — now we have 10^100 permutations — that’s an enormous number. Nevertheless, I prefer to put it this way because now we are not hiding behind words like ‘kubernetes’, but facing the problem as it is.
Why am I so fascinated by this issue? Partly because, working in the world of NLP and AI a few years ago, we often discussed the problem of combinatorial explosion around 5-6 years back. Instead of versions, we had separate words, and instead of products, we had sentences and paragraphs. Although the issues in NLP and AI remain largely unresolved, it's important to acknowledge that substantial progress has been made over the past few years. (In my opinion, the progress could have been greater if people in the industry paid a little less attention to machine learning and a bit more to other techniques — but that's already off-topic).higher level of isolation, as if one controller is broken, the problem is confined to that specific context).Returning to the world of DevOps and microservices. We face a huge problem, disguised as an elephant in a curiosity cabinet — because what I often hear is, 'just take Kubernetes and Helm, and everything will be fine!' But no, everything will not be fine if we leave things as they are. Moreover, an analytical solution to this problem does not seem feasible due to its complexity. Just like in NLP, we should first approach this problem by narrowing the search space — in this case by excluding outdated permutations.
One of the things that could help — I wrote last year about the need to maintain minimal variance between versions released to clients.
It is also important to note that a well-designed CI/CD process significantly helps reduce variations. However, the current state of CI/CD is not sufficiently good for solving the permutation problem without additional tools for accounting and tracking components. Such an experimental system might look as follows:
Developers write tests (this is a critical stage — because otherwise, we have no evaluation criterion — this is akin to data labeling in machine learning).
Each component (project) has its own CI system — this process is well-established today, and the issue of creating a CI system for a standalone component is largely resolved.
- Developers write tests (this is a critical stage — because otherwise we don't have a criterion for evaluation — it's similar to data annotation in machine learning).
- Each component (project) gets its own CI system — this process is well-developed today, and the issue of creating a CI system for a single component has largely been resolved.
- "Smart Integration System" gathers results from various CI systems and consolidates component projects into a final product, initiates testing, and ultimately calculates the shortest path to achieving the required functionality based on existing components and risk factors. If an update is not possible, this system notifies developers about the existing components and which one is encountering an error. I want to emphasize again that the testing system here is critically important — as the integration system uses tests as a criterion for evaluation.
- The CD system then receives data from the "Smart Integration System" and directly carries out the update. This stage completes the cycle.
In summary, one of my biggest challenges right now is the lack of such a "Smart Integration System" that connects various components into a product, thus allowing tracking of how the product is assembled as a whole. I'd be interested in the community's thoughts on this (spoiler — I am currently working on a project , which could become such a smart integration system).
One last thing I want to mention is that, for me, a monolith is not acceptable for any project of at least medium size. I am very skeptical of attempts to accelerate implementation time and development quality by reverting to a monolith. Firstly, the monolith has a similar component management problem — among the various libraries it consists of, however, this is not as noticeable and primarily manifests in the time developers spend. The consequence of the monolith problem is the actual impossibility of making changes to the code — and an extremely slow speed of development.
Microservices improve the situation; however, the microservice architecture then faces the problem of combinatorial explosion at the integration stage. Yes, overall, we have moved the same problem — from the development stage to the integration stage. However, in my opinion, the microservices approach still leads to better results, and teams achieve results faster (probably mainly due to the reduced size of the development unit — or batch size). However, the transition from monoliths to microservices has not yet resulted in sufficient process improvements — the combinatorial explosion of microservices versions is a huge problem, and we have significant potential for improvement as we address this issue.
Source: habr.com
