🥇Service mesh use cases

Note. transl.: The author of this article (Luc Perkins) is a developer advocate at CNCF, home to such Open Source projects as Linkerd, SMI (Service Mesh Interface) and Kuma (by the way, have you also wondered why Istio is not on this list?. .). Once again trying to bring a better understanding of the trendy hype called “service mesh” to the DevOps community, he lists 16 characteristic features that such solutions provide.

Today service mesh is one of the hottest topics in software engineering (and rightfully so!). I see this technology as incredibly promising and dream of seeing it widely adopted (of course, when it makes sense). However, she is still surrounded by a halo of mystery for most people. However, even those who well familiar with her, they often find it difficult to formulate its advantages and what exactly it is (including your obedient servant). In the article I will try to remedy the situation by listing various use cases "service grids" *.

* Approx. transl.: hereinafter in the article such a translation (“service mesh”) will be used for the still new term service mesh.

But first I want to make a few remarks:

I have never worked with service meshes and have not used them outside of my educational projects. On the other hand, I was the one who wrote a bunch of documentation for Twitter's internal service mesh in 2015 (then it wasn't even called a "service mesh" yet) and was involved in the development of the site and documentation for linkerdso it means something.
My list is indicative and incomplete. Use cases that I don't know are quite possible, and new ones are sure to emerge over time as the technology advances and its popularity grows.
At the same time, not every existing service mesh implementation supports all of the above use cases. Therefore, my expressions like "service mesh can ..." should be read as "individual, and possibly all popular service mesh implementations can ...".
The order of the examples does not matter.

Short list:

service discovery;
encryption;
authentication and authorization;
load balancing;
circuit breaking;
autoscaling;
canary deployments;
blue-green deployments;
health check;
load shedding;
traffic mirroring;
insulation;
request rate limiting, retries and timeouts;
telemetry;
Audit;
visualization.

1. Service discovery

TL;DR: Connect to other services on the network using simple names.

Services should be able to automatically "find" each other using appropriate names - for example, service.api.production, pets/staging or cassandra. Cloud environments are resilient, and a single name can hide many instances of a service. It is clear that in such a situation it is physically impossible to hardcode all IP addresses.

Plus, when one service finds another, it should be able to send requests to that service without fear that they will end up at the input of its idle instance. In other words, the service mesh should keep track of the health of all service instances and keep the list of hosts as up to date as possible.

Each service mesh implements a service discovery mechanism in its own way. At the moment, the most common way is to delegate to external processes like DNS Kubernetes. In the past, on Twitter, we used a naming system for this purpose. Finagle. In addition, the service mesh technology makes it possible for custom naming mechanisms to appear (although I have not yet seen any SM implementation with such functionality).

2. Encryption

TL;DR: Get rid of unencrypted traffic between services and let this process be automated and scalable.

It's nice to know that attackers can't get into your internal network. Firewalls are great for this. But what happens if a hacker does get in? Will he be able to do whatever he wants with intraservice traffic? Let's hope this doesn't happen though. In order to prevent such a scenario, a zero-trust network should be implemented in which all traffic between services is encrypted. Most modern service meshes achieve this through mutual TLS (mutual TLS, mTLS). In some cases, mTLS works in entire clouds and clusters (I think interplanetary communications will someday be similar).

Of course, for mTLS service mesh optional. Each service can take care of its own TLS, but this means that it will need to find a way to generate certificates, distribute them to service hosts, include code in the application that will load these certificates from files. Yes, don't forget to renew these certificates at regular intervals. Service meshes automate mTLS with systems like SPIFFE, which, in turn, automate the process of issuing and rotating certificates.

3. Authentication and authorization

TL;DR: Establish who the requester is and determine what they are allowed to do before the request reaches the service.

Services often want to know who performs a request (authentication), and, using this information, decide whether that the given subject is allowed to do (authorization). In this case, behind the pronoun "who" can hide:

Other services. This is called "authentication peer". For example, service web wants to access the service db. Service meshes usually solve these problems using mTLS: certificates in this case act as a necessary identifier.
Some human users. This is called "authentication request". For example, user haxor69 wants to buy a new lamp. Service meshes provide various mechanisms, for example, JSON Web Tokens.
Many of us have done this in application code. A request comes in, we scan the table users, find the user and compare the password, then check the column permissions etc. In the case of a service mesh, this happens before the request even reaches the service.

After we have established who the request came from, we need to determine what this subject is allowed to do. Some service meshes allow you to set basic policies (about who can do what) as YAML files or on the command line, while others offer integration with frameworks like Open Policy Agent. The ultimate goal is to ensure that your services accept any requests, safely assuming they come from a trusted source. и the action is allowed.

4. Load balancing

TL;DR: Distribute the load across service instances in a specific pattern.

A "service" within a service sect very often consists of many identical instances. For example, today the service cache consists of 5 copies, and tomorrow their number may increase to 11. Requests sent to cache, should be distributed according to a specific purpose. For example, minimizing latency or maximizing the chance of hitting a healthy instance. The most commonly used algorithm is round-robin, but there are many others, such as the weighted method. (weighted) requests (you can select preferred targets), ring (ring) hashing (using consistent hashing for upstream hosts) or least requests method (preference is given to the instance with the fewest requests).

Classic balancers have other features such as HTTP caching and DDoS protection, but they are not very relevant for east-west traffic. mesh). Of course, it is not necessary to use a service mesh for load balancing, but it allows you to set and control load balancing policies for each service from a centralized control layer, thereby eliminating the need to run and configure individual load balancers in the network stack.

5. Circuit breaking

TL;DR: Stop traffic to a problematic service and control damage under worst-case scenarios.

If for some reason the service is unable to handle the traffic, the service mesh provides several solutions to this problem (others will be discussed in the relevant sections). Circuit breaking is the toughest option for disconnecting a service from traffic. However, it does not make sense on its own - a back-up plan is needed. Back pressure can be provided (back pressure) on services making requests (don't forget to configure your service mesh for this!), or, for example, coloring the status page red and redirecting users to another version of the “falling whale” page (“Twitter is down”).

Service grids allow not only to determine when followed by shutdown and that this will follow. In this case, "when" can include any combination of given parameters: total number of requests for a certain period, number of concurrent connections, pending requests, active retries, etc.

You don't want to overuse circuit breaking, but it's nice to know there's a back-up plan in case of emergency.

6. Autoscale

TL;DR: Increase or decrease the number of service instances depending on the given criteria.

Service meshes are not schedulers, so they are not carry out scaling yourself. However, they can provide information on which planners will base their decisions. Because service meshes have access to all traffic between services, they have a wealth of information about what's going on: which services are having problems, which are underutilized (power allocated to them is wasted), and so on.

For example, Kubernetes scales services based on pods' CPU and memory usage. (see our report "Autoscaling and resource management in Kubernetes"- approx. transl.), but if you choose to scale based on any other metric (in our case, traffic related), you'll need a custom metric. Management like this shows how to do it with Envoy, Istio и Prometheusbut the process itself is rather complicated. We would like the service mesh to simplify it, allowing you to simply set conditions like “increase the number of service instances authif the number of pending requests exceeds the threshold within a minute."

7. Canary Deployments

TL;DR: Test new features or service versions on a subset of users.

Let's say you're developing some SaaS product and intend to roll out a cool new version of it. You tested it in staging and it worked great. But still overcome certain concerns about its behavior in real conditions. In other words, it is required to test the new version on real tasks without risking the trust of users. Canary deployments are great for this. They allow you to demonstrate a new feature to a subset of users. This subset may consist of the most loyal users or those who work with the free version of the product, or users who have expressed a desire to be "guinea pigs".

Service meshes implement this by allowing you to specify criteria for who sees which version of the application, and routing traffic accordingly. At the same time, nothing changes for the services themselves. Version 1.0 of the service believes that all requests come from users who should see it, and version 1.1 believes the same about its users. And you, meanwhile, can change the percentage of traffic between the old and the new version, redirecting a growing number of users to the new one, if it works stably and your "guinea pigs" give the go-ahead.

8. Blue-Green Deployments

TL;DR: Roll out a cool new feature, but be prepared to bring it all back immediately.

Meaning blue-green deployments is to roll out a new “blue” service, running it in parallel with the old, “green” one. If everything goes smoothly and the new service performs well, then the old one can be phased out. (Alas, someday this new “blue” service will repeat the fate of the “green” and disappear ...) Blue-green deployments differ from canary ones in that the new function covers all at once users (and not a part); the point here is to have a "safe haven" ready in case something goes wrong.

Service meshes offer a very convenient way to test a "blue" service and instantly switch to a working "green" in case of problems. Not to mention the fact that along the way they give a lot of information (see "Telemetry" below) about the work of the "blue", which helps to understand whether it is ready for full operation.

Note. transl.: You can read more about different deployment strategies in Kubernetes (including the mentioned canary, blue / green and others) in this article.

9. Health check

TL;DR: Keep track of which service instances are up and respond to those that are not.

Health check (health check) helps to decide whether service instances are ready to receive and process traffic. For example, in the case of HTTP services, a health check might look like a GET request to an endpoint /health... Answer 200 OK will mean that the instance is healthy, any other - that it is not ready to receive traffic. Service meshes allow you to specify both the way in which the health check will be performed, and the frequency with which this check will be carried out. This information can then be used for other purposes, such as load balancing and circuit breaking.

Thus, the health check is not a stand-alone use case, but is usually used to achieve other goals. Also, depending on the results of health checks, external (in relation to other service mesh targets) actions may be required: for example, updating a status page, creating an issue on GitHub, or filling out a JIRA ticket. And the service mesh offers a convenient mechanism for automating all of this.

10. Load shedding

TL;DR: Redirect traffic in response to a temporary spike in usage.

If a certain service is overloaded with traffic, you can temporarily redirect some of this traffic to another place (i.e. "dump", "transfer" (shed) him there). For example, to a backup service or data center, or to a permanent Pulsar topic. As a result, the service will continue to process some of the requests instead of crashing and stopping processing everything at all. Load shedding is preferable to circuit breaking, but it is still undesirable to abuse it. It allows you to prevent cascading failures that result in downstream services falling.

11. Traffic parallelization/mirroring

TL;DR: Send a single request to multiple locations at once.

Sometimes it becomes necessary to send a request (or some selection of requests) to several services at once. A typical example is sending part of the production traffic to the staging service. Production's main web server sends a request to a downstream service products.production and only to him. And the service mesh intelligently copies this request and sends it to products.staging, which the web server is not even aware of.

Another related service mesh use case that can be implemented on top of traffic parallelization is regression testing. It involves sending the same requests to different versions of the service and checking if all versions behave the same way. I have not yet seen a service mesh implementation with an integrated regression testing system like Diffybut the idea itself seems promising.

12. Insulation

TL;DR: Break your service mesh into mini-networks.

Also known as segmentation, isolation is the art of dividing the service mesh into logically separate segments that know nothing about each other. Isolation is a bit like creating virtual private networks. The fundamental difference is that you can still enjoy all the benefits of a service mesh (like service discovery), but with added security. For example, if an attacker manages to infiltrate a service on one of the subnets, he will not be able to see what services are running on other subnets or intercept their traffic.

In addition, the benefits can be organizational. You might want to subnet services based on company structure and free developers from the cognitive burden of keeping the entire service mesh in mind.

13. Rate limiting, retries and timeouts

TL;DR: It is no longer necessary to include urgent task of request management in the codebase.

All these things could be considered separate use cases, but I decided to combine them because of one common feature: they take over the tasks of managing the request life cycle, which are usually handled by application libraries. If you are developing a Ruby on Rails web server (not integrated with a service mesh) that makes requests to backend services via gRPC, the application will have to decide what to do if N requests fail. You will also have to find out how much traffic these services will be able to handle and hardcode these parameters using a special library. Plus, the application will have to decide when to give up and let the request die (by timeout). And in order to change any of the above parameters, the web server will have to be stopped, reconfigured and restarted.

Delegating these tasks to the service mesh means not only that service developers don't have to think about them, but also that they can be considered in a more global way. If a complex chain of services is used, say A –> B –> C –> D –> E, the entire request life cycle must be considered. If the task is to extend the timeouts in service C, it is logical to do this all at once, and not in parts: by updating the service code and waiting for the pull request to be accepted and the CI system to deploy the updated service.

14. Telemetry

TL;DR: Collect all the necessary (and not quite) information from services.

Telemetry is a general term that includes metrics, distributed tracing, and logs. Service meshes offer mechanisms for collecting and processing all three types of data. This is where things get a bit fuzzy, as the number of possible options is too large. To collect metrics Prometheus and other tools to collect logs you can use fluent, Loki, Vector and more. (for example, ClickHouse with our log house for K8s - approx. transl.), for distributed tracing there is Jaeger and so on. Each service mesh may support some tools and not others. It will be interesting to see if the project can open telemetry provide some convergence.

In this case, the advantage of the service mesh technology is that sidecar containers can, in principle, collect all of the above data from their services. In other words, you have at your disposal a single system for collecting telemetry, and the service mesh can process all this information in various ways. For example:

tail'it logs from some service in the CLI;
monitor request volume from the service mesh dashboard;
collect distributed traces and redirect them to a system like Jaeger.

Attention, subjective judgment: Generally speaking, telemetry is an area in which a strong intervention of the service mesh is undesirable. Gathering basic information and tracking on the fly some golden metrics like success rates and latency is fine, but let's hope we don't see the rise of Frankenstein stacks that try to replace specialized systems, some of which are already doing great. and well studied.

15. Audit

TL;DR: Those who forget the lessons of history are doomed to repeat them.

Auditing is the art of observing important events in a system. In the case of a service mesh, this could mean keeping track of who made requests to specific endpoints of certain services, or how many times a certain security-related event occurred in the last month.

It is clear that audit is very closely related to telemetry. The difference is that telemetry is usually associated with things like performance and technical agility, while auditing can be related to legal and other issues that go beyond the strictly technical realm (for example, GDPR compliance). on data protection).

16. Visualization

TL;DR: Long live React.js - an inexhaustible source of fancy interfaces.

Perhaps there is a better term, but I don't know it. I just mean a graphical representation of the service mesh or some of its components. These visualizations can include indicators like average latency, sidecar container configuration information, health check results, and alerts.

Working in a service-oriented environment comes with a much higher cognitive load than His Majesty the Monolith. Therefore, cognitive pressure should be reduced at all costs. A simple graphical interface for a service mesh with the ability to click on a button and get the desired result can be crucial for the growth of this technology.

Were not included in the list

I originally intended to include a few more use cases on the list, but then decided not to. Here they are, along with the reasons for my decision:

Multi-datacenter. In my view, this is not so much a use case, but a narrow and specific scope of service meshes or some set of functions like service discovery.
ingress and egress. This is a related area, but I've limited myself (perhaps artificially) to the "east-west traffic" use case. Ingress and egress deserve a separate article.

Conclusion

That's all for now! Again, this list is highly arbitrary and likely incomplete. If you think that I missed something, or made a mistake in something, contact me on Twitter (@luckerkins). Please respect the rules of decency.

PS from translator

As a basis for the title illustration for the article, an image from the article "What is a Service Mesh (and when to use one)?» (author - Gregory MacKinnon). It shows how some of the functionality from the applications (in green) has moved to the service mesh that provides interconnections between them (in blue).