On the internet (service mesh), and here's another one. Hurrah! But why? Because I want to express my opinion that it would have been better if service meshes had emerged 10 years ago, before container platforms like Docker and Kubernetes. Iām not claiming my viewpoint is better or worse than others, but since service meshes are quite complex entities, having multiple perspectives will help to better understand them.
I will talk about the dotCloud platform, which was built on over a hundred microservices and supported thousands of applications in containers. I will explain the problems we encountered during its development and launch, and how service meshes could have helped (or not).
The story of dotCloud
I have already written about the history of dotCloud and the architecture choices for this platform, but I havenāt said much about the network layer. If you donāt want to dive into reading about dotCloud, hereās the gist: itās a Platform-as-a-Service (PaaS) that allows clients to run a wide range of applications (Java, PHP, Python, etc.), with support for a variety of data services (MongoDB, MySQL, Redis, etc.) and a workflow similar to Heroku: you upload your code to the platform, it builds container images and deploys them.
I will describe how traffic was directed on the dotCloud platform. Not because it was particularly impressive (although the system worked quite well for its time!), but primarily because with modern tools such a design can easily be implemented in a short time by a modest team, if they need a way to route traffic between a bunch of microservices or a bunch of applications. Thus, we can compare options: what you get if you develop everything yourself or use an existing service mesh. The standard choice: build it yourself or buy.
Traffic routing for hosted applications
Applications on dotCloud can provide HTTP and TCP endpoints.
HTTP endpoints are dynamically added to the load balancer cluster configuration . This is similar to what resources do today in Kubernetes and a load balancer like .
Clients connect to the HTTP endpoints through the corresponding domains, provided that the domain name points to dotCloud's load balancers. Nothing special.
TCP endpoints are associated with the port number, which is then passed to all containers of this stack through environment variables.
Clients can connect to TCP endpoints using the appropriate hostname (something like gateway-X.dotcloud.com) and port number.
This hostname resolves to the cluster of servers 'nats' (unrelated to ), which will route incoming TCP connections to the correct container (or, in the case of load-balanced services, to the correct containers).
If you are familiar with Kubernetes, this may remind you of services .
On the dotCloud platform, there was no equivalent of services : for simplicity, access to services occurred equally from both inside and outside the platform.
Everything was organized quite straightforwardly: the initial implementations of HTTP and TCP routing networks were probably only a few hundred lines of Python. Simple (I would say naive) algorithms that were refined as the platform grew and additional requirements emerged.
Extensive refactoring of existing code was not necessary. In particular, can directly use the address obtained through environment variables.
How does this differ from modern service meshes?
Limited observability. We had absolutely no metrics for TCP routing. As for HTTP routing, later versions introduced detailed HTTP metrics with error codes and response times, but modern service meshes go even further by providing integration with metric collection systems, such as Prometheus.
Observability is important not only from an operational perspective (to aid in troubleshooting) but also when releasing new features. This concerns safe and .
Routing efficiency was also limited. In the dotCloud routing grid, all traffic had to pass through a cluster of dedicated routing nodes. This meant potential crossing of multiple AZ (availability zone) boundaries and significant increased latency. I remember troubleshooting code that made over a hundred SQL requests per page, opening a new connection to the SQL server for each request. When run locally, the page loads instantly, but in dotCloud, loading takes several seconds because each TCP connection (and subsequent SQL request) requires tens of milliseconds. In this particular case, persistent connections resolved the issue.
Modern service meshes handle these issues better. Firstly, they check that the connections are routed from the source. The logical flow remains the same: client ā mesh ā service, but now the mesh operates locally rather than on remote nodes, so the connection client ā mesh is local and very fast (microseconds instead of milliseconds).
Modern service meshes also implement smarter load balancing algorithms. By monitoring the health of backends, they can direct more traffic to faster backends, resulting in improved overall performance.
Security is also better. The dotCloud routing grid operated entirely on EC2 Classic and did not encrypt traffic (based on the assumption that if someone managed to place a sniffer on EC2 network traffic, you already have big problems). Modern service meshes transparently protect all our traffic, for instance, with mutual TLS authentication and subsequent encryption.
Traffic routing for platform services
Alright, weāve discussed traffic between applications, but what about the dotCloud platform itself?
The platform itself consisted of about a hundred microservices responsible for different functions. Some accepted requests from others, while some were background workers that connected to other services but did not accept connections themselves. In any case, each service must know the endpoints of the addresses it needs to connect to.
Many high-level services can utilize the routing network described above. In fact, many of the more than a hundred microservices on dotCloud were deployed as regular applications on the dotCloud platform itself. However, a small number of low-level services (in particular, those that implement this routing network) needed something simpler, with fewer dependencies (since they could not depend on themselves to operate ā the old chicken and egg problem).
These low-level, essential services were deployed by running containers directly on several key nodes. Standard platform services such as the assembler, scheduler, and runner were not involved. If you want to compare this to modern container platforms, it resembles the operation of a control plane directly on nodes, instead of delegating the task to Kubernetes. This is quite similar to the concept of static modules (pods) used by bootkube when booting an autonomous cluster. docker run directly on the nodes, instead of delegating the task to Kubernetes. This is quite similar to the concept , which is used by or when booting an autonomous cluster.
These services were exposed in a simple and crude way: their names and addresses were listed in a YAML file; each client had to take a copy of this YAML file for deployment.
On one hand, this is extremely reliable because it does not require support from an external key/value store such as Zookeeper (keep in mind, at that time etcd or Consul did not yet exist). On the other hand, it complicated service movement. Each time a service was moved, all clients had to obtain the updated YAML file (and potentially restart). Not very convenient!
Subsequently, we began implementing a new scheme where each client connected to a local proxy server. Instead of an address and port, it was sufficient to know just the service's port number and connect through localhost. The local proxy server manages this connection and directs it to the actual server. Now, when moving the backend to another machine or scaling, instead of updating all clients, only all these local proxies need to be updated; and restarting is no longer required.
(It was also planned to encapsulate traffic in TLS connections and set up another proxy server on the receiving side, as well as to check TLS certificates without involving the receiving service, which is configured to accept connections only on localhost).
This is very similar to from Airbnb, but the significant difference is that SmartStack is implemented and deployed in production, while the internal routing system of dotCloud was shelved when dotCloud turned into Docker.
I personally consider SmartStack to be one of the predecessors of systems like Istio, Linkerd, and Consul Connect, because they all follow the same pattern:
- Running a proxy on each node.
- Clients connect to the proxy.
- The control plane updates the proxy server configuration as backends change.
- ā¦ Profit!
Modern implementation of a service mesh
If we need to implement such a mesh today, we can use similar principles. For example, setting up an internal DNS zone, mapping service names to addresses in the space 127.0.0.0/8. Then run HAProxy on each node of the cluster, accepting connections on each service address (in this subnet 127.0.0.0/8) and redirecting/balancing the load to the corresponding backends. The HAProxy configuration can be managed , allowing backend information to be stored in etcd or Consul and automatically pushing the updated configuration to HAProxy when needed.
This is roughly how Istio works! But with some differences:
- One of the local cloud providers in the USA also utilizes instead of HAProxy.
- It retains backend configuration through Kubernetes API instead of etcd or Consul.
- Services are allocated addresses in the internal subnet (Kubernetes ClusterIP addresses) instead of 127.0.0.0/8.
- It has an additional component (Citadel) for adding mutual TLS authentication between clients and servers.
- Supports new features such as circuit breaking, distributed tracing, canary deployments, etc.
Let's briefly look at some differences.
Envoy Proxy
Envoy Proxy was written by Lyft [a competitor to Uber in the taxi market ā translator's note]. It is very similar to other proxies (like HAProxy, Nginx, Traefikā¦), but Lyft created their own because they needed features not available in other proxies, and it seemed more sensible to make a new one than to extend the existing.
Envoy can be used on its own. If I have a specific service that needs to connect to other services, I can configure it to connect to Envoy, and then dynamically adjust and reconfigure Envoy with the locations of the other services, gaining many excellent additional features, such as observability. Instead of a custom client library or embedding call tracing in code, we route traffic to Envoy, and it collects metrics for us.
But Envoy can also operate as the data plane for the service mesh. This means that now for this service mesh, Envoy is configured as the control plane. (control plane).
The control plane
In the control plane, Istio relies on the Kubernetes API. This is not very different from using confd,which relies on etcd or Consul to view a set of keys in a data store. Istio queries a set of Kubernetes resources through the Kubernetes API.
Meanwhile,I found this to be helpful, which states that:
The Kubernetes API server is a 'dumb server' that offers storage, version management, validation, updating, and semantic handling of API resources.
Istio is designed to work with Kubernetes; and if you want to use it outside of Kubernetes, you need to run an instance of the Kubernetes API server (and the auxiliary etcd service).
Service Addresses
Istio relies on ClusterIP addresses allocated by Kubernetes, so Istio services receive an internal address (not in the range 127.0.0.0/8).
Traffic to the ClusterIP address for a specific service in the Kubernetes cluster without Istio is intercepted by kube-proxy and sent to the backend of that proxy. If you are interested in the technical details, kube-proxy sets iptables rules (or IPVS load balancers, depending on how itās configured) to rewrite the destination IP addresses of connections going to the ClusterIP address.
After installing Istio in the Kubernetes cluster, nothing changes until it is explicitly enabled for a given consumer or even the entire namespace by introducing a sidecar container into custom pods. This container will run an instance of Envoy and set up a number of iptables rules to intercept traffic going to other services and redirect that traffic to Envoy.
When integrating with Kubernetes DNS, this means our code can connect by service name, and everything "just works". In other words, our code issues requests like http://api/v1/users/4242, then api resolves the request to 10.97.105.48, iptables rules intercept connections from 10.97.105.48 and redirect them to a local Envoy proxy, and this local proxy will forward the request to the actual API backend. Phew!
Extra touches
Istio also provides end-to-end encryption and authentication through mTLS (mutual TLS). This is handled by a component called Citadel.
There is also a component Mixer, which Envoy can query for unstructured output the request to make a special decision about this request based on various factors, such as headers, backend load, etcā¦ (donāt worry: there are many ways to ensure Mixer works, and even if it fails, Envoy will continue to operate normally as a proxy).
And of course, we mentioned observability: Envoy collects a massive amount of metrics while providing distributed tracing. In a microservices architecture, if a single API request needs to pass through microservices A, B, C, and D, distributed tracing will add a unique identifier to the request upon entry and preserve this identifier through the sub-requests to all these microservices, allowing tracking of all related calls, their latencies, and so on.
Develop or buy
Istio has a reputation for being a complex system. In contrast, building the routing mesh I described at the beginning of this post is relatively simple with existing tools. So is it worth creating your own service mesh instead?
If we have modest needs (no need for observability, circuit breakers, and other intricacies), thoughts of developing our own tool come to mind. But if we are using Kubernetes, it may not even be necessary, as Kubernetes already provides the basic tools for service discovery and load balancing.
But if we have advanced requirements, then "buying" a service mesh seems like a much better option. (It isn't always strictly "buying" since Istio comes with open source, but we still need to invest engineering time to understand how it works, deploy it, and manage it).
What to choose: Istio, Linkerd, or Consul Connect?
So far, we have only talked about Istio, but it is not the only service mesh. A popular alternative is , and there is also .
What to choose?
Honestly, I donāt know. At the moment, I donāt consider myself competent enough to answer this question. There are several comparisons of these tools and even .
One promising approach is to use a tool like . It implements an abstraction layer to simplify and unify the APIs provided by service meshes. Instead of studying the specific (and, in my opinion, relatively complex) APIs of various service meshes, we can use simpler constructs from SuperGloo ā and easily switch from one to another, as if we have an intermediate configuration format that describes HTTP interfaces and backends capable of generating actual configuration for Nginx, HAProxy, Traefik, Apache...
I have played a bit with Istio and SuperGloo, and in the next article, I want to show how to add Istio or Linkerd to an existing cluster using SuperGloo, and how well the latter handles its job of allowing switching from one service mesh to another without rewriting configurations.
Source: habr.com
