Back to microservices with Istio. Part 1

Note. transl.: Service meshes have definitely become a hot topic in today's infrastructure for applications following microservice architecture. While Istio may be on the radar of many DevOps engineers, it is a fairly new product that, while complex in terms of features it provides, can take a significant amount of time to get to know. The German engineer Rinor Maloku, who is in charge of cloud computing for large clients at the telecommunications company Orange Networks, has written a wonderful series of materials that allow you to quickly and deeply dive into Istio. He begins his story with what Istio can do and how you can quickly see it with your own eyes.

Istio — Open Source-project, developed in collaboration with teams from Google, IBM and Lyft. It solves the complexities that arise in applications based on microservices, for example, such as:

• traffic management: timeouts, retries, load balancing;
• Security: end user authentication and authorization;
• observability: tracing, monitoring, logging.

All of them can be solved at the application level, however after that your services will no longer be "micro". All the extra effort to address these issues is a waste of company resources that could be used directly for business value. Consider an example:

Project Manager: How long does it take to add a feedback feature?
Developer: Two sprints.

MP: What?.. It's just CRUD!
R: Doing CRUD is the easy part of the task, but we still need to authenticate and authorize users and services. Since the network is unreliable, you will need to implement repeated requests, as well as circuit breaker pattern in clients. Also, to make sure that the whole system did not crash, timeouts and bulk heads (See later in the article for more details on both mentioned patterns.), and in order to detect problems, monitoring, tracing, […]

MP: Oh, let's just put this feature into the Product service then.

I think the idea is clear: the amount of steps and effort required to add a single service is huge. In this article, we'll take a look at how Istio removes all of the complexity mentioned above (not targeted by business logic) from services.

Note: The article assumes that you have working knowledge of Kubernetes. Otherwise, I recommend reading my introduction to Kubernetes and only then continue reading this material.

Istio idea

In a world without Istio, one service makes direct requests to another, and in case of failure, the service must handle it itself: make a new attempt, provide for a timeout, open a circuit breaker, etc.

Network traffic in Kubernetes

Istio, on the other hand, offers a specialized solution that is completely separate from services and functions by interfering with network interaction. And thus it implements:

• fault tolerance: based on the status code in the response, it understands if the request failed and resubmits it.
• Canary Rollouts: redirects only a fixed percentage of requests to the new version of the service.
• Monitoring and Metrics: how long did it take for the service to respond?
• Tracing and observability: Adds special headers to each request and traces them across the cluster.
• Security: Retrieves a JWT token, authenticates and authorizes users.

These are just a few of the possibilities (really just a few!) to intrigue you. Now let's dive into the technical details!

Architecture

Istio intercepts all network traffic and applies a set of rules to it, inserting a smart proxy in the form of a sidecar container into each pod. Proxies that activate all possibilities form a data plane, and they can be dynamically adjusted with Control plane.

data plane

The proxies that are inserted into the pods allow Istio to easily achieve the requirements we need. For example, let's check the retries and circuit breaker functions.

How retries and circuit breaking are implemented in Envoy

To summarize:

1. Envoy (we are talking about a proxy located in a sidecar container, which is distributed and how separate product - approx. transl.) sends a request to the first instance of service B and fails.
2. Envoy Sidecar is trying again (retry). (1)
3. The failed request is returned to the proxy that called it.
4. This opens the Circuit Breaker and calls the next service for subsequent requests. (2)

This means that you don't have to use the next Retry library, you don't have to make your own implementation of Circuit Breaking and Service Discovery in the X, Y or Z programming language. All this and more is available out of the box in Istio and does not require no code changes.

Great! Now you may want to go on a voyage with Istio, but there are still some doubts, open questions. If this is a universal solution for all occasions in life, then you have a legitimate suspicion: after all, all such solutions are in fact not suitable for any case.

And finally you ask: “Is it customizable?”

Now you are ready for a sea voyage - and let's get acquainted with Control Plane.

Control plane

It consists of three components: Pilot, Mixer и Citadel, which together configure Envoys to route traffic, enforce policies, and collect telemetry data. Schematically, it all looks like this:

Interaction of Control Plane with Data Plane

Envoys (i.e. data plane) are configured with Kubernetes CRD (Custom Resource Definitions) defined by Istio and specifically designed for this purpose. What this means to you is that they are just another resource in Kubernetes with a familiar syntax. Once created, this resource will be picked up by the control plane and applied to Envoys.

Relationship of services to Istio

We have described Istio's relationship to services, but not the other way around: how do services relate to Istio?

To be honest, services know about the presence of Istio as well as fish know about water, when they ask themselves: “What is water anyway?”.

Illustration Victoria Dimitrakopoulos: How do you like the water? - What is water anyway?

Thus, you can take a working cluster and after deploying the Istio components, the services in it will continue to work, and after removing these components, everything will be fine again. It is clear that in this case you will lose the opportunities provided by Istio.

Enough theory - let's put this knowledge into practice!

Istio in practice

Istio requires a Kubernetes cluster with at least 4 vCPUs and 8 GB of RAM available. To quickly raise the cluster and follow the instructions from the article, I recommend using the Google Cloud Platform, which offers new users free $300.

After creating the cluster and setting up access to Kubernetes through the console utility, you can install Istio through the Helm package manager.

Helm Installation

Install the Helm client on your computer as described in official documentation. We will use it to generate templates for installing Istio in the next section.

Installation

Download Istio resources from latest release (the original author's link to version 1.0.5 has been changed to the current one, i.e. 1.0.6 - approx. transl.), extract the contents to a single directory, which I will refer to as [istio-resources].

For easy identification of Istio resources, create a namespace in the K8s cluster istio-system:

$ kubectl create namespace istio-system

Complete the installation by navigating to the directory [istio-resources] and running the command:

$ helm template install/kubernetes/helm/istio 
  --set global.mtls.enabled=false 
  --set tracing.enabled=true 
  --set kiali.enabled=true 
  --set grafana.enabled=true 
  --namespace istio-system > istio.yaml

This command will output the key components of Istio to a file istio.yaml. We have modified the standard template for ourselves by specifying the following parameters:

• global.mtls.enabled installed in false (i.e. mTLS authentication is disabled - approx. transl.)to simplify our dating process;
• tracing.enabled enables request tracing with Jaeger;
• kiali.enabled installs Kiali into a cluster to visualize services and traffic;
• grafana.enabled installs Grafana to visualize the collected metrics.

Apply the generated resources with the command:

$ kubectl apply -f istio.yaml

Installation of Istio in the cluster is complete! Wait until all pods in the namespace istio-system will be able Running or Completedby running the command below:

$ kubectl get pods -n istio-system

We are now ready to continue on to the next section, where we will raise and run the application.

Sentiment Analysis Application Architecture

Let's use the example of the Sentiment Analysis microservice application used in the already mentioned Introduction article to Kubernetes. It is complex enough to show the possibilities of Istio in practice.

The application consists of four microservices:

1. Service SA-Frontend, which serves the front-end application on Reactjs;
2. Service SA Web App, which serves Sentiment Analysis queries;
3. Service SA Logicwhich performs itself sentiment analysis;
4. Service SA Feedback, which receives feedback from users on the accuracy of the analysis performed.

In this diagram, in addition to services, we also see the Ingress Controller, which in Kubernetes routes incoming requests to the corresponding services. Istio uses a similar concept as part of the Ingress Gateway, details of which will follow.

Launching an application with a proxy from Istio

For further operations mentioned in the article, clone your repository istio-mastery. It contains the application and manifests for Kubernetes and Istio.

Inserting sidecars

Insertion can be made automatically or manually. To automatically insert sidecar containers, you need to set the label to the namespace istio-injection=enabled, which is done by the following command:

$ kubectl label namespace default istio-injection=enabled
namespace/default labeled

Now each pod that will be deployed in the default namespace (default) will get its sidecar container. To verify this, let's deploy a test application by going to the root directory of the repository [istio-mastery] and running the following command:

$ kubectl apply -f resource-manifests/kube
persistentvolumeclaim/sqlite-pvc created
deployment.extensions/sa-feedback created
service/sa-feedback created
deployment.extensions/sa-frontend created
service/sa-frontend created
deployment.extensions/sa-logic created
service/sa-logic created
deployment.extensions/sa-web-app created
service/sa-web-app created

Having deployed the services, check that the pods have two containers (with the service itself and its sidecar) by running the command kubectl get pods and making sure that under the column READY value specified 2/2, symbolizing that both containers are running:

$ kubectl get pods
NAME                           READY     STATUS    RESTARTS   AGE
sa-feedback-55f5dc4d9c-c9wfv   2/2       Running   0          12m
sa-frontend-558f8986-hhkj9     2/2       Running   0          12m
sa-logic-568498cb4d-2sjwj      2/2       Running   0          12m
sa-logic-568498cb4d-p4f8c      2/2       Running   0          12m
sa-web-app-599cf47c7c-s7cvd    2/2       Running   0          12m

Visually it looks like this:

Envoy proxy in one of the pods

Now that the application is up and running, we need to allow incoming traffic to enter the application.

Ingress Gateway

The best practice to achieve this (allow traffic in the cluster) is through Ingress Gateway in Istio, which is located at the “edge” of the cluster and allows you to enable Istio features such as routing, load balancing, security, and monitoring for incoming traffic.

The Ingress Gateway component and the service that forwards it outward were installed on the cluster during the Istio installation. To find out the external IP address of a service, run:

$ kubectl get svc -n istio-system -l istio=ingressgateway
NAME                   TYPE           CLUSTER-IP     EXTERNAL-IP
istio-ingressgateway   LoadBalancer   10.0.132.127   13.93.30.120

We will continue to access the application using this IP (I will refer to it as EXTERNAL-IP), so for convenience, we will write the value to a variable:

$ EXTERNAL_IP=$(kubectl get svc -n istio-system 
  -l app=istio-ingressgateway 
  -o jsonpath='{.items[0].status.loadBalancer.ingress[0].ip}')

If you try to access this IP through a browser now, you will get a Service Unavailable error, because by default Istio blocks all incoming trafficuntil Gateway is defined.

Gateway resource

Gateway is a CRD (Custom Resource Definition) in Kubernetes, defined after installing Istio in a cluster and enabling the ability to specify ports, protocol, and hosts for which we want to allow incoming traffic.

In our case, we want to allow HTTP traffic on port 80 for all hosts. The problem is realized by the following definition (http-gateway.yaml):

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: http-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
- "*"

This configuration needs no explanation except for the selector istio: ingressgateway. With this selector, we can specify which Ingress Gateway to apply the configuration to. In our case, this is the Ingress Gateway controller, which was installed by default in Istio.

The configuration is applied by calling the following command:

$ kubectl apply -f resource-manifests/istio/http-gateway.yaml gateway.networking.istio.io/http-gateway created

The gateway now allows access to port 80 but has no idea where to route the requests. For this you will need Virtual Services.

Virtual Service resource

The VirtualService tells the Ingress Gateway how to route requests that are allowed within the cluster.

Requests to our application coming through the http-gateway must be sent to the sa-frontend, sa-web-app and sa-feedback services:

Routes to be configured with VirtualServices

Consider the requests that should be sent to SA-Frontend:

• Exact match along the way / should be sent to SA-Frontend to get index.html;
• Paths with a prefix /static/* should be sent to SA-Frontend to get static files used in the frontend, such as CSS and JavaScript;
• Paths matching the regular expression '^.*.(ico|png|jpg)$', must be sent to SA-Frontend, because These are the pictures displayed on the page.

The implementation is achieved by the following configuration (sa-virtualservice-external.yaml):

kind: VirtualService
metadata:
  name: sa-external-services
spec:
  hosts:
  - "*"
  gateways:
  - http-gateway                      # 1
  http:
  - match:
    - uri:
        exact: /
    - uri:
        exact: /callback
    - uri:
        prefix: /static
    - uri:
        regex: '^.*.(ico|png|jpg)

Важные моменты:



 	
 Этот VirtualService относится к запросам, приходящим через http-gateway;


 	
 В destination определяется сервис, куда отправляются запросы.




Примечание: Конфигурация выше хранится в файле sa-virtualservice-external.yaml, который также содержит настройки для маршрутизации в SA-WebApp и SA-Feedback, но был сокращён здесь в статье для лаконичности.

Применим VirtualService вызовом:

$ kubectl apply -f resource-manifests/istio/sa-virtualservice-external.yaml
virtualservice.networking.istio.io/sa-external-services created
Примечание: Когда мы применяем ресурсы Istio, Kubernetes API Server создаёт событие, которое получает Istio Control Plane, и уже после этого новая конфигурация применяется к прокси-серверам Envoy каждого pod'а. А контроллер Ingress Gateway представляется очередным Envoy, сконфигурированным в Control Plane. Всё это на схеме выглядит так:


Конфигурация Istio-IngressGateway для маршрутизации запросов
Приложение Sentiment Analysis стало доступным по http://{EXTERNAL-IP}/. Не переживайте, если вы получаете статус Not Found: иногда требуется чуть больше времени для того, чтобы конфигурация вступила в силу и кэши Envoy обновились.
Перед тем, как продолжить, поработайте немного с приложением, чтобы сгенерировать трафик (его наличие необходимо для наглядности в последующих действиях — прим. перев.).
Kiali : наблюдаемость
Чтобы попасть в административный интерфейс Kiali, выполните следующую команду:
$ kubectl port-forward 
    $(kubectl get pod -n istio-system -l app=kiali 
    -o jsonpath='{.items[0].metadata.name}') 
    -n istio-system 20001
… и откройте http://localhost:20001/, залогинившись под admin/admin. Здесь вы найдете множество полезных возможностей, например, для проверки конфигурации компонентов Istio, визуализации сервисов по информации, собранной при перехвате сетевых запросов, получения ответов на вопросы «Кто к кому обращается?», «У какой версии сервиса возникают сбои?» и т.п. В общем, изучите возможности Kiali перед тем, как двигаться дальше — к визуализации метрик с Grafana.

Grafana: визуализация метрик
Собранные в Istio метрики попадают в Prometheus и визуализируются с Grafana. Чтобы попасть в административный интерфейс Grafana, выполните команду ниже, после чего откройте http://localhost:3000/:
$ kubectl -n istio-system port-forward 
    $(kubectl -n istio-system get pod -l app=grafana 
    -o jsonpath={.items[0].metadata.name}) 3000
Кликнув на меню Home слева сверху и выбрав Istio Service Dashboard в левом верхнем углу, начните с сервиса sa-web-app, чтобы посмотреть на собранные метрики:

Здесь нас ждёт пустое и совершенно скучное представление — руководство никогда такое не одобрит. Давайте же создадим небольшую нагрузку следующей командой:
$ while true; do 
    curl -i http://$EXTERNAL_IP/sentiment 
    -H "Content-type: application/json" 
    -d '{"sentence": "I love yogobella"}'; 
    sleep .8; done
Вот теперь у нас гораздо более симпатичные графики, а в дополнение к ним — замечательные инструменты Prometheus для мониторинга и Grafana для визуализации метрик, что позволят нам узнать о производительности, состоянии здоровья, улучшениях/деградации в работе сервисов на протяжении времени.
Наконец, посмотрим на трассировку запросов в сервисах.
Jaeger : трассировка
Трассировка нам потребуется, потому что чем больше у нас сервисов, тем сложнее добраться до причины сбоя. Посмотрим на простой случай из картинки ниже:


Типовой пример случайного неудачного запроса
Запрос приходит, падает — в чём же причина? Первый сервис? Или второй? Исключения есть в обоих — давайте посмотрим на логи каждого. Как часто вы ловили себя за таким занятием? Наша работа больше похожа на детективов программного обеспечения, а не разработчиков…
Это широко распространённая проблема в микросервисах и решается она распределёнными системами трассировки, в которых сервисы передают друг другу уникальный заголовок, после чего эта информация перенаправляется в систему трассировки, где она сопоставляется с данными запроса. Вот иллюстрация:


Для идентификации запроса используется TraceId
В Istio используется Jaeger Tracer, который реализует независимый от вендоров фреймворк OpenTracing API. Получить доступ к пользовательского интерфейсу Jaeger можно следующей командой:
$ kubectl port-forward -n istio-system 
    $(kubectl get pod -n istio-system -l app=jaeger 
    -o jsonpath='{.items[0].metadata.name}') 16686
Теперь зайдите на http://localhost:16686/ и выберите сервис sa-web-app. Если сервис не показан в выпадающем меню — проявите/сгенерируйте активность на странице и обновите интерфейс. После этого нажмите на кнопку Find Traces, которая покажет самые последние трейсы — выберите любой — покажется детализированная информация по всем трейсам:

Этот трейс показывает:

 Запрос приходит в istio-ingressgateway (это первое взаимодействие с одним из сервисов, и для запроса генерируется Trace ID), после чего шлюз направляет запрос в сервис sa-web-app.
 В сервисе sa-web-app запрос подхватывается Envoy sidecar'ом, создаётся «ребёнок» в span'е (поэтому мы видим его в трейсах) и перенаправляется в контейнер sa-web-app. (Span — логическая единица работы в Jaeger, имеющая название, время начало операции и её продолжительность. Span'ы могут быть вложенными и упорядоченными. Ориентированный ациклический граф из span'ов образует trace. — прим. перев.)
 Здесь запрос обрабатывается методом sentimentAnalysis. Эти трейсы уже сгенерированы приложением, т.е. для них потребовались изменения в коде.
 С этого момента инициируется POST-запрос в sa-logic. Trace ID должен быть проброшен из sa-web-app.
 …

Примечание: На 4 шаге приложение должно увидеть заголовки, сгенерированные Istio, и передать их в последующие запросы, как показано на изображении ниже:


(A) За проброс заголовков отвечает Istio; (B) За заголовки отвечают сервисы
Istio делает основную работу, т.к. генерирует заголовки для входящих запросов, создаёт новые span'ы в каждом sidecare'е и пробрасывает их. Однако без работы с заголовками внутри сервисов полный путь трассировки запроса будет утерян.
Необходимо учитывать (пробрасывать) следующие заголовки:
x-request-id
x-b3-traceid
x-b3-spanid
x-b3-parentspanid
x-b3-sampled
x-b3-flags
x-ot-span-context
Это несложная задача, однако для упрощения её реализации уже существует множество библиотек — например, в сервисе sa-web-app клиент RestTemplate пробрасывает эти заголовки, если просто добавить библиотеки Jaeger и OpenTracing в его зависимости.
Заметьте, что приложение Sentiment Analysis демонстрирует реализации на Flask, Spring и ASP.NET Core.
Теперь, когда стало ясно, что мы получаем из коробки (или почти «из коробки»), рассмотрим вопросы тонко настраиваемой маршрутизации, управления сетевым трафиком, безопасности и т.п.!
Прим. перев.: об этом читайте в следующей части материалов по Istio от Rinor Maloku, переводы которых последуют в нашем блоге в ближайшее время. UPDATE (14 марта): Вторая часть уже опубликована.
P.S. от переводчика
Читайте также в нашем блоге:

 «Назад к микросервисам вместе с Istio»: часть 2 (маршрутизация, управление трафиком), часть 3 (аутентификация и авторизация);
 «Conduit — легковесный service mesh для Kubernetes»;
 «Что такое service mesh и почему он мне нужен [для облачного приложения с микросервисами]?»;
 «Иллюстрированное руководство по устройству сети в Kubernetes. Части 1 и 2»;
 «Как этот sidecar-контейнер оказался здесь [в Kubernetes]?».

Источник: habr.com
route:

- destination:

host: sa-frontend             # 2

port:

number: 80

Important points:

 This VirtualService refers to requests coming through http-gateway;
 В destination defines the service to which the requests are sent.

Note: The configuration above is stored in a file sa-virtualservice-external.yaml, which also contains settings for routing to SA-WebApp and SA-Feedback, but has been shortened here in the article for brevity.
Apply VirtualService by calling:

Note: When we apply Istio resources, the Kubernetes API Server fires an event that the Istio Control Plane receives, and after that the new configuration is applied to each pod's Envoy proxy. And the Ingress Gateway controller appears to be another Envoy configured in the Control Plane. All this looks like this in the diagram:


Istio-IngressGateway configuration for request routing
Sentiment Analysis is now available on http://{EXTERNAL-IP}/. Don't worry if you get Not Found status: sometimes it takes a little longer for the configuration to take effect and for the Envoy caches to update.
Before proceeding, play a little with the application to generate traffic (its presence is necessary for clarity in subsequent actions - approx. transl.).
Kiali: observability
To get to the Kiali admin interface, run the following command:

…and open http://localhost:20001/by logging in as admin/admin. Here you will find many useful features, for example, to check the configuration of Istio components, visualize services from information collected by intercepting network requests, get answers to the questions “Who is contacting whom?”, “Which version of the service is experiencing failures?” and so on. In general, explore the possibilities of Kiali before moving on to visualizing metrics with Grafana.

Grafana: visualization of metrics
The metrics collected in Istio end up in Prometheus and are visualized with Grafana. To get to the Grafana admin interface, run the command below, then open http://localhost:3000/:

By clicking on the menu Home top left and select Istio Service Dashboard in the top left corner, start with service sa-web-appto view the collected metrics:

Here we are waiting for an empty and completely boring performance - management will never approve of this. Let's create a small load with the following command:

Now we have much prettier graphs, and in addition to them, the wonderful tools Prometheus for monitoring and Grafana for visualizing metrics, which will allow us to learn about performance, health status, improvements / degradation in services over time.
Finally, let's look at request tracing in services.
Jaeger: tracing
We will need tracing, because the more services we have, the more difficult it is to get to the cause of the failure. Let's look at a simple case from the picture below:


Typical example of a random failed request
Request comes, falls - what is the reason? First service? Or second? There are exceptions in both - let's look at the logs of each. How often have you caught yourself doing this? Our job is more like software detectives than developers…
This is a widespread problem in microservices and is solved by distributed tracing systems, in which services pass a unique header to each other, after which this information is redirected to the tracing system, where it is compared with the request data. Here is an illustration:


TraceId is used to identify the request
Istio uses Jaeger Tracer, which implements a vendor-independent OpenTracing API framework. You can access the Jaeger user interface with the following command:

Now go to http://localhost:16686/ and select a service sa-web-app. If the service is not shown in the dropdown menu, show/generate activity on the page and update the interface. After that click on the button Find traces, which will show the most recent traces - select any - detailed information on all traces will appear:

This trace shows:

 The request comes in istio-ingressgateway (this is the first interaction with one of the services, and a Trace ID is generated for the request), after which the gateway sends the request to the service sa-web-app.
 In the service sa-web-app the request is picked up by the Envoy sidecar, a "child" is created in the span (that's why we see it in traces) and redirected to the container sa-web-app. (Team - a logical unit of work in Jaeger, having a name, the start time of the operation and its duration. Spans can be nested and ordered. A directed acyclic graph of spans forms a trace. - approx. transl.)
 Here the request is processed by the method sentimentAnalysis. These traces are already generated by the application, i.e. they required code changes.
 From this moment, a POST request is initiated in sa-logic. Trace ID must be forwarded from sa-web-app.
 ...

Note: In step 4, the application should see the headers generated by Istio and pass them to subsequent requests, as shown in the image below:


(A) Header forwarding is the responsibility of Istio; (B) Services are responsible for headers
Istio does the bulk of the work because generates headers for incoming requests, creates new spans in each sidecare and forwards them. However, without working with headers inside services, the full request trace path will be lost.
The following headers must be considered (forwarded):

This is a simple task, but to simplify its implementation, there is already many libraries - for example, in the sa-web-app service, the RestTemplate client forwards these headers if you simply add the Jaeger and OpenTracing libraries to its dependencies.
Note that the Sentiment Analysis application demonstrates implementations in Flask, Spring, and ASP.NET Core.
Now that it's clear what we're getting out of the box (or almost out of the box), let's look at fine-tuning routing, network traffic management, security, and more!
Note. transl.: read about this in the next part of the materials on Istio from Rinor Maloku, the translations of which will follow on our blog in the near future. UPDATED (March 14th): The second part already published.
PS from translator
Read also on our blog:

 "Back to microservices with Istio": part 2 (routing, traffic control), part 3 (authentication and authorization);
 «Conduit - lightweight service mesh for Kubernetes";
 «What is a service mesh and why do I need it [for a cloud application with microservices]?";
 «An illustrated guide to networking in Kubernetes. Parts 1 and 2";
 «How did this sidecar container end up here [in Kubernetes]?».

Source: habr.com