🥇Envoy. 1. Introduction

Greetings! This is a brief article addressing the questions: "What is envoy?", "Why is it needed?" and "Where to start?".

What it is

Envoy is an L4-L7 load balancer written in C++, designed for high performance and availability. On one hand, it is somewhat analogous to nginx and haproxy, comparable to them in terms of performance. On the other hand, it is more geared towards microservice architecture and provides functionality on par with load balancers in Java and Go, such as zuul or traefik.

A comparison table of haproxy/nginx/envoy; it does not claim absolute truth but provides a general overview.

nginx
haproxy
envoy
traefik

stars on github
11.2k/mirror
1.1k/mirror
12.4k
27.6k

written in
C
C
C++
go

API
none
socket only/push
dataplane/pull
pull

active healthcheck
none
yes
yes
yes

Open tracing
external plugin
none
yes
yes

JWT
external plugin
none
yes
none

Extension
Lua/C
Lua/C
Lua/C++
none

Why

This is a young project; it lacks many features, and some are still in early alpha. However, envoy, due to its youth, it is evolving rapidly and already has many interesting capabilities: dynamic configuration, numerous ready-made filters, a simple interface for writing your own filters.
From this, application areas arise, but to start, here are two anti-patterns:

Serving static content.

The thing is, at the moment, there envoy is no caching support. The guys from Google are trying to fix this.. The idea is to implement all the intricacies (the header zoo) of RFC compliance at once and to create an interface for specific implementations. But for now, this is not even alpha; the architecture is still under discussion. envoy It is open (while I was writing the article, a PR was merged, but this point is still relevant). PR In the meantime, use nginx for static content.

Static configuration.

You can use it, but

it was not created for this purpose. The capabilities in static configuration will not be revealed. There are many points: envoy Editing the configuration in YAML, you will make mistakes, cursing the developers for their verbosity and thinking that nginx/haproxy configs, while less structured, are more concise. This is the essence. The configuration of Nginx and Haproxy was created for manual editing, while

was designed for code generation. All configuration is described in envoy , generating it from proto files makes it much harder to err. protobufScenarios like canary, blue/green deployment, and many others are implemented well only in dynamic configuration. I'm not saying it can't be done in static; we are all doing it. But for this, you need to rely on workarounds, in any of the balancers, including

that, too. envoy Tasks where Envoy is indispensable:

Tasks where Envoy is indispensable:

Traffic balancing in complex and dynamic systems. This includes service mesh, but it isn't necessarily limited to that.
The need for functionality such as distributed tracing, complex authorization, or other features available in envoy out of the box or easily implemented, while in nginx/haproxy, one must rely on lua and dubious plugins.

Both options need to ensure high performance if necessary.

How it works

Envoy is distributed in binaries only as a Docker image. The image already contains an example of static configuration. However, we are only interested in it for understanding the structure.

envoy.yaml static configuration

static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address:
        protocol: TCP
        address: 0.0.0.0
        port_value: 10000
    filter_chains:
    - filters:
      - name: envoy.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["*"]
              routes:
              - match:
                  prefix: "\/"
                route:
                  host_rewrite: www.google.com
                  cluster: service_google
          http_filters:
          - name: envoy.router
  clusters:
  - name: service_google
    connect_timeout: 0.25s
    type: LOGICAL_DNS
    # Comment out the following line to test on v6 networks
    dns_lookup_family: V4_ONLY
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: service_google
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: www.google.com
                port_value: 443
    transport_socket:
      name: envoy.transport_sockets.tls
      typed_config:
        "@type": type.googleapis.com/envoy.api.v2.auth.UpstreamTlsContext
        sni: www.google.com

Dynamic configuration

What problem are we trying to solve? You can't just reload the load balancer configuration under load; 'minor' issues will arise:

Configuration validation.

The config can be large, possibly very large; if we reload it all at once, the chances of an error occurring somewhere increase.

Long-lived connections.

When initializing a new listener, one must take care of connections operating on the old one; if changes occur frequently and there are long-lived connections, a compromise must be found. Hello, Kubernetes ingress on nginx.

Active health checks.

If we have active health checks, we should verify them all on the new config before sending traffic. If there are many upstreams, this takes time. Hello, haproxy.

How is this resolved in envoy, by loading the config dynamically, based on the pool model, it can be split into separate parts and the part that hasn't changed does not need to be reinitialized. For example, a listener, which is expensive to reinitialize, changes rarely.

Configuration envoy (from the file above) has the following entities:

listener — a listener hanging on a specific ip/port
virtual host — a virtual host by domain name
route — a load balancing rule
cluster — a group of upstreams with balancing parameters
endpoint — the address of the upstream instance

Each of these entities plus some others can be populated dynamically; for this purpose, the configuration specifies the address of the service from which the config will be obtained. The service can be REST or gRPC, gRPC is preferable.

The services are named accordingly: LDS, VHDS, RDS, CDS, and EDS. It is possible to combine static and dynamic configurations, with the limitation that a dynamic resource cannot be indicated in static.

For most tasks, implementing the last three services is sufficient; they are called ADS (Aggregated Discovery Service), for java and Go has a ready-made gRPC dataplane implementation in which it is enough to just fill in the objects from your source.

The configuration takes the following form:

envoy.yaml dynamic configuration

dynamic_resources:
  ads_config:
    api_type: GRPC
    grpc_services:
      envoy_grpc:
        cluster_name: xds_clr
  cds_config:
    ads: {}
static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address:
        protocol: TCP
        address: 0.0.0.0
        port_value: 10000
    filter_chains:
    - filters:
      - name: envoy.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager
          stat_prefix: ingress_http
          rds:
            route_config_name: local_route
            config_source:
              ads: {}
          http_filters:
          - name: envoy.router
  clusters:
  - name: xds_clr
    connect_timeout: 0.25s
    type: LOGICAL_DNS
    dns_lookup_family: V4_ONLY
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: xds_clr
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: xds
                port_value: 6565

Upon starting envoy with this config, it will connect to the control-plane and try to request the configuration for RDS, CDS, and EDS. The interaction process is described here.

In short, envoy it sends a request, specifying the type of resource requested, the version, and the node parameters. In response, it receives the resource and version; if the version hasn't changed on the control-plane, it does not respond.
There are 4 interaction options:

One gRPC stream for all resource types, sending the complete state of the resource.
Separate streams, full state.
Single stream, incremental state.
Separate streams, incremental state.

Incremental xDS helps reduce traffic between the control plane and envoy, which is relevant for large configurations. However, it complicates interactions, as the request contains a list of resources for subscribing and unsubscribing.

In our example, ADS is used — a single stream for RDS, CDS, EDS and in non-incremental mode. To enable incremental mode, you need to specify api_type: DELTA_GRPC

Since the request contains node parameters, we can send different resources to different instances at the control plane envoy, which is convenient for building a service mesh.

Warmup

At envoy upon startup or when receiving a new configuration from the control plane, a warmup process for resources is initiated. It is divided into listener warmup and cluster warmup. The former starts when changes occur in RDS/LDS, and the latter in CDS/EDS. This means that if only upstreams change, the listener is not recreated.

During the warmup process, dependent resources from the control plane are awaited for the duration of the timeout. If the timeout expires, the initialization will fail, and the new listener will not start listening on the port.
Initialization order: EDS, CDS, active health check, RDS, LDS. With active health checks enabled, traffic will be sent to the upstream only after one successful health check.

If the listener was recreated, the old one transitions to the DRAIN state and will be removed after all connections are closed or the timeout expires. --drain-time-s, by default 10 minutes.

To be continued.

Source: habr.com