CPU yekushandisa bhenji yeIstio uye Linkerd

Nhanganyaya

Isu tiri Shopify akatanga kuendesa Istio sevhisi mesh. Muchidimbu, zvese zvakanaka, kunze kwechinhu chimwe chete: inodhura.

В mabhenji akaburitswa nokuti Istio inoti:

NeIstio 1.1, proxy inoshandisa ingangoita 0,6 vCPUs (virtual cores) pa 1000 zvikumbiro pasekondi.

Kudunhu rekutanga musevhisi mesh (2 proxies kudivi rega rega rekubatanidza), isu tichave ne1200 cores chete yeproxy, pamwero wemiriyoni zvikumbiro pasekondi. Zvinoenderana neGoogle's mutengo Calculator, inoshanda kuita ingangoita madhora makumi mana / mwedzi / musimboti wekugadzirisa. n1-standard-64, ndiko kuti, iyi nharaunda chete ichatipa mari inodarika zviuru makumi mashanu zvemadhora pamwedzi pamikumbiro yemiriyoni pasekondi.

Ivan Sim (Ivan Sim) nemaziso zvichienzaniswa service mesh kunonoka gore rapfuura uye yakavimbisa zvakafanana kune ndangariro uye processor, asi hazvina kushanda:

Sezviri pachena, values-istio-test.yaml ichawedzera zvakanyanya CPU zvikumbiro. Kana ndaita masvomhu angu nemazvo, unoda angangoita makumi maviri nemana eCPU cores epanera rekutonga uye 24 CPU yemumiriri wega wega. Handina zvakawanda kudaro. Ndichadzokorora bvunzo kana zvimwe zviwanikwa zvapihwa kwandiri.

Ini ndaida kuzvionera ndega kuti kuita kweIstio kwakafanana sei kune imwe yakavhurika sosi sevhisi mesh: Linkerd.

Service mesh installation

Chokutanga pane zvose, ndakaiisa muchikwata SuperGloo:

$ supergloo init
installing supergloo version 0.3.12
using chart uri https://storage.googleapis.com/supergloo-helm/charts/supergloo-0.3.12.tgz
configmap/sidecar-injection-resources created
serviceaccount/supergloo created
serviceaccount/discovery created
serviceaccount/mesh-discovery created
clusterrole.rbac.authorization.k8s.io/discovery created
clusterrole.rbac.authorization.k8s.io/mesh-discovery created
clusterrolebinding.rbac.authorization.k8s.io/supergloo-role-binding created
clusterrolebinding.rbac.authorization.k8s.io/discovery-role-binding created
clusterrolebinding.rbac.authorization.k8s.io/mesh-discovery-role-binding created
deployment.extensions/supergloo created
deployment.extensions/discovery created
deployment.extensions/mesh-discovery created
install successful!

Ndakashandisa SuperGloo nekuti inoita kuti bootstrapping mesh yebasa ive nyore. Ndaisafanira kuita zvakawanda. Isu hatishandise SuperGloo mukugadzira, asi yakanakira basa rakadaro. Ini ndaifanira kushandisa chaizvo akati wandei mirairo kune yega yega sevhisi mesh. Ndakashandisa masumbu maviri ekuzviparadzanisa nevamwe - rimwe nerimwe reIstio neLinkerd.

Muedzo uyu wakaitwa paGoogle Kubernetes Injini. Ndakashandisa Kubernetes 1.12.7-gke.7 uye dziva remanodhi n1-standard-4 ine otomatiki node kuyera (zvishoma 4, yepamusoro 16).

Ipapo ini ndakaisa ese ese meshes masevhisi kubva kumutsetse wekuraira.

First Linkerd:

$ supergloo install linkerd --name linkerd
+---------+--------------+---------+---------------------------+
| INSTALL |     TYPE     | STATUS  |          DETAILS          |
+---------+--------------+---------+---------------------------+
| linkerd | Linkerd Mesh | Pending | enabled: true             |
|         |              |         | version: stable-2.3.0     |
|         |              |         | namespace: linkerd        |
|         |              |         | mtls enabled: true        |
|         |              |         | auto inject enabled: true |
+---------+--------------+---------+---------------------------+

Zvadaro Istio:

$ supergloo install istio --name istio --installation-namespace istio-system --mtls=true --auto-inject=true
+---------+------------+---------+---------------------------+
| INSTALL |    TYPE    | STATUS  |          DETAILS          |
+---------+------------+---------+---------------------------+
| istio   | Istio Mesh | Pending | enabled: true             |
|         |            |         | version: 1.0.6            |
|         |            |         | namespace: istio-system   |
|         |            |         | mtls enabled: true        |
|         |            |         | auto inject enabled: true |
|         |            |         | grafana enabled: true     |
|         |            |         | prometheus enabled: true  |
|         |            |         | jaeger enabled: true      |
+---------+------------+---------+---------------------------+

Iyo crash-loop yakatora maminetsi mashoma, uye ipapo ma control panel akagadzikana.

(Cherechedza: SuperGloo inotsigira Istio 1.0.x chete parizvino. Ndakadzokorora kuedza neIstio 1.1.3, asi handina kuona musiyano unooneka.)

Kumisikidza Istio otomatiki Deployment

Kuita kuti Istio iise iyo sidecar Envoy, isu tinoshandisa sidecar jekiseni - MutatingAdmissionWebhook. Hatisi kuzotaura nezvazvo munyaya ino. Rega ndingotaura kuti uyu mutongi anotarisisa kuwanikwa kwemapodhi ese matsva uye anowedzera zvine simba sidecar uye initContainer, iyo inotarisira mabasa. iptables.

Isu paShopify takanyora yedu yekuwana controller kuti tiite sidecars, asi kune iyi bhenji ndakashandisa controller inouya neIstio. Iyo controller inobaya sidecars nekukasira kana paine nzira yekudimbudzira munzvimbo yezita istio-injection: enabled:

$ kubectl label namespace irs-client-dev istio-injection=enabled
namespace/irs-client-dev labeled

$ kubectl label namespace irs-server-dev istio-injection=enabled
namespace/irs-server-dev labeled

Kumisikidza otomatiki Linkerd deployment

Kumisikidza Linkerd sidecar embedding, isu tinoshandisa zvirevo (ndakazviwedzera nemaoko kuburikidza kubectl edit):

metadata:
  annotations:
    linkerd.io/inject: enabled

$ k edit ns irs-server-dev 
namespace/irs-server-dev edited

$ k get ns irs-server-dev -o yaml
apiVersion: v1
kind: Namespace
metadata:
  annotations:
    linkerd.io/inject: enabled
  name: irs-server-dev
spec:
  finalizers:
  - kubernetes
status:
  phase: Active

Istio Mhosva Kushivirira Simulator

Isu takavaka yekutadza kushivirira simulator inonzi Istio yekuyedza traffic yakasarudzika kuShopify. Isu taida chishandiso kugadzira tsika topology iyo yaizomiririra chimwe chikamu chesevhisi girafu yedu, yakagadziridzwa zvine simba kuti ienzanisire mamwe mabasa.

Shopify's infrastructure iri pasi pemutoro unorema panguva yekutengesa flash. Panguva imwecheteyo, Shopify inokurudzira vatengesi kuti vabate kutengesa kwakadaro kakawanda. Vatengi vakakura dzimwe nguva vanonyevera nezve kutengeswa kweflash kwakarongwa. Vamwe vanotiitisa tisingazvifungiri chero nguva, masikati kana usiku.

Isu taida yedu yekusimba simulator kuti ienzanisire mafambiro ebasa anofanana nevepamusoro uye mabasa akaremedza Shopify zvivakwa munguva yakapfuura. Chinangwa chikuru chekushandisa mesh sevhisi ndechekuti isu tinoda kuvimbika uye kukanganisa kushivirira padanho retiweki, uye zvakakosha kwatiri kuti mesh yesevhisi igone kubata nemitoro yakambovhiringa masevhisi.

Pamwoyo wekutadza kushivirira simulator inzvimbo yevashandi, inoita sevhisi mesh node. Iyo yevashandi node inogona kugadzirwa statically pakutanga kana dynamically kuburikidza neREST API. Isu tinoshandisa dhizaini dhizaini yevashandi node kugadzira workflows nenzira yekudzoreredza bvunzo.

Heino muenzaniso wemaitiro akadaro:

• Isu tinovhura maseva gumi se bar sevhisi inodzosera mhinduro 200/OK mushure me100 ms.
• Isu tinotangisa vatengi gumi - imwe neimwe inotumira zvikumbiro zana pasekondi bar.
• Ese gumi masekonzi tinobvisa 10 sevha uye kutarisa zvikanganiso 5xx pamutengi.

Pakupera kwekufamba kwebasa, tinoongorora matanda uye metrics uye tarisa kana bvunzo dzakapfuura. Nenzira iyi tinodzidza nezve kuita kwesevhisi mesh yedu uye tinomhanyisa bvunzo yekudzora kuyedza fungidziro dzedu nezve kushivirira kukanganisa.

(Cherechedza: Isu tiri kufunga kuvhura kuvhura iyo Istio kukanganisa kushivirira simulator, asi hatisati tagadzirira kuzviita.)

Istio kukanganisa kushivirira simulator yebasa mesh bhenji

Isu tiri kumisikidza akati wandei anoshanda node ye simulator:

• irs-client-loadgen: 3 replicas inotumira 100 zvikumbiro pasekondi irs-client.
• irs-client: 3 replicas inogamuchira chikumbiro, mirira 100ms uye tumira chikumbiro kune irs-server.
• irs-server: 3 replicas inodzoka 200/OK mushure me100 ms.

Nekugadzirisa uku, tinogona kuyera kufamba kwakagadzikana kwemotokari pakati pe9 endpoints. Sidecars mukati irs-client-loadgen и irs-server gamuchira zvikumbiro zana pasekondi, uye irs-client - 200 (inouya uye inobuda).

Isu tinotevedzera mashandisirwo ezviwanikwa kuburikidza DataDognekuti isu hatina Prometheus cluster.

Mhinduro

Kudzora mapaneru

Kutanga, takaongorora mashandisirwo eCPU.

Linkerd control panel ~ 22 millicore

Istio control panel: ~ 750 millicore

Iyo Istio control panel inoshandisa inenge 35 nguva yakawanda CPU zviwanikwakupfuura Linkerd. Ehe, zvese zvakaiswa nekusarudzika, uye istio-telemetry inoshandisa yakawanda processor zviwanikwa pano (inogona kuvharwa nekudzima mamwe mabasa). Kana tikabvisa chikamu ichi, isu tichiri kuwana anopfuura 100 millicores, kureva 4 zvakapetwakupfuura Linkerd.

Sidecar proxy

Takazoedza kushandisa proxy. Panofanirwa kuve nehukama hwemutsara nehuwandu hwezvikumbiro, asi kune yega yega sidecar pane imwe pamusoro inokanganisa curve.

Linkerd: ~ 100 millicores ye irs-client, ~ 50 millicores ye irs-client-loadgen

Mhedzisiro yacho inotaridzika zvine musoro, nekuti mutengi wemutengi anogashira zvakapetwa kaviri traffic seyeloadgen proxy: pachikumbiro chese chinobuda kubva kuloadgen, mutengi ane imwe inouya uye imwe inobuda.

Istio / Nhume: ~ 155 millicores ye irs-client, ~ 75 millicores ye irs-client-loadgen

Isu tinoona mibairo yakafanana yeIstio sidecars.

Asi kazhinji, Istio/Envoy proxies inodya inenge 50% yakawanda CPU zviwanikwakupfuura Linkerd.

Isu tinoona iyo yakafanana chirongwa padivi reseva:

Linkerd: ~ 50 millicore ye irs-server

Istio/Envoy: ~80 millicores ye irs-server

Padivi reseva, sidecar Istio/Envoy inodya inenge 60% yakawanda CPU zviwanikwakupfuura Linkerd.

mhedziso

Iyo Istio Envoy proxy inoshandisa 50+% yakawanda CPU pane Linkerd pane yedu yakateedzerwa basa rekuita. Iyo Linkerd control panel inoshandisa zvishoma zviwanikwa pane Istio, kunyanya kune epakati zvikamu.

Tichiri kufunga nezvekudzikisa mari idzi. Kana mune mazano, ndapota goverai!

Source: www.habr.com