CPU noj benchmark rau Istio thiab Linkerd

CPU noj benchmark rau Istio thiab Linkerd

Taw qhia

Peb nyob hauv Khw Tshaj Tawm pib xa Istio ua ib qho kev pabcuam mesh. Hauv paus ntsiab lus, txhua yam zoo, tsuas yog rau ib yam: nws kim heev.

Π’ luam tawm cov qauv ntsuas rau Istio nws hais tias:

Nrog Istio 1.1, lub npe siv kwv yees li 0,6 vCPUs (virtual cores) ib 1000 thov ib ob.

Rau thawj cheeb tsam hauv kev pabcuam mesh (2 tus neeg sawv cev ntawm txhua sab ntawm kev sib txuas), peb yuav muaj 1200 cores rau tus neeg sawv cev, ntawm tus nqi ntawm ib lab thov ib ob. Raws li Google tus nqi laij lej, nws ua haujlwm tau kwv yees li $ 40 / hli / core rau kev teeb tsa n1-standard-64, uas yog, thaj av no ib leeg yuav raug nqi ntau dua 50 txhiab daus las ib hlis rau 1 lab thov ib ob.

Ivan Sim (Ivan Sim) pom kev sib piv kev pabcuam mesh qeeb xyoo tas los thiab tau cog lus zoo ib yam rau kev nco thiab processor, tab sis nws tsis ua haujlwm:

Thaj, qhov tseem ceeb-istio-test.yaml yuav ua rau kev thov CPU loj dua. Yog tias kuv tau ua tiav kuv cov lej kom raug, koj xav tau kwv yees li 24 CPU cores rau cov tswj vaj huam sib luag thiab 0,5 CPU rau txhua tus npe. Kuv tsis muaj ntau npaum li ntawd. Kuv yuav rov sim dua thaum muaj peev txheej ntxiv rau kuv.

Kuv xav pom kuv tus kheej li cas zoo li Istio qhov kev ua tau zoo rau lwm qhov qhib qhov kev pabcuam mesh: Linkerd.

Service mesh installation

Ua ntej tshaj plaws, kuv ntsia nws hauv ib pawg SuperGloo:

$ supergloo init
installing supergloo version 0.3.12
using chart uri https://storage.googleapis.com/supergloo-helm/charts/supergloo-0.3.12.tgz
configmap/sidecar-injection-resources created
serviceaccount/supergloo created
serviceaccount/discovery created
serviceaccount/mesh-discovery created
clusterrole.rbac.authorization.k8s.io/discovery created
clusterrole.rbac.authorization.k8s.io/mesh-discovery created
clusterrolebinding.rbac.authorization.k8s.io/supergloo-role-binding created
clusterrolebinding.rbac.authorization.k8s.io/discovery-role-binding created
clusterrolebinding.rbac.authorization.k8s.io/mesh-discovery-role-binding created
deployment.extensions/supergloo created
deployment.extensions/discovery created
deployment.extensions/mesh-discovery created
install successful!

Kuv siv SuperGloo vim nws ua rau bootstrapping cov kev pabcuam mesh yooj yim dua. Kuv tsis tau ua ntau. Peb tsis siv SuperGloo hauv kev tsim khoom, tab sis nws yog qhov zoo tagnrho rau txoj haujlwm no. Kuv yuav tsum tau siv ob peb lo lus txib rau txhua qhov kev pabcuam mesh. Kuv siv ob pawg rau kev sib cais - ib qho rau Istio thiab Linkerd.

Qhov kev sim tau ua nyob rau hauv Google Kubernetes Cav. Kuv siv Kubernetes 1.12.7-gke.7 thiab ib lub pas dej ntawm nodes n1-standard-4 nrog tsis siv neeg node scaling (yam tsawg kawg 4, siab tshaj 16).

Tom qab ntawd kuv tau nruab ob qhov kev pabcuam meshes los ntawm kab hais kom ua.

Thawj Linkerd:

$ supergloo install linkerd --name linkerd
+---------+--------------+---------+---------------------------+
| INSTALL |     TYPE     | STATUS  |          DETAILS          |
+---------+--------------+---------+---------------------------+
| linkerd | Linkerd Mesh | Pending | enabled: true             |
|         |              |         | version: stable-2.3.0     |
|         |              |         | namespace: linkerd        |
|         |              |         | mtls enabled: true        |
|         |              |         | auto inject enabled: true |
+---------+--------------+---------+---------------------------+

Tom qab ntawd Istio:

$ supergloo install istio --name istio --installation-namespace istio-system --mtls=true --auto-inject=true
+---------+------------+---------+---------------------------+
| INSTALL |    TYPE    | STATUS  |          DETAILS          |
+---------+------------+---------+---------------------------+
| istio   | Istio Mesh | Pending | enabled: true             |
|         |            |         | version: 1.0.6            |
|         |            |         | namespace: istio-system   |
|         |            |         | mtls enabled: true        |
|         |            |         | auto inject enabled: true |
|         |            |         | grafana enabled: true     |
|         |            |         | prometheus enabled: true  |
|         |            |         | jaeger enabled: true      |
+---------+------------+---------+---------------------------+

Lub voj sib tsoo siv ob peb feeb, thiab tom qab ntawd cov tswj vaj huam sib luag tau ruaj khov.

(Faj seeb: SuperGloo tsuas yog txhawb Istio 1.0.x rau tam sim no. Kuv rov ua qhov kev sim nrog Istio 1.1.3, tab sis tsis pom qhov txawv txav.)

Teeb tsa Istio Automatic Deployment

Yuav kom Istio nruab lub sidecar Envoy, peb siv lub sidecar injector βˆ’ MutatingAdmissionWebhook. Peb yuav tsis tham txog nws hauv kab lus no. Cia kuv cia li hais tias qhov no yog ib tug maub los uas saib xyuas kev nkag tau ntawm tag nrho cov tshiab pods thiab dynamically ntxiv ib tug sidecar thiab initContainer, uas yog lub luag hauj lwm rau cov hauj lwm. iptables.

Peb ntawm Shopify tau sau peb tus kheej kev nkag mus tswj los siv cov tsheb sib tw, tab sis rau qhov ntsuas no kuv siv tus maub los uas los nrog Istio. Tus maub los txhaj cov sidecars los ntawm lub neej ntawd thaum muaj qhov shortcut hauv lub npe istio-injection: enabled:

$ kubectl label namespace irs-client-dev istio-injection=enabled
namespace/irs-client-dev labeled

$ kubectl label namespace irs-server-dev istio-injection=enabled
namespace/irs-server-dev labeled

Teeb tsa kev xa tawm tsis siv neeg Linkerd

Txhawm rau teeb tsa Linkerd sidecar embedding, peb siv annotations (Kuv ntxiv lawv manually ntawm kubectl edit):

metadata:
  annotations:
    linkerd.io/inject: enabled

$ k edit ns irs-server-dev 
namespace/irs-server-dev edited

$ k get ns irs-server-dev -o yaml
apiVersion: v1
kind: Namespace
metadata:
  annotations:
    linkerd.io/inject: enabled
  name: irs-server-dev
spec:
  finalizers:
  - kubernetes
status:
  phase: Active

Istio Fault Tolerance Simulator

Peb tau tsim lub simulator ua txhaum cai hu ua Istio los sim nrog cov tsheb sib txawv rau Shopify. Peb xav tau ib lub cuab yeej los tsim ib qho kev cai topology uas yuav sawv cev rau ib feem ntawm peb cov kev pabcuam graph, dynamically configured rau cov qauv tshwj xeeb workloads.

Shopify's infrastructure yog nyob rau hauv hnyav load thaum lub sij hawm flash muag. Tib lub sijhawm, Shopify xav kom cov neeg muag khoom tuav xws li kev muag khoom ntau zaus. Cov neeg siv khoom loj qee zaum ceeb toom txog kev npaj muag muag. Lwm tus coj lawv npaj txhij txog rau peb txhua lub sijhawm nruab hnub lossis hmo ntuj.

Peb xav kom peb lub tshuab simulator resilience ua qauv kev ua haujlwm uas haum rau cov khoom lag luam thiab cov khoom ua haujlwm uas tau dhau los ntawm Shopify cov txheej txheem yav dhau los. Lub hom phiaj tseem ceeb ntawm kev siv mesh kev pabcuam yog tias peb xav tau kev ntseeg siab thiab kev ua txhaum cai ntawm lub network theem, thiab nws yog ib qho tseem ceeb rau peb tias cov kev pabcuam mesh tau txais txiaj ntsig zoo nrog cov kev pabcuam uas yav dhau los cuam tshuam.

Nyob rau hauv lub plawv ntawm lub txim txhaum simulator yog ib tug neeg ua hauj lwm node, uas ua raws li ib tug kev pab cuam mesh node. Cov neeg ua haujlwm node tuaj yeem teeb tsa tau zoo thaum pib lossis hloov pauv ntawm REST API. Peb siv dynamic configuration ntawm cov neeg ua hauj lwm nodes los tsim workflows nyob rau hauv daim ntawv ntawm regression xeem.

Nov yog ib qho piv txwv ntawm cov txheej txheem zoo li no:

  • Peb tso tawm 10 servers li bar kev pabcuam uas rov qab teb 200/OK tom qab 100 ms.
  • Peb tso tawm 10 tus neeg siv khoom - txhua tus xa 100 thov ib ob rau bar.
  • Txhua 10 vib nas this peb tshem tawm 1 server thiab saib xyuas qhov yuam kev 5xx ntawm tus neeg siv khoom.

Thaum kawg ntawm kev ua haujlwm, peb tshuaj xyuas cov cav thiab ntsuas thiab xyuas seb qhov kev xeem dhau los. Qhov no peb kawm txog kev ua tau zoo ntawm peb cov kev pabcuam mesh thiab khiav qhov kev xeem rov qab los sim peb cov kev xav txog kev ua txhaum cai.

(Ceeb Toom: Peb tab tom xav txog kev qhib qhov chaw rau Istio txhaum simulator, tab sis tseem tsis tau npaj ua li ntawd.)

Istio txhaum kam rau ua simulator rau kev pab cuam mesh benchmark

Peb teeb tsa ntau qhov chaw ua haujlwm ntawm lub simulator:

  • irs-client-loadgen: 3 replicas uas xa 100 thov ib ob ib irs-client.
  • irs-client: 3 replicas uas tau txais qhov kev thov, tos 100ms thiab xa daim ntawv thov mus rau irs-server.
  • irs-server: 3 replicas uas rov qab los 200/OK tom qab 100 ms.

Nrog rau qhov kev teeb tsa no, peb tuaj yeem ntsuas qhov kev khiav tsheb ruaj khov ntawm 9 qhov kawg. Sidecars hauv irs-client-loadgen ΠΈ irs-server tau txais 100 thov ib ob, thiab irs-client - 200 (ntxiv thiab tawm).

Peb taug qab kev siv cov ntaub ntawv los ntawm DataDogvim peb tsis muaj Prometheus pawg.

Π Π΅Π·ΡƒΠ»ΡŒΡ‚Π°Ρ‚Ρ‹

Tswj vaj huam sib luag

Ua ntej, peb tshuaj xyuas CPU noj.

CPU noj benchmark rau Istio thiab Linkerd
Linkerd tswj vaj huam sib luag ~ 22 millicore

CPU noj benchmark rau Istio thiab Linkerd
Istio tswj vaj huam sib luag: ~ 750 millicore

Istio tswj vaj huam sib luag siv kwv yees li 35 npaug ntau dua cov peev txheej CPUtshaj Linkerd. Tau kawg, txhua yam yog ntsia los ntawm lub neej ntawd, thiab istio-telemetry siv ntau cov khoom siv ntawm no (nws tuaj yeem ua tsis taus los ntawm kev ua haujlwm tsis zoo). Yog tias peb tshem tawm cov khoom no, peb tseem tau txais ntau dua 100 millicores, uas yog 4 zaug ntxivtshaj Linkerd.

Sidecar npe

Peb tom qab ntawd sim siv lub npe. Yuav tsum muaj kev sib raug zoo nrog tus naj npawb ntawm kev thov, tab sis rau txhua lub tsheb muaj qee qhov nyiaj siv ua haujlwm uas cuam tshuam rau qhov nkhaus.

CPU noj benchmark rau Istio thiab Linkerd
Linkerd: ~ 100 millicores rau irs-neeg siv khoom, ~ 50 millicores rau irs-neeg-loadgen

Cov txiaj ntsig tau zoo, vim tias tus neeg siv khoom npe tau txais ob zaug ntau dua li cov neeg siv khoom thauj khoom: rau txhua qhov kev thov tawm los ntawm loadgen, tus neeg siv khoom muaj ib qho tuaj thiab ib qho tawm.

CPU noj benchmark rau Istio thiab Linkerd
Istio/Envoy: ~ 155 millicores rau irs-neeg siv khoom, ~ 75 millicores rau irs-neeg-loadgen

Peb pom cov txiaj ntsig zoo sib xws rau Istio sidecars.

Tab sis feem ntau, Istio/Envoy proxies haus kwv yees li 50% ntau cov peev txheej CPUtshaj Linkerd.

Peb pom tib lub tswv yim ntawm sab server:

CPU noj benchmark rau Istio thiab Linkerd
Linkerd: ~ 50 millicore rau irs-server

CPU noj benchmark rau Istio thiab Linkerd
Istio/Envoy: ~ 80 millicore rau irs-server

Nyob rau sab server, sidecar Istio/Envoy noj kwv yees li 60% ntau cov peev txheej CPUtshaj Linkerd.

xaus

Istio Envoy proxy siv 50+% ntau dua CPU dua li Linkerd ntawm peb cov haujlwm simulated. Linkerd tswj vaj huam sib luag siv nyiaj tsawg dua Istio, tshwj xeeb tshaj yog rau cov khoom tseem ceeb.

Peb tseem tab tom xav txog yuav ua li cas txo cov nqi no. Yog koj muaj tswv yim thov qhia!

Tau qhov twg los: www.hab.com

Ntxiv ib saib