Benchmark ea tšebeliso ea CPU bakeng sa Istio le Linkerd

Benchmark ea tšebeliso ea CPU bakeng sa Istio le Linkerd

Selelekela

Re kene Shopify o ile a qala ho tsamaisa Istio joalo ka mesh ea litšebeletso. Ha e le hantle, ntho e 'ngoe le e' ngoe e ntle, ntle le ntho e le 'ngoe: e theko e boima.

В litekanyetso tse hatisitsoeng bakeng sa Istio e re:

Ka Istio 1.1, moemeli o sebelisa hoo e ka bang 0,6 vCPUs (virtual cores) ka likopo tse 1000 motsotsoana.

Bakeng sa sebaka sa pele sa mesh ea lits'ebeletso (li-proxies tse 2 ka lehlakoreng le leng la khokahano), re tla ba le li-cores tse 1200 bakeng sa proxy feela, ka sekhahla sa likopo tse milione ka motsotsoana. Ho latela sebali sa theko ea Google, e sebetsa e ka ba $40/khoeli/motheo bakeng sa tlhophiso. n1-standard-64, ke hore, sebaka sena feela se tla re lefella ho feta liranta tse likete tse 50 ka khoeli bakeng sa likopo tse limilione tse 1 motsotsoana.

Ivan Sim (Ivan Sim) papisong le pono service mesh e liehile selemong se fetileng mme e ts'episitse se tšoanang bakeng sa memori le processor, empa ha ea ka ea sebetsa:

Kamoo ho bonahalang kateng, values-istio-test.yaml e tla eketsa likopo tsa CPU ka botebo. Haeba ke entse lipalo tsa ka ka nepo, o hloka li-cores tse ka bang 24 bakeng sa phanele ea taolo le 0,5 CPU bakeng sa proxy ka 'ngoe. Ha ke na ntho e kalo. Ke tla pheta liteko ha ke abeloa lisebelisoa tse ling.

Ke ne ke batla ho iponela hore na ts'ebetso ea Istio e ts'oana joang le mesh e 'ngoe e bulehileng ea lits'ebeletso: Linkerd.

Ho kenya lits'ebeletso tsa mesh

Pele ho tsohle, ke e kentse ka har'a sehlopha SuperGloo:

$ supergloo init
installing supergloo version 0.3.12
using chart uri https://storage.googleapis.com/supergloo-helm/charts/supergloo-0.3.12.tgz
configmap/sidecar-injection-resources created
serviceaccount/supergloo created
serviceaccount/discovery created
serviceaccount/mesh-discovery created
clusterrole.rbac.authorization.k8s.io/discovery created
clusterrole.rbac.authorization.k8s.io/mesh-discovery created
clusterrolebinding.rbac.authorization.k8s.io/supergloo-role-binding created
clusterrolebinding.rbac.authorization.k8s.io/discovery-role-binding created
clusterrolebinding.rbac.authorization.k8s.io/mesh-discovery-role-binding created
deployment.extensions/supergloo created
deployment.extensions/discovery created
deployment.extensions/mesh-discovery created
install successful!

Ke sebelisitse SuperGloo hobane e etsa hore bootstrapping mesh ea lits'ebeletso e be bonolo haholo. Ke ne ke sa tlameha ho etsa ho hongata. Ha re sebelise SuperGloo tlhahiso, empa e loketse mosebetsi o joalo. Ke ile ka tlameha ho sebelisa litaelo tse 'maloa bakeng sa mesh ka 'ngoe ea litšebeletso. Ke sebelisitse lihlopha tse peli ho itšehla thajana - e 'ngoe bakeng sa Istio le Linkerd.

Teko e entsoe ho Google Kubernetes Engine. Ke sebelisitse Kubernetes 1.12.7-gke.7 le letamo la nodes n1-standard-4 ka ho lekanya node ea othomathike (bonyane 4, boholo ba 16).

Eaba ke kenya li-meshes tsa tšebeletso ka bobeli ho tloha molaong oa taelo.

Sehokelo sa Pele:

$ supergloo install linkerd --name linkerd
+---------+--------------+---------+---------------------------+
| INSTALL |     TYPE     | STATUS  |          DETAILS          |
+---------+--------------+---------+---------------------------+
| linkerd | Linkerd Mesh | Pending | enabled: true             |
|         |              |         | version: stable-2.3.0     |
|         |              |         | namespace: linkerd        |
|         |              |         | mtls enabled: true        |
|         |              |         | auto inject enabled: true |
+---------+--------------+---------+---------------------------+

Joale Istio:

$ supergloo install istio --name istio --installation-namespace istio-system --mtls=true --auto-inject=true
+---------+------------+---------+---------------------------+
| INSTALL |    TYPE    | STATUS  |          DETAILS          |
+---------+------------+---------+---------------------------+
| istio   | Istio Mesh | Pending | enabled: true             |
|         |            |         | version: 1.0.6            |
|         |            |         | namespace: istio-system   |
|         |            |         | mtls enabled: true        |
|         |            |         | auto inject enabled: true |
|         |            |         | grafana enabled: true     |
|         |            |         | prometheus enabled: true  |
|         |            |         | jaeger enabled: true      |
+---------+------------+---------+---------------------------+

The crash-loop e ile ea nka metsotso e seng mekae, 'me joale liphanele tsa taolo li tsitsitse.

(Tlhokomeliso: SuperGloo e tšehetsa Istio 1.0.x hajoale feela. Ke ile ka pheta teko ea Istio 1.1.3, empa ha kea hlokomela phapang leha e le efe e hlokomelehang.)

Ho theha Istio Automatic Deployment

Ho etsa hore Istio e kenye Moemeli oa koloi e ka thōko, re sebelisa ente ea koloi e ka thōko - MutatingAdmissionWebhook. Re ke ke ra bua ka eona sehloohong sena. E re ke bolele hore enoa ke molaoli ea hlokomelang phihlello ea li-pods tsohle tse ncha mme a eketsa ka matla koloi e ka thōko le initContainer, e ikarabellang bakeng sa mesebetsi. iptables.

Rona ho Shopify re ile ra ngola molaoli oa rona oa phihlello ho kenya tšebetsong li-sidecars, empa bakeng sa benchmark ena ke sebelisitse molaoli ea tlang le Istio. Molaoli o kenya li-sidecars ka mokhoa o ikhethileng ha ho na le tsela e khuts'oane sebakeng sa mabitso istio-injection: enabled:

$ kubectl label namespace irs-client-dev istio-injection=enabled
namespace/irs-client-dev labeled

$ kubectl label namespace irs-server-dev istio-injection=enabled
namespace/irs-server-dev labeled

Ho theha ho tsamaisoa ha Linkerd ka boiketsetso

Ho theha embedding ea Linkerd sidecar, re sebelisa litlhaloso (ke li kentse ka letsoho ka kubectl edit):

metadata:
  annotations:
    linkerd.io/inject: enabled

$ k edit ns irs-server-dev 
namespace/irs-server-dev edited

$ k get ns irs-server-dev -o yaml
apiVersion: v1
kind: Namespace
metadata:
  annotations:
    linkerd.io/inject: enabled
  name: irs-server-dev
spec:
  finalizers:
  - kubernetes
status:
  phase: Active

Istio Fault Tolerance Simulator

Re thehile simulator ea ho mamella liphoso e bitsoang Istio ho leka sephethephethe se ikhethileng ho Shopify. Re ne re hloka sesebelisoa sa ho theha topology ea tloaelo e neng e tla emela karolo e itseng ea kerafo ea rona ea lits'ebeletso, e hlophisitsoeng hantle hore e be mohlala oa mesebetsi e itseng.

Lisebelisoa tsa Shopify li tlas'a moroalo o boima nakong ea thekiso ea flash. Ka nako e ts'oanang, Shopify e khothalletsa barekisi ho tšoara thekiso e joalo khafetsa. Ka linako tse ling bareki ba kholo ba lemosa ka thekiso e reriloeng ea flash. Ba bang ba re tšoarela tsona re sa lebella ka nako leha e le efe, motšehare kapa bosiu.

Re ne re batla hore simulator ea rona ea boits'oaro e be mohlala oa phallo ea mosebetsi e tsamaellanang le li-topology le meroalo e mengata ea mesebetsi e kentseng thepa ea Shopify nakong e fetileng. Sepheo se seholo sa ho sebelisa mesh ea tšebeletso ke hore re hloka ho tšepahala le mamello ea phoso boemong ba marang-rang, 'me ke habohlokoa ho rona hore mesh ea tšebeletso e sebetsane ka katleho le meroalo e neng e sitisa litšebeletso pele.

Bohareng ba simulator ea ho mamella liphoso ke node ea basebetsi, e sebetsang e le node ea mesh ea litšebeletso. Node ea basebetsi e ka hlophisoa ka mokhoa o tsitsitseng qalong kapa ka matla ka REST API. Re sebelisa tlhophiso e matla ea li-node tsa basebetsi ho theha phallo ea mosebetsi ka mokhoa oa liteko tsa ho khutlisa.

Mona ke mohlala oa ts'ebetso e joalo:

  • Re hlahisa li-server tse 10 joalo ka bar tshebeletso e kgutlisang karabelo 200/OK ka mor'a 100 ms.
  • Re qala bareki ba 10 - e mong le e mong o romela likopo tse 100 motsotsoana ho bar.
  • Metsotsoana e meng le e meng e 10 re tlosa seva e le 'ngoe ebe re beha leihlo liphoso 5xx ho moreki.

Qetellong ea mosebetsi, re hlahloba li-log le metrics ebe re hlahloba hore na tlhahlobo e fetile. Ka tsela ena re ithuta ka ts'ebetso ea mesh ea lits'ebeletso tsa rona mme re etsa tlhahlobo ea khatello ho leka mehopolo ea rona mabapi le mamello ea liphoso.

(Tlhokomeliso: Re ntse re nahana ka ho bula simulator ea Istio fault tolerance, empa ha re so itokisetse ho etsa joalo.)

Istio fault tolerance simulator bakeng sa benchmark ea lits'ebeletso tsa mesh

Re theha li-node tse 'maloa tse sebetsang tsa simulator:

  • irs-client-loadgen: Likopi tse 3 tse romellang likopo tse 100 motsotsoana irs-client.
  • irs-client: Likopi tse 3 tse amohelang kopo, ema 100ms ebe u fetisetsa kopo ho irs-server.
  • irs-server: Likopi tse 3 tse khutlang 200/OK ka mor'a 100 ms.

Ka tlhophiso ena, re ka lekanya phallo e tsitsitseng ea sephethephethe lipakeng tsa li-endpoints tsa 9. Sidecars in irs-client-loadgen и irs-server amohela likopo tse 100 motsotsoana, le irs-client - 200 (tse kenang le tse tsoang).

Re lekola tšebeliso ea lisebelisoa ka DataDoghobane ha re na sehlopha sa Prometheus.

Liphetho

Liphanele tsa taolo

Taba ea pele, re ile ra hlahloba tšebeliso ea CPU.

Benchmark ea tšebeliso ea CPU bakeng sa Istio le Linkerd
Phanele ea taolo ea Linkerd ~ 22 millicore

Benchmark ea tšebeliso ea CPU bakeng sa Istio le Linkerd
Phanele ea taolo ea Istio: ~ 750 millicore

Phanele ea taolo ea Istio e sebelisa hoo e ka bang Lisebelisoa tsa CPU ka makhetlo a 35ho feta Linkerd. Ha e le hantle, ntho e 'ngoe le e' ngoe e kenngoa ka ho sa feleng, 'me istio-telemetry e sebelisa lisebelisoa tse ngata tsa processor mona (e ka sitisoa ka ho thibela mesebetsi e meng). Haeba re tlosa karolo ena, re ntse re fumana ho feta 100 millicores, ke hore 4 makhetlo a mangho feta Linkerd.

Moemeli oa Sidecar

Ka mor'a moo, re ile ra leka ho sebelisa proxy. Ho lokela ho ba le kamano e haufi le palo ea likopo, empa bakeng sa koloi e 'ngoe le e 'ngoe e ka thōko ho na le karolo e ka holimo e amang curve.

Benchmark ea tšebeliso ea CPU bakeng sa Istio le Linkerd
Linkerd: ~ 100 millicores bakeng sa irs-client, ~ 50 millicores bakeng sa irs-client-loadgen

Liphetho li shebahala li utloahala, hobane moemeli oa bareki o fumana sephethephethe se habeli ho feta proxy ea loadgen: bakeng sa kopo e 'ngoe le e' ngoe e tsoang ho loadgen, moreki o na le e 'ngoe e kenang le e tsoang.

Benchmark ea tšebeliso ea CPU bakeng sa Istio le Linkerd
Istio/Envoy: ~ 155 millicores bakeng sa irs-client, ~ 75 millicores bakeng sa irs-client-loadgen

Re bona liphetho tse tšoanang bakeng sa li-sidecars tsa Istio.

Empa ka kakaretso, li-proxies tsa Istio / Envoy lia ja hoo e ka bang 50% ho feta mehloli ea CPUho feta Linkerd.

Re bona morero o tšoanang lehlakoreng la seva:

Benchmark ea tšebeliso ea CPU bakeng sa Istio le Linkerd
Linkerd: ~ 50 millicore bakeng sa irs-server

Benchmark ea tšebeliso ea CPU bakeng sa Istio le Linkerd
Istio/Envoy: ~ 80 millicore bakeng sa irs-server

Ka lehlakoreng la seva, sidecar Istio/Envoy ea ja hoo e ka bang 60% ho feta mehloli ea CPUho feta Linkerd.

fihlela qeto e

Moemeli oa Istio Envoy o sebelisa 50+% CPU ho feta Linkerd mosebetsing oa rona o qapiloeng. Sehlopha sa taolo sa Linkerd se sebelisa lisebelisoa tse fokolang haholo ho feta Istio, haholo-holo bakeng sa likarolo tsa mantlha.

Re ntse re nahana ka mokhoa oa ho fokotsa litšenyehelo tsena. Haeba u na le maikutlo, ka kopo arolelana!

Source: www.habr.com

Eketsa ka tlhaloso