🥇Mazano mapfumbamwe ekuvandudza Kubernetes kuita

Mhoroi mose! Zita rangu ndinonzi Oleg Sidorenkov, ndinoshanda kuDomClick semukuru weboka rezvivakwa. Isu tanga tichishandisa Kubik mukugadzira kweanopfuura makore matatu, uye panguva ino takasangana nezvakawanda zvakasiyana zvinonakidza nguva nazvo. Nhasi ini ndichakuudza sei, nenzira kwayo, iwe unogona kudzvanya zvakanyanya kuita kunze kwevanilla Kubernetes yeboka rako. Ready steady go!

Imi mose munoziva kwazvo kuti Kubernetes is scalable open source system yemidziyo orchestration; zvakanaka, kana mashanu mabhinari anoshanda mashiripiti nekugadzirisa kutenderera kwehupenyu hwema microservices ako munzvimbo ye server. Pamusoro pezvo, chishandiso chinoshanduka-shanduka chinogona kuunganidzwa seLego kuitira kugadziridza kwakanyanya kwemabasa akasiyana.

Uye zvese zvinoita kunge zvakanaka: kanda maseva musumbu sehuni mubhokisi remoto, uye iwe hauzoziva chero kusuwa. Asi kana uri wemhoteredzo, uchafunga, kuti: “Ndingaita sei kuti moto urambe uchipfuta ndokuchengeta sango?” Mune mamwe mazwi, maitiro ekutsvaga nzira dzekuvandudza zvivakwa uye kuderedza mutengo.

1. Tarisa timu uye zviwanikwa zvekushandisa

Imwe yenzvimbo dzakajairika, asi nzira dzinoshanda kuunza zvikumbiro / miganhu. Govanisa maapplication nemazita, uye nzvimbo dzemazita nezvikwata zvebudiriro. Usati watumirwa, isa kukosha kwekushandisa kwekushandisa processor nguva, ndangariro, uye ephemeral kuchengetedza.

resources:
   requests:
     memory: 2Gi
     cpu: 250m
   limits:
     memory: 4Gi
     cpu: 500m

Kuburikidza neruzivo, takasvika pamhedziso: haufanirwe kuwedzera zvikumbiro kubva pamiganhu nekanopfuura kaviri. Huwandu hwemasumbu hunoverengerwa zvichibva pane zvikumbiro, uye kana iwe ukapa zvikumbiro mutsauko mune zviwanikwa, semuenzaniso, 5-10 nguva, zvino fungidzira zvichaitika kune yako node kana izere nemapods uye kamwe kamwe inogamuchira mutoro. Hapana chakanaka. Pazvishoma, throttling, uye pamwero mukuru, unozoonekana kune mushandi uye wowana cyclic mutoro pane asara node mushure mekunge mapodhi atanga kufamba.

Mukuwedzera, nerubatsiro limitranges Pakutanga, unogona kuseta zviwanikwa zvemudziyo - zvishoma, zvakanyanya uye default:

➜  ~ kubectl describe limitranges --namespace ops
Name:       limit-range
Namespace:  ops
Type        Resource           Min   Max   Default Request  Default Limit  Max Limit/Request Ratio
----        --------           ---   ---   ---------------  -------------  -----------------------
Container   cpu                50m   10    100m             100m           2
Container   ephemeral-storage  12Mi  8Gi   128Mi            4Gi            -
Container   memory             64Mi  40Gi  128Mi            128Mi          2

Usakanganwa kudzikamisa namespace zviwanikwa kuitira kuti timu imwe isakwanise kutora zviwanikwa zve cluster:

➜  ~ kubectl describe resourcequotas --namespace ops
Name:                   resource-quota
Namespace:              ops
Resource                Used          Hard
--------                ----          ----
limits.cpu              77250m        80
limits.memory           124814367488  150Gi
pods                    31            45
requests.cpu            53850m        80
requests.memory         75613234944   150Gi
services                26            50
services.loadbalancers  0             0
services.nodeports      0             0

Sezvinoonekwa kubva pane tsananguro resourcequotas, kana timu ye ops ichida kuendesa mapodhi anodya imwe gumi cpu, mugadziri haatenderi izvi uye anokanda chikanganiso:

Error creating: pods "nginx-proxy-9967d8d78-nh4fs" is forbidden: exceeded quota: resource-quota, requested: limits.cpu=5,requests.cpu=5, used: limits.cpu=77250m,requests.cpu=53850m, limited: limits.cpu=10,requests.cpu=10

Kuti ugadzirise dambudziko rakadaro, unogona kunyora chishandiso, semuenzaniso, senge izvi, inokwanisa kuchengetedza uye kuzvipira mamiriro ekuraira zviwanikwa.

2. Sarudza iyo yakakwana yekuchengetedza faira

Pano ndinoda kubata pamusoro penyaya yezvinoramba zviripo uye disk subsystem yeKubernetes worker nodes. Ndinovimba kuti hapana anoshandisa "Cube" pane HDD mukugadzira, asi dzimwe nguva SSD yenguva dzose haichakwani. Takasangana nedambudziko apo matanda aive achiuraya dhisiki nekuda kweI / O mashandiro, uye hapana akawanda mhinduro:

Shandisa yakakwirira-inoshanda SSDs kana chinja kuNVMe (kana iwe uchigadzirisa yako hardware).
Deredza kutema matanda.
Ita "smart" kuenzanisa kwemapodhi anobata dhisiki chibharo (podAntiAffinity).

Iyo skrini iri pamusoro inoratidza zvinoitika pasi pe nginx-ingress-controller kune dhisiki kana access_logs matanda akagoneswa (~ 12 zviuru matanda/sec). Mamiriro aya, hongu, anogona kutungamira mukushatiswa kwese maapplication pane ino node.

Kana iri PV, maiwe, handina kumboedza zvese maonero Mavhoriyamu Anoramba Aripo. Shandisa yakanakisa sarudzo inokodzera iwe. Nhoroondo, zvakaitika munyika yedu kuti chikamu chiduku chebasa chinoda mavhoriyamu eRWX, uye kare kare vakatanga kushandisa NFS kuchengetedza basa iri. Cheap uye ... zvakakwana. Ehe, iye neni takadya shit - akuropafadze, asi takadzidza kuigadzirisa, uye musoro wangu haucharwadzi. Uye kana zvichibvira, enda kune S3 chinhu chekuchengetedza.

3. Unganidza mifananidzo yakagadziridzwa

Zvakanakisa kushandisa midziyo-yakagadziridzwa mifananidzo kuitira kuti Kubernetes ikwanise kuitora nekukurumidza uye nekuzviita zvakanyanya.

Optimized zvinoreva kuti mifananidzo:

ine application imwe chete kana kuita basa rimwe chete;
diki muhukuru, nekuti mifananidzo mikuru inopfuudzwa yakaipisisa pane network;
vane hutano uye kugadzirira magumo anobvumira Kubernetes kuita chiito muchiitiko chekuderera;
shandisa midziyo-inoshamwaridzika masisitimu ekushandisa (seAlpine kana CoreOS), ayo anonyanya kupikisa zvikanganiso zvekugadzirisa;
shandisa akawanda-matanho anovaka kuitira kuti iwe unokwanisa chete kutumira akaunganidzwa maapplication uye kwete anoperekedza masosi.

Kune akawanda maturusi uye masevhisi anokutendera iwe kuti utarise uye nekunatsiridza mifananidzo pane nhunzi. Izvo zvakakosha kuti ugare uchivachengeta kusvika parizvino uye kuongororwa kuchengetedzwa. Somugumisiro unowana:

Yakaderedzwa network load pane yese cluster.
Kuderedza koni yekutanga nguva.
Diki saizi yako yese Docker registry.

4. Shandisa DNS cache

Kana tikataura nezve mitoro yakakwira, saka hupenyu hwakashata pasina kugadzirisa iyo cluster's DNS system. Pane imwe nguva, vagadziri veKubernetes vakatsigira yavo kube-dns mhinduro. Yakaitwawo pano, asi software iyi haina kunyanya kurongeka uye haina kuburitsa kuita kwaidiwa, kunyangwe zvaiita senge riri nyore basa. Ipapo coredns yakaonekwa, iyo yatakachinjira kwairi uye tisina kusuwa; yakazove iyo default DNS sevhisi muK8s. Pane imwe nguva, takakura kusvika 40 zviuru rps kuDNS system, uye iyi mhinduro zvakare haina kukwana. Asi, nerombo rakanaka, Nodelocaldns akabuda, aka node local cache, aka NodeLocal DNSCache.

Nei tichishandisa izvi? Pane tsikidzi muLinux kernel iyo, kana akawanda mafoni kuburikidza nekupikisa NAT pamusoro peUDP, anotungamira kune mamiriro emujaho ekupinda mumatafura econtrack, uye chikamu chetraffic kuburikidza neNAT chinorasika (rwendo rwese kuburikidza neSevhisi ndiNAT). Nodelocaldns inogadzirisa dambudziko iri nekubvisa NAT uye kusimudzira chinongedzo kuTCP kuenda kumusoro kweDNS, pamwe nemunharaunda caching kumusoro kweDNS mibvunzo (kusanganisira pfupi pfupi 5-yechipiri isina cache).

5. Skerai mapodhi akachinjika uye akachinjika otomatiki

Unogona here kutaura nechivimbo kuti ma microservices ako akagadzirira kuwedzera kaviri kusvika katatu mumutoro? Nzira yekugovera sei zviwanikwa kune zvikumbiro zvako? Kuchengeta mapodhi akati wandei achimhanya kupfuura mutoro webasa kungave kushoma, asi kuvachengeta kumashure kunomhanyisa njodzi yenguva yekudzikira kubva pakuwedzera kamwe kamwe kwetraffic kuenda kubasa. Masevhisi akadai Horizontal Pod Autoscaler и Vertical Pod Autoscaler.

VPA inokutendera iwe kusimudza otomatiki zvikumbiro / miganhu yemidziyo yako mupodhi zvinoenderana nekushandiswa chaiko. Zvingabatsira sei? Kana iwe uine mapods asingakwanise kuyerwa akachinjika nekuda kwechimwe chikonzero (izvo zvisina kuvimbika zvachose), saka unogona kuedza kuisa shanduko kune zviwanikwa zvayo kuVPA. Chimiro chayo inzira yekurudziro yakavakirwa panhoroondo uye yazvino dhata kubva kumetric-server, saka kana iwe usingade kushandura otomatiki zvikumbiro / miganhu, unogona kungotarisa zvakakurudzirwa zviwanikwa zvemidziyo yako uye nekugadzirisa zvigadziriso kuchengetedza CPU uye. ndangariro musumbu.

Mufananidzo wakatorwa kubva https://levelup.gitconnected.com/kubernetes-autoscaling-101-cluster-autoscaler-horizontal-pod-autoscaler-and-vertical-pod-2a441d9ad231

Iyo scheduler muKubernetes inogara yakavakirwa pane zvikumbiro. Chero kukosha kwaunoisa ipapo, iyo scheduler ichatsvaga yakakodzera node yakavakirwa pairi. Iyo miganho tsika inodiwa kuti cubelet inzwisise nguva yekutsikitsira kana kuuraya pod. Uye sezvo iyo chete yakakosha parameter iko kukosha kwekukumbira, VPA ichashanda nayo. Pese paunoyera application wakamira, unotsanangura izvo zvikumbiro zvinofanirwa kunge zviri. Chii chichaitika kumiganhu ipapo? Iyi parameter ichayerwawo zvakaenzanirana.

Semuenzaniso, heano akajairwa pod marongero:

resources:
   requests:
     memory: 250Mi
     cpu: 200m
   limits:
     memory: 500Mi
     cpu: 350m

Injini yekurudziro inoona kuti application yako inoda 300m CPU uye 500Mi kuti imhanye nemazvo. Iwe uchawana zvinotevera marongero:

resources:
   requests:
     memory: 500Mi
     cpu: 300m
   limits:
     memory: 1000Mi
     cpu: 525m

Sezvambotaurwa pamusoro, uku kuyera kuenzanirana kunoenderana nezvikumbiro / miganhu reshiyo mune manifest:

CPU: 200m → 300m: reshiyo 1:1.75;
Chiyeuchidzo: 250Mi → 500Mi: reshiyo 1:2.

Nezvekutaura HPA, ipapo iyo nzira yekushanda inonyanya kujeka. Metrics akadai seCPU uye ndangariro zvakavharirwa, uye kana avhareji yezvese replicas ikadarika chikumbaridzo, chikumbiro chinoyerwa ne +1 sub kudzamara kukosha kwadonha pasi pechikumbaridzo kana kusvika huwandu hwepamusoro hwezvinyorwa hwasvika.

Mufananidzo wakatorwa kubva https://levelup.gitconnected.com/kubernetes-autoscaling-101-cluster-autoscaler-horizontal-pod-autoscaler-and-vertical-pod-2a441d9ad231

Kuwedzera kune akajairwa metrics seCPU uye ndangariro, unogona kuseta zvikumbaridzo pane yako tsika metrics kubva kuPrometheus uye kushanda navo kana iwe uchifunga kuti ndiyo inonyanya kuratidzwa yenguva yekuyera application yako. Kana iyo application ikangodzikama pazasi peyakatarwa metric pachikumbaridzo, HPA ichatanga kuyera mapodhi pasi kusvika pahuwandu hwehuwandu hwema replicas kana kusvika mutoro wasangana nechikumbaridzo chakataurwa.

6. Usakanganwa nezveNode Affinity uye Pod Affinity

Haasi ese ma node anomhanya pane imwechete hardware, uye haasi ese mapodhi anoda kumhanya compute-yakanyanya maapplication. Kubernetes inobvumidza iwe kuseta iyo nyanzvi yemanodhi uye pods uchishandisa Node Affinity и Pod Affinity.

Kana uine nodes dzakakodzera compute-intensive operations, saka nokuda kwekushanda kwakanyanya zviri nani kusungira zvikumbiro kune node dzinoenderana. Kuita kushandiswa uku nodeSelector ine node label.

Ngatitii une nodes mbiri: imwe ine CPUType=HIGHFREQ nenhamba huru yemakori anotsanya, imwe ine MemoryType=HIGHMEMORY zvimwe ndangariro uye kukurumidza kuita. Nzira iri nyore ndeyekugovera kutumirwa kune node HIGHFREQnekuwedzera kune chikamu spec chisarudzi ichi:

…
nodeSelector:
	CPUType: HIGHFREQ

Imwe nzira inodhura uye yakananga yekuita izvi ndeyekushandisa nodeAffinity mumunda affinity chikamu spec. Pane zvingasarudzwa zviviri:

requiredDuringSchedulingIgnoredDuringExecution: kugadzika kwakaoma (mugadziri achaendesa mapodhi chete pane chaiwo node (uye hapana kumwe kumwe));
preferredDuringSchedulingIgnoredDuringExecution: kurongeka kwakapfava (mugadziri achaedza kuendesa kune dzakanangana node, uye kana izvo zvikatadza, ichaedza kuendesa kune inotevera inowanikwa node).

Iwe unogona kutsanangura chaiyo syntax yekugadzirisa zvinyorwa zvenode, senge In, NotIn, Exists, DoesNotExist, Gt kana Lt. Nekudaro, rangarira kuti nzira dzakaomarara mune refu runyorwa rwemazita dzinononoka kuita sarudzo mumamiriro ezvinhu akaoma. Nemamwe mashoko, zvichengetedze zviri nyore.

Sezvambotaurwa pamusoro, Kubernetes inobvumidza iwe kuseta kuwirirana kwemapodhi azvino. Kureva, iwe unogona kuve nechokwadi chekuti mamwe mapodhi anoshanda pamwe chete nemamwe mapodhi munzvimbo imwechete inowanikwa (inoenderana nemakore) kana node.

В podAffinity minda affinity chikamu spec minda yakafanana inowanikwa seyakaitika nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution и preferredDuringSchedulingIgnoredDuringExecution. Musiyano chete ndewekuti matchExpressions ichasunga mapods kune node iri kutomhanyisa podhi ine label iyoyo.

Kubernetes inopawo munda podAntiAffinity, iyo, pane zvinopesana, haisungiri pod kune node ine mapodhi chaiwo.

Nezvekutaura nodeAffinity Mazano akafanana anogona kupiwa: edza kuchengeta mitemo iri nyore uye inonzwisisika, usaedza kuwedzeredza iyo pod yakatarwa neyakaoma yemitemo. Zviri nyore kwazvo kugadzira mutemo usingaenderane nemamiriro echikwata, kugadzira mutoro usingakoshi pamugadziri uye kuderedza kuita kwese.

7. Taints & Tolerances

Pane imwe nzira yekugadzirisa iyo scheduler. Kana uine boka guru rine mazana emanodhi uye zviuru zve microservices, saka zvakaoma zvikuru kusabvumira mamwe mapodhi kuti agamuchirwe pane dzimwe node.

Iyo nzira yekusvibisa-kurambidza mitemo-inobatsira neizvi. Semuyenzaniso, mune mamwe mamiriro ezvinhu unogona kurambidza mamwe nodes kubva kumhanya pods. Kuisa tsvina kune imwe node iwe unofanirwa kushandisa sarudzo taint mu kubectl. Taura kiyi uye kukosha uye wozosvibisa senge NoSchedule kana NoExecute:

$ kubectl taint nodes node10 node-role.kubernetes.io/ingress=true:NoSchedule

Izvo zvakakoshawo kuziva kuti iyo taint mechanism inotsigira matatu makuru mhedzisiro: NoSchedule, NoExecute и PreferNoSchedule.

NoSchedule zvinoreva kuti parizvino hapazove neinopindirana yekupinda mune iyo pod yakatarwa tolerations, haizokwanisi kuiswa pane node (mumuenzaniso uyu node10).
PreferNoSchedule - yakareruka shanduro NoSchedule. Muchiitiko ichi, mugadziri achaedza kusagovera mapodhi asina chinopindirana tolerations pa node, asi ichi hachisi muganhu wakaoma. Kana pasina zviwanikwa musumbu, ipapo mapodhi anozotanga kuendesa pane ino node.
NoExecute - iyi mhedzisiro inokonzeresa kuburitswa kwepods iyo isina inopindirana yekupinda tolerations.

Sezvineiwo, hunhu uhu hunogona kukanzurwa uchishandisa nzira yekushivirira. Izvi zviri nyore kana paine "inorambidzwa" node uye iwe unongoda kuisa masevhisi ezvivakwa pairi. Kuzviita sei? Bvumira mapodhi chete ayo ane kushivirira kwakakodzera.

Hezvino izvo iyo pod yakatarwa yaizotaridzika senge:

spec:
   tolerations:
     - key: "node-role.kubernetes.io/ingress"
        operator: "Equal"
        value: "true"
        effect: "NoSchedule"

Izvi hazvireve kuti iyo inotevera redeploy ichawira pane ino node, iyi haisi iyo Node Affinity mechanism uye nodeSelector. Asi nekubatanidza akati wandei maficha, unogona kuwana inoshanduka-shanduka marongero.

8. Set Podhi Deployment Priority

Nekuda kwekuti une mapods akapihwa node hazvireve kuti ese mapods anofanirwa kubatwa neakaenzana. Semuenzaniso, ungangoda kutumira mamwe mapodhi pamberi pevamwe.

Kubernetes inopa nzira dzakasiyana dzekugadzirisa Pod Kukosha uye Preemption. Kurongeka kunosanganisira zvikamu zvakati: chinhu PriorityClass uye tsanangudzo dzemunda priorityClassName mune iyo pod yakatarwa. Ngatitarisei muenzaniso:

apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: high-priority
value: 99999
globalDefault: false
description: "This priority class should be used for very important pods only"

Isu tinosika PriorityClass, zvipe zita, tsananguro uye kukosha. Iyo yepamusoro value, kunowedzera kukoshesa. Hukoshi hunogona kuita chero 32-bit integer isingasviki kana kuenzana ne1. Hushe hwepamusoro hwakachengeterwa mishoni-yakakosha system mapodhi ayo haagone kuregererwa. Displacement inongoitika chete kana podhi yepamusoro-soro isina nzvimbo yekutenderera, ipapo mamwe mapodhi kubva kune imwe node achabviswa. Kana iyi michina yakakuomera iwe, unogona kuwedzera sarudzo preemptionPolicy: Never, uye ipapo hapazovi nekuregererwa, iyo pod ichamira pakutanga mumutsetse uye kumirira kuti mugadziri awane zviwanikwa zvepachena.

Zvadaro, tinogadzira podhi yatinoratidza zita racho priorityClassName:

apiVersion: v1
kind: Pod
metadata:
  name: static-web
  labels:
    role: myrole
 spec:
  containers:
    - name: web
      image: nginx
      ports:
        - name: web
          containerPort: 80
          protocol: TCP
  priorityClassName: high-priority

Iwe unogona kugadzira akawanda ekutanga makirasi sezvaunoda, kunyangwe zvichikurudzirwa kuti usatakurwe neizvi (taura, zvigumire kune yakaderera, yepakati uye yepamusoro pekutanga).

Nekudaro, kana zvichidikanwa, unogona kuwedzera kugona kwekuisa masevhisi akakosha senge nginx-ingress-controller, coredns, nezvimwe.

9. Gadzirisa ETCD cluster

ETCD inogona kunzi uropi hwesumbu rese. Izvo zvakakosha kuchengetedza kushanda kweiyo dhatabhesi padanho repamusoro, sezvo kukurumidza kwekushanda muCube kunoenderana nazvo. Iyo yakaringana mwero, uye panguva imwecheteyo, mhinduro yakanaka ingave yekuchengeta iyo ETCD cluster pane master node kuitira kuve nekunonoka kunonoka kune kube-apiserver. Kana iwe usingakwanisi kuita izvi, zvino isa ETCD pedyo sezvinobvira, neine bandwidth yakanaka pakati pevatori vechikamu. Uyewo teerera kuti ingani nodes kubva kuETCD inogona kudonha pasina kukuvadza kune sumbu

Ramba uchifunga kuti kuwedzera zvakanyanya nhamba yenhengo musumbu kunogona kuwedzera kukanganisa kushivirira pamutengo wekuita, zvese zvinofanirwa kuve zvine mwero.

Kana tikataura nezve kumisikidza sevhisi, pane mashoma ekurudziro:

Iva nemidziyo yakanaka, zvichienderana nehukuru hwesumbu (iwe unogona kuverenga pano).
Tweak mashoma ma paramita kana iwe wakaparadzira sumbu pakati pema DCs kana network yako nemadhisiki anosiya zvakawanda zvingadiwa (unogona kuverenga pano).

mhedziso

Ichi chinyorwa chinotsanangura mapoinzi ayo chikwata chedu chinoedza kutevedzera. Iyi haisi nhanho-ne-nhanho tsananguro yezviito, asi sarudzo dzinogona kubatsira pakugadzirisa cluster pamusoro. Zviri pachena kuti sumbu rimwe nerimwe rakasiyana nenzira yaro, uye zvigadziriso zvekugadzirisa zvinogona kusiyana zvakanyanya, saka zvingave zvinonakidza kuwana mhinduro yako pamusoro pekutarisa kwaunoita Kubernetes cluster uye kuti unovandudza sei kuita kwayo. Govera ruzivo rwako mumashoko, zvichave zvinonakidza kuziva.

Source: www.habr.com

Zvipfumbamwe Kubernetes Performance Tips