🥇Ama-Liveness probe e-Kubernetes angaba yingozi

Qaphela. transl.: Unjiniyela oholayo ovela e-Zalando, u-Henning Jacobs, uye waqaphela ngokuphindaphindiwe izinkinga phakathi kwabasebenzisi be-Kubernetes ekuqondeni injongo ye-liveness (kanye nokulungela) ama-probe kanye nokusetshenziswa kwawo okulungile. Ngakho-ke, waqoqa imicabango yakhe kuleli nothi elinamandla, elizoba yingxenye yemibhalo ye-K8s.

Ukuhlolwa kwezempilo, kwaziwa e-Kubernetes ngokuthi i-liveness probes (okungukuthi, ngokwezwi nezwi, “ukuhlolwa kokusebenza” - cishe. transl.), kungaba yingozi impela. Ngincoma ukuthi uzigweme uma kungenzeka: okuhlukile kuphela lapho kudingekile ngempela futhi wazi ngokugcwele okucacisiwe nemiphumela yokusetshenziswa kwazo. Lolu shicilelo luzokhuluma ngokuhlolwa kokuphila kanye nokulungela, futhi luzokutshela nokuthi yiziphi izimo kufanelekile futhi akufanele uwasebenzise.

Uzakwethu uSandor usanda kwabelana ku-Twitter ngamaphutha avame kakhulu ahlangana nawo, okuhlanganisa nalawo ahlobene nokusetshenziswa kokuhlolwa kokulungela/okuphilayo:

Ilungiselelwe ngokungalungile livenessProbe ingashubisa izimo zomthwalo omkhulu (ukuvalwa kwebhola leqhwa + isiqukathi esingaba side/isikhathi sokuqalisa isicelo) futhi kuholele kweminye imiphumela engemihle njengokwehla kokuncika. (bhekafuthi isihloko sami sakamuva mayelana nokukhawulela inani lezicelo kwinhlanganisela ye-K3s+ACME). Kubi nakakhulu uma i-liveness probe ihlanganiswa nokuhlolwa kwezempilo, okuyisizindalwazi sangaphandle: ukwehluleka kwe-DB eyodwa kuzoqalisa kabusha zonke iziqukathi zakho!

Umlayezo ojwayelekile "Ungasebenzisi ama-liveness probe" kulokhu akusizi kakhulu, ngakho-ke ake sibheke ukuthi ukuhlolwa kokulungela nokuphila kahle kungokwabani.

Qaphela: Iningi lokuhlola okungezansi lifakwe ekuqaleni kumadokhumenti kanjiniyela wangaphakathi we-Zalando.

Ukulungela kanye namasheke e-Liveness

I-Kubernetes inikeza izindlela ezimbili ezibalulekile ezibizwa ngokuthi i-liveness probes kanye ne-probes yokulungela. Ngezinye izikhathi benza isenzo esithile—njengokuthumela isicelo se-HTTP, ukuvula uxhumano lwe-TCP, noma ukwenza umyalo esitsheni—ukuqinisekisa ukuthi uhlelo lokusebenza lusebenza njengoba kulindelekile.

Kubernetes isebenzisa ukulungela probeukuqonda lapho isitsha sesilungele ukwamukela ithrafikhi. I-pod ibhekwa njengelungele ukusetshenziswa uma zonke iziqukathi zayo sezilungile. Okunye ukusetshenziswa kwalo mshini ukulawula ukuthi imaphi ama-pods asetshenziswa njenge-backends yezinsizakalo ze-Kubernetes (ikakhulukazi i-Ingress).

Impilo iyaphenya siza u-Kubernetes aqonde ukuthi sekuyisikhathi sokuqala kabusha isiqukathi. Isibonelo, isheke elinjalo likuvumela ukuthi ubambe i-deadlock lapho uhlelo lokusebenza lubhajwa endaweni eyodwa. Ukuqalisa kabusha isiqukathi kulesi simo kusiza ukususa uhlelo lokusebenza phansi naphezu kwamaphutha, kodwa kungase futhi kuholele ekuhlulekeni kokwehla (bona ngezansi).

Uma uzama ukufaka isibuyekezo sohlelo lokusebenza esihluleka ukuhlolwa kokuphila/ukulungela, ukukhishwa kwayo kuzomiswa njengoba u-Kubernetes elinde isimo. Ready kusuka kuwo wonke ama-pods.

Isibonelo:

Nasi isibonelo sophenyo lokulungela ukuhlola indlela /health nge-HTTP enezilungiselelo ezizenzakalelayo (isikhathi: Imizuzwana eyi-10, isikhathi siphelile: isekhondi elingu-1, umkhawulo wempumelelo: 1, umkhawulo wokuhluleka: 3):

# часть общего описания deployment'а/стека
podTemplate:
  spec:
    containers:
    - name: my-container
      # ...
      readinessProbe:
        httpGet:
          path: /health
          port: 8080

Izincomo

Ngama-microservices ane-HTTP endpoint (REST, njll.) njalo chaza i-probe yokulungela, ehlola ukuthi isicelo (i-pod) sikulungele ukwamukela ithrafikhi.
Qiniseka ukuthi i-probe yokulungela ihlanganisa ukutholakala kwembobo yangempela yeseva yewebhu:
- usebenzisa izimbobo ngezinjongo zokulawula, ezibizwa ngokuthi "admin" noma "management" (isibonelo, 9090), ukuze readinessProbe, qiniseka ukuthi isiphetho sibuya KULUNGILE kuphela uma imbobo ye-HTTP eyinhloko (njenge-8080) isilungele ukwamukela ithrafikhi*;
  *Ngiyazi okungenani ngecala elilodwa eZalando lapho kungenzeki khona lokhu, okungukuthi. readinessProbe Ngihlole imbobo "yokuphatha", kodwa iseva ngokwayo ayizange iqale ukusebenza ngenxa yezinkinga zokulayisha inqolobane.
- ukunamathisela i-probe yokulungela echwebeni elihlukile kungaholela eqinisweni lokuthi ukugcwala ngokweqile echwebeni elikhulu ngeke kubonakale ekuhlolweni kwezempilo (okungukuthi, i-pool pool kuseva igcwele, kodwa isheke lezempilo lisabonisa ukuthi konke kulungile. ).
Qiniseka ukuthi i-probe yokulungela inika amandla ukuqaliswa/ukufuduka kwesizindalwazi;
- Indlela elula yokufeza lokhu ukuxhumana neseva ye-HTTP kuphela ngemuva kokuthi ukuqalisa sekuqediwe (isibonelo, ukuthutha isizindalwazi sisuka ku- I-Flyway njalo njalo.); okungukuthi, esikhundleni sokushintsha isimo sokuhlolwa kwezempilo, ungaqali nje iseva yewebhu kuze kuqedwe ukuthuthwa kwedathabheyisi*.
  * Ungaphinda usebenzise ukufuduka kwesizindalwazi kusuka ezitsheni ze-init ngaphandle kwe-pod. Ngisengumlandeli wezinhlelo zokusebenza ezizimele, okungukuthi, lezo lapho isitsha sohlelo lokusebenza sazi kanjani ukuletha isizindalwazi esimweni esifiselekayo ngaphandle kokuxhumana kwangaphandle.
Sebenzisa httpGet ukuhlola ukulungela ngezindawo ezijwayelekile zokuhlola impilo (isibonelo, /health).
Qonda amapharamitha wokuhlola azenzakalelayo (interval: 10s, timeout: 1s, successThreshold: 1, failureThreshold: 3):
- izinketho ezizenzakalelayo zisho ukuthi i-pod izoba ayilungile cishe ngemva kwemizuzwana engu-30 (ukuhlola oku-3 okuhlulekile).
Sebenzisa imbobo ehlukile "ye-admin" noma "yokuphatha" uma isitaki sobuchwepheshe (isb. i-Java/Spring) siyivumela, ukuhlukanisa ukuphathwa kwezempilo namamethrikhi kuthrafikhi evamile:
- kodwa ungakhohlwa ngephuzu lesi-2.
Uma kunesidingo, i-probe yokulungela ingasetshenziswa ukufudumeza/ukulayisha inqolobane bese ubuyisela ikhodi yesimo engu-503 kuze kube yilapho isiqukathi sifudumala:
- Futhi ngincoma ukuthi ufunde isheke elisha startupProbe, kuvele kunguqulo 1.16 (sibhale ngakho ngesiRashiya lapha - cishe. transl.).

Amapulangwe

Unganciki kokuncika kwangaphandle (njengezindawo zokugcina idatha) lapho wenza ukuhlolwa kokulungela/ukuba nempilo - lokhu kungaholela ekuhlulekeni okukhulu:
- Njengesibonelo, ake sithathe insizakalo ye-REST ebabazekayo enama-pods ayi-10 kuye ngesizindalwazi esisodwa se-Postgres: lapho isheke lincike ekuxhumekeni okusebenzayo ku-DB, wonke ama-pod ayi-10 angase ahluleke uma kukhona ukubambezeleka ohlangothini lwenethiwekhi/DB - ngokuvamile konke kugcina kubi kakhulu kunalokho obekungakwenza;
- Sicela uqaphele ukuthi i-Spring Data ihlola uxhumo lwesizindalwazi ngokuzenzakalelayo*;
  * Lokhu ukuziphatha okuzenzakalelayo kwe-Spring Data Redis (okungenani kwakuyisikhathi sokugcina ngibheka), okwaholela ekuhlulekeni "kwenhlekelele": lapho i-Redis ingatholakali isikhathi esifushane, wonke ama-pods "aphahlazeka".
- “okwangaphandle” ngalo mqondo kungasho futhi amanye ama-pods ohlelo olufanayo, okungukuthi, isheke akufanele lincike esimweni samanye ama-pods eqoqo elifanayo ukuvimbela ukuphahlazeka okudabukisayo:
  - imiphumela ingase yehluke kuzinhlelo zokusebenza ezinesimo sokusatshalaliswa (isibonelo, ukugcinwa kwesikhashana kwenkumbulo kuma-pods).
Ungasebenzisi i-liveness probe kuma-pods (okuhlukile yizimo lapho kudingekile ngempela futhi wazi ngokugcwele imininingwane nemiphumela yokusetshenziswa kwazo):
- I-liveness probe ingasiza ekubuyiseleni iziqukathi ezilengayo, kodwa njengoba ukwazi ukulawula ngokugcwele uhlelo lwakho lokusebenza, izinto ezifana nezinqubo ezilengayo kanye ne-deadlock akufanele neze zenzeke: enye indlela engcono kakhulu ukuphahlazeka ngamabomu uhlelo bese ulubuyisela esimweni sangaphambilini esizinzile;
- i-liveness probe ehlulekile izobangela isiqukathi ukuthi siqale kabusha, ngaleyo ndlela kwenze kube nzima imiphumela yamaphutha ahlobene nokuqalisa: ukuqalisa kabusha isiqukathi kuzoholela esimeni sokuphumula (okungenani ngesikhathi sokuqalisa isicelo, isho imizuzwana engu-30+), kubangele amaphutha amasha, ukwandisa umthwalo kwezinye iziqukathi nokwandisa amathuba okuhluleka kwawo, njll.;
- amasheke e-liveness ahlanganiswe nokuncika kwangaphandle kuyinhlanganisela embi kakhulu engenzeka, ukwehluleka okusongelayo okusongelayo: ukubambezeleka okuncane ohlangothini lwesizindalwazi kuzoholela ekuqaliseni kabusha kwazo zonke iziqukathi zakho!
Imingcele yokuphila nokulungela ukuhlola kumele kwehluke:
- ungasebenzisa i-liveness probe ngesheke lezempilo elifanayo, kodwa umkhawulo wokuphendula ophezulu (failureThreshold), isibonelo, yabela isimo ayilungile ngemuva kwemizamo emi-3 futhi ucabange ukuthi i-liveness probe yehlulekile ngemuva kwemizamo eyi-10;
Ungasebenzisi ama-exec checks, njengoba zihlotshaniswa nezinkinga ezaziwayo eziholela ekubukeni kwezinqubo ze-zombie:
- imininingwane: bona isethulo ngochwepheshe be-Datadog.

Isifingqo

Sebenzisa ama-probes ukuze uthole ukuthi i-pod isilungele ukwamukela ithrafikhi.
Sebenzisa ama-liveness probe kuphela lapho edingeka ngempela.
Ukusetshenziswa okungalungile kokuhlolwa kokulungela/ukuphila kungaholela ekutholakaleni okuncishisiwe kanye nokwehluleka kwe-cascading.

Izinto ezengeziwe esihlokweni

I-Kubernetes docs: Lungiselela I-Liveness and Readiness Probes;
I-Kubernetes Liveness and Readiness Probes ibuyekezwe kabusha: Ungakugwema Kanjani Ukuzidubula Kolunye Unyawo;
I-NRE Labs Outage Post-Mortem (uphinde akhulume nge-livenessProbe).

Buyekeza inombolo 1 kusukela ngo-2019-09-29

Mayelana neziqukathi ze-init zokuthutha kwesizindalwazi: Umbhalo waphansi wengeziwe.

U-EJ wangikhumbuza mayelana ne-PDB: enye yezinkinga ngokuhlolwa kokuphila ukuntuleka kokuxhumana phakathi kwama-pods. Kubernetes has Izabelomali Zokuphazamiseka KwePod (PDB) ukukhawulela inani lokuhluleka okwenzeka ngesikhathi esisodwa uhlelo lokusebenza, nokho ukuhlola akunaki i-PDB. Ngokufanelekile, singatshela ama-K8 ukuthi "Qala kabusha i-pod eyodwa uma ukuhlolwa kwayo kwehluleka, kodwa ungawaqali kabusha wonke ukuze ugweme ukwenza izinto zibe zimbi nakakhulu."

UBryan wayibeka kahle: “Sebenzisa ukuhlola okuphilayo uma wazi kahle ukuthi yini into engcono kakhulu ongayenza ukubulala isicelo"(futhi, ungathatheki).

Buyekeza inombolo 2 kusukela ngo-2019-09-29

Mayelana nokufunda imibhalo ngaphambi kokuyisebenzisa: Ngidale isicelo esihambisanayo (isicelo sesici) ukwengeza imibhalo mayelana nama-liveness probe.

I-PS evela kumhumushi

Funda futhi kubhulogi yethu:

Source: www.habr.com

Ama-Liveness probe e-Kubernetes angaba yingozi