🥇 Binciken rayuwa a cikin Kubernetes na iya zama haɗari

Lura. fassara: Injiniyan jagora daga Zalando, Henning Jacobs, ya lura da matsaloli akai-akai tsakanin masu amfani da Kubernetes wajen fahimtar manufar rayuwa (da shirye-shiryen) bincike da kuma amfani da su daidai. Saboda haka, ya tattara tunaninsa a cikin wannan bayanin mai ƙarfi, wanda a ƙarshe zai zama wani ɓangare na takaddun K8s.

Binciken lafiya, wanda aka sani a Kubernetes kamar bincike na rayuwa (watau, a zahiri, “gwajin iya aiki” - kusan fassarar.), na iya zama haɗari sosai. Ina ba da shawarar nisantar su idan zai yiwu: keɓance kawai shine lokacin da suke da mahimmanci da gaske kuma kuna da cikakkiyar masaniya game da ƙayyadaddun bayanai da sakamakon amfaninsu. Wannan ɗaba'ar za ta yi magana game da raye-raye da shirye-shiryen shirye-shiryen, kuma za ta gaya muku a waɗanne lokuta ya cancanci kuma bai kamata ku yi amfani da su ba.

Abokin aiki na Sandor kwanan nan ya raba akan Twitter kurakuran da ya fi yawan cin karo da su, gami da waɗanda ke da alaƙa da yin amfani da bincike na shirye-shirye/rayuwa:

Ba daidai ba saitin livenessProbe na iya tsananta yanayi mai nauyi (kashewar ƙwallon dusar ƙanƙara + mai yuwuwar kwantena mai tsayi / lokacin fara aikace-aikacen) kuma yana haifar da wasu mummunan sakamako kamar raguwar dogaro. (duba kuma labarina na kwanan nan game da iyakance adadin buƙatun a cikin haɗin K3s+ACME). Har ma ya fi muni idan aka haɗu da binciken rayuwa tare da duba lafiyar jiki, wanda shine bayanan waje: gazawar DB guda ɗaya zata sake kunna duk kwantena!

Saƙon gaba ɗaya "Kada ku yi amfani da bincike na rayuwa" a cikin wannan yanayin ba ya taimaka sosai, don haka bari mu dubi abin da shirye-shiryen shirye-shiryen da kuma rayuwa ke dubawa.

Lura: Yawancin gwajin da ke ƙasa an haɗa su ne a cikin takaddun haɓaka na ciki na Zalando.

Shirye-shirye da Binciken Rayuwa

Kubernetes yana ba da mahimman hanyoyi guda biyu da ake kira bincike na rayuwa da shirye-shiryen bincike. Suna yin wasu ayyuka lokaci-lokaci-kamar aika buƙatun HTTP, buɗe haɗin TCP, ko aiwatar da umarni a cikin akwati-don tabbatar da cewa aikace-aikacen yana aiki kamar yadda aka zata.

Kubernetes yana amfani shirye-shiryen bincikedon gane lokacin da kwantena ya shirya don karɓar zirga-zirga. Ana ɗaukar kwas ɗin a shirye don amfani idan duk kwantenansa suna shirye. Ɗayan amfani da wannan hanyar ita ce sarrafa waɗanne kwas ɗin da aka yi amfani da su azaman abin baya don ayyukan Kubernetes (kuma musamman Ingress).

Binciken rayuwa taimaka Kubernetes fahimtar lokacin da lokaci yayi don sake kunna akwati. Misali, irin wannan cak ɗin yana ba ku damar kutse lokacin da aikace-aikacen ya makale a wuri guda. Sake kunna akwati a cikin wannan jihar yana taimakawa cire aikace-aikacen daga ƙasa duk da kurakurai, amma kuma yana iya haifar da gazawar cascading (duba ƙasa).

Idan kayi ƙoƙarin tura sabuntawar aikace-aikacen da ya gaza bincikar rayuwa / shirye-shiryen, za a dakatar da fitar da shi yayin da Kubernetes ke jiran matsayin Ready daga dukkan bangarorin.

Alal misali:

Ga misali na binciken shirye-shiryen bincika hanya /health ta hanyar HTTP tare da saitunan tsoho (lokaci lokaci: 10 seconds, timeout: 1 seconds, bakin nasara: 1, gazawar kofa: 3):

# часть общего описания deployment'а/стека
podTemplate:
  spec:
    containers:
    - name: my-container
      # ...
      readinessProbe:
        httpGet:
          path: /health
          port: 8080

shawarwari

Don microservices tare da ƙarshen HTTP (REST, da sauransu) ko da yaushe ayyana wani shiri bincike, wanda ke bincika ko aikace-aikacen (pod) yana shirye don karɓar zirga-zirga.
Tabbatar da binciken shirye-shiryen yana rufe samin ainihin tashar sabar gidan yanar gizo:
- amfani da tashoshin jiragen ruwa don dalilai na gudanarwa, wanda ake kira "admin" ko "management" (misali, 9090), don readinessProbe, Tabbatar cewa ƙarshen ƙarshen kawai ya dawo OK idan tashar HTTP ta farko (kamar 8080) tana shirye don karɓar zirga-zirga *;
  *Ina sane da akalla shari'a daya a Zalando inda hakan bai faru ba, watau. readinessProbe Na duba tashar "management", amma uwar garken kanta bai fara aiki ba saboda matsalolin loda cache.
- haɗa bincike na shirye-shiryen zuwa wani tashar jiragen ruwa na daban na iya haifar da gaskiyar cewa nauyin da ke kan babban tashar jiragen ruwa ba zai bayyana ba a cikin binciken lafiya (wato, tafkin zaren da ke kan uwar garke ya cika, amma har yanzu binciken lafiyar ya nuna cewa komai yana da kyau). ).
Tabbatar cewa Binciken shirye-shiryen yana ba da damar farawa / ƙaura;
- Hanya mafi sauƙi don cimma wannan ita ce tuntuɓar uwar garken HTTP kawai bayan an gama farawa (misali, ƙaura daga bayanan bayanai Flyway da sauransu.); wato, maimakon canza yanayin duba lafiyar jiki, kawai kar a fara sabar gidan yanar gizo har sai an kammala ƙauran bayanan bayanai*.
  * Hakanan zaka iya gudanar da ƙaura na bayanan bayanai daga kwantena init a wajen kwas ɗin. Har yanzu ina sha'awar aikace-aikacen da ke da kai, wato, waɗanda kwandon aikace-aikacen ya san yadda ake shigar da ma'ajin bayanai zuwa yanayin da ake so ba tare da haɗin kai na waje ba.
Amfani httpGet don duba shirye-shiryen ta hanyar wuraren duba lafiyar lafiya (misali, /health).
Fahimtar sigogin rajistan tsoho (interval: 10s, timeout: 1s, successThreshold: 1, failureThreshold: 3):
- Zaɓuɓɓukan tsoho suna nufin kwaf ɗin zai zama ba shiri bayan kamar dakika 30 (bayan duba lafiyar lafiya 3).
Yi amfani da keɓantaccen tashar jiragen ruwa don "admin" ko "Gudanarwa" idan tarin fasaha (misali Java/Spring) ya ba shi damar, don raba kula da lafiya da ma'auni daga zirga-zirga na yau da kullun:
- amma kar a manta da batu na 2.
Idan ya cancanta, ana iya amfani da binciken shirye-shiryen don dumama / loda cache da dawo da lambar matsayi 503 har sai kwandon ya dumama:
- Ina kuma ba da shawarar ku karanta sabon rajistan startupProbe, ya bayyana a cikin sigar 1.16 (mun rubuta game da shi cikin harshen Rashanci a nan - kimanin. fassara).

Caveats

Kar a dogara ga abin dogaro na waje (kamar ɗakunan ajiya na bayanai) lokacin gudanar da gwaje-gwajen shirye-shirye / rayuwa - wannan na iya haifar da gazawar lalacewa:
- A matsayin misali, bari mu ɗauki sabis na REST mai ban sha'awa tare da kwasfan fayiloli 10 dangane da bayanan Postgres ɗaya: lokacin da rajistan ya dogara da haɗin aiki zuwa DB, duk kwas ɗin 10 na iya gazawa idan akwai jinkiri a gefen hanyar sadarwa / DB - yawanci shi. duk ya ƙare muni fiye da yadda zai iya;
- Da fatan za a lura cewa Bayanan bazara yana bincika haɗin bayanan ta tsohuwa*;
  * Wannan shine tsohuwar halayen Spring Data Redis (aƙalla shine karo na ƙarshe da na bincika), wanda ya haifar da gazawar "mummunan bala'i": lokacin da Redis bai samu na ɗan lokaci ba, duk kwafs ɗin "sun rushe".
- “na waje” a wannan ma’ana na iya nufin sauran kwas ɗin aikace-aikacen iri ɗaya, wato, a zahiri bai kamata cak ɗin ya dogara da yanayin sauran kwas ɗin gungu ɗaya ba don hana faɗuwar haɗari:
  - Sakamakon zai iya bambanta don aikace-aikace tare da yanayin rarraba (misali, caching a cikin ƙwaƙwalwar ajiya a cikin kwas ɗin).
Kada a yi amfani da bincike mai rai don kwasfa (banda lokuta ne lokacin da suke da mahimmanci kuma kuna da cikakkiyar masaniya game da ƙayyadaddun bayanai da sakamakon amfaninsu):
- Binciken rayuwa zai iya taimakawa wajen dawo da kwantena da aka rataye, amma tun da kuna da cikakken iko akan aikace-aikacenku, abubuwa kamar tsarin ratayewa da kulle-kulle bai kamata su faru ba: mafi kyawun madadin shine da gangan a lalata aikace-aikacen kuma a dawo da shi cikin kwanciyar hankali na baya;
- Binciken rayuwa da ya gaza zai sa akwati ya sake farawa, ta haka zai iya haifar da illa ga kurakurai masu alaƙa: sake kunna akwati zai haifar da raguwar lokaci (aƙalla na tsawon lokacin fara aikace-aikacen, faɗi 30-m seconds), haifar da sabbin kurakurai. , ƙara nauyi akan sauran kwantena da ƙara yiwuwar gazawar su, da dai sauransu;
- Binciken rayuwa da aka haɗa tare da dogaro na waje shine mafi munin haɗin haɗin da zai yuwu, yana barazanar gazawa: ɗan jinkiri a gefen bayanan zai haifar da sake farawa da duk kwantena!
Siga na rayuwa da shirye-shiryen cak dole ne ya bambanta:
- Kuna iya amfani da binciken rayuwa tare da duba lafiyar lafiya iri ɗaya, amma mafi girman matakin mayar da martani (failureThreshold), misali, sanya matsayi ba shiri bayan 3 ƙoƙari kuma la'akari da cewa binciken rayuwa ya gaza bayan 10 ƙoƙari;
Kar a yi amfani da cak, tunda suna da alaƙa da sanannun matsalolin da ke haifar da bayyanar hanyoyin aljanu:
- bayani: gani gabatarwa daga kwararrun Datadog.

Takaitaccen

Yi amfani da shirye-shiryen bincike don tantance lokacin da kwaf ɗin ya shirya don karɓar zirga-zirga.
Yi amfani da binciken rayuwa kawai lokacin da ake buƙatar su da gaske.
Amfani mara kyau na binciken shirye-shirye/rayuwa na iya haifar da raguwar samuwa da gazawar lalacewa.

Ƙarin kayan aiki akan batun

Kubernetes docs: Sanya Rayuwar Rayuwa da Binciken Shiryewa;
Kubernetes Rayuwa da Shirye-shiryen Bincika An Sake Ziyara: Yadda Za A Guji Harba Kanku A Wani Kafar;
NRE Labs Outage Post-Mutuwa (ya kuma yi magana game da livenessProbe).

Sabunta lamba 1 daga 2019-09-29

Game da kwantena init don ƙaura na bayanai: An ƙara bayanin ƙafa.

EJ ya tuna min game da PDB: daya daga cikin matsalolin tare da duba rayuwa shine rashin daidaituwa tsakanin kwasfa. Kubernetes yana da Kasafin Kuɗi na Rushewa (PDB) don iyakance adadin gazawar lokaci guda aikace-aikacen zai iya fuskanta, duk da haka cak ɗin ba sa la'akari da PDB. Da kyau, za mu iya gaya wa K8s su "Sake kunna kwafsa ɗaya idan gwajinsa ya gaza, amma kar a sake kunna su duka don guje wa yin muni."

Bryan ya sanya shi daidai: "Yi amfani da binciken rayuwa lokacin da kuka san ainihin menene mafi kyawun abin yi shine kashe aikace-aikacen"(sake, kar a tafi da shi).

Sabunta lamba 2 daga 2019-09-29

Game da karanta takardun kafin amfani: Na ƙirƙiri buƙatar da ta dace (bukatar neman aiki) don ƙara takaddun bayanai game da binciken rayuwa.

PS daga mai fassara

Karanta kuma a kan shafinmu:

source: www.habr.com

Binciken rayuwa a cikin Kubernetes na iya zama haɗari