Iingcebiso kunye namaqhinga kaKubernetes: iimpawu zokuvalwa kakuhle kwi-NGINX kunye ne-PHP-FPM

Imeko eqhelekileyo xa kuphunyezwa i-CI / CD kwi-Kubernetes: isicelo kufuneka sikwazi ukungamkeli izicelo ezintsha zabathengi ngaphambi kokuyeka ngokupheleleyo, kwaye okona kubaluleke kakhulu, ukugqiba ngempumelelo esele ikhona.

Ukuthobela le meko kukuvumela ukuba ufezekise ixesha lokunciphisa i-zero ngexesha lokuthunyelwa. Nangona kunjalo, naxa usebenzisa iinyanda ezithandwa kakhulu (ezifana ne-NGINX kunye ne-PHP-FPM), unokuhlangabezana nobunzima obuya kukhokelela kukwanda kweempazamo ngokuthunyelwa ngakunye...

Ithiyori. Uphila njani iPod

Sele sipapashe ngokweenkcukacha malunga nomjikelo wobomi bepod Oku kubhaliwe. Kumxholo wesihloko esiqwalaselwayo, sinomdla koku kulandelayo: okwangoku xa i-pod ingena kurhulumente Ukupheliswa, izicelo ezitsha ziyayeka ukuthunyelwa kuyo (ipod isuswe kuluhlu lweendawo zokugqibela zenkonzo). Ke, ukuphepha ixesha lokuphumla ngexesha lokuthunyelwa, kwanele ukuba sisombulule ingxaki yokumisa isicelo ngokuchanekileyo.

Kufuneka ukhumbule kwakhona ukuba ixesha loxolelo elimiselweyo li Imizuzwana engama-30: emva koku, i-pod iya kupheliswa kwaye isicelo kufuneka sibe nexesha lokucubungula zonke izicelo ngaphambi kweli xesha. Qaphela:: nangona nasiphi na isicelo esithatha ngaphezulu kwemizuzwana eyi-5-10 sele siyingxaki, kwaye ukuvala ngobubele akusayi kuphinda kuncede...

Ukuqonda ngcono okwenzekayo xa i-pod iphela, jonga nje lo mzobo ulandelayo:

A1, B1 - Ukufumana utshintsho malunga nobume beziko
A2 - Ukuhamba SIGTERM
B2 - Ukususa i-pod kwiindawo zokugqibela
B3 - Ukufumana utshintsho (uluhlu lwesiphelo lutshintshile)
B4 - Hlaziya imithetho ye-iptables

Nceda uqaphele: ukucima i-pod yesiphelo kunye nokuthumela i-SIGTERM akwenzeki ngokulandelelana, kodwa ngokuhambelana. Kwaye ngenxa yokuba i-Ingress ayifumani ngokukhawuleza uluhlu oluhlaziyiweyo lwee-Endpoints, izicelo ezintsha ezivela kubaxhasi ziya kuthunyelwa kwi-pod, eya kubangela impazamo ye-500 ngexesha lokupheliswa kwe-pod. (ukufumana imathiriyeli eneenkcukacha ngalo mba, thina iguqulelwe). Le ngxaki kufuneka isonjululwe ngezi ndlela zilandelayo:

• Thumela uQhagamshelwano: vala kwiiheader zempendulo (ukuba oku kuchaphazela isicelo seHTTP).
• Ukuba akunakwenzeka ukwenza utshintsho kwikhowudi, ke inqaku elilandelayo lichaza isisombululo esiza kukuvumela ukuba uqhube izicelo kude kube sekupheleni kwexesha lobabalo.

Ithiyori. Indlela i-NGINX kunye ne-PHP-FPM ephelisa ngayo iinkqubo zabo

NGINX

Masiqale nge-NGINX, kuba yonke into ibonakala ngakumbi okanye ingaphantsi kwayo. Ukuntywila kwithiyori, sifunda ukuba i-NGINX inenkqubo enye kunye "nabasebenzi" abaliqela - ezi ziinkqubo zabantwana eziqhuba izicelo zabaxhasi. Ukhetho olufanelekileyo lunikiwe: usebenzisa umyalelo nginx -s <SIGNAL> Cima iinkqubo nokuba kukuvalwa okukhawulezileyo okanye imo yokuvala ngobubele. Ngokucacileyo, lukhetho lokugqibela olunomdla kuthi.

Emva koko yonke into ilula: kufuneka udibanise i-preStop-hook Umyalelo oza kuthumela uphawu oluthandekayo lokuvala. Oku kunokwenziwa kuSaziso, kwibhloko yesikhongozeli:

       lifecycle:
          preStop:
            exec:
              command:
              - /usr/sbin/nginx
              - -s
              - quit

Ngoku, xa i-pod ivaliwe, siya kubona oku kulandelayo kwiilogi zesikhongozeli se-NGINX:

2018/01/25 13:58:31 [notice] 1#1: signal 3 (SIGQUIT) received, shutting down
2018/01/25 13:58:31 [notice] 11#11: gracefully shutting down

Kwaye oku kuya kuthetha into esiyifunayo: i-NGINX ilindele ukuba izicelo zizalise, kwaye emva koko ibulala inkqubo. Nangona kunjalo, ngezantsi siza kuqwalasela ingxaki eqhelekileyo ngenxa yokuba, kunye nomyalelo nginx -s quit inkqubo iphela ngendlela engeyiyo.

Kwaye ngeli nqanaba senziwe nge-NGINX: ubuncinci ukusuka kwizigodo unokuqonda ukuba yonke into isebenza njengoko kufanelekile.

Yintoni ingxaki nge-PHP-FPM? Ijongana njani nokuvalwa okuthandekayo? Masiyiqonde.

I-PHP-FPM

Kwimeko ye-PHP-FPM, kukho ulwazi oluncinci oluncinci. Ukuba ugxininisa incwadi esemthethweni ngokwe-PHP-FPM, iya kuthi imiqondiso elandelayo ye-POSIX yamkelwe:

1. SIGINT, SIGTERM - ukuvala ngokukhawuleza;
2. SIGQUIT - Ukuvalwa okuthandekayo (into esiyifunayo).

Imiqondiso eseleyo ayidingeki kulo msebenzi, ngoko siya kulushiya uhlalutyo lwabo. Ukuphelisa inkqubo ngokuchanekileyo, kuya kufuneka ubhale le hook ilandelayo ye-preStop:

        lifecycle:
          preStop:
            exec:
              command:
              - /bin/kill
              - -SIGQUIT
              - "1"

Ukuqala nje kokubona, oku kuphela okufunekayo ukwenza ukuvalwa okuthandekayo kuzo zombini izikhongozeli. Nangona kunjalo, umsebenzi unzima kunokuba ubonakala. Apha ngezantsi kukho iimeko ezimbini apho ukuvala ngobubele kungazange kusebenze kwaye kubangele ukungafumaneki kwexesha elifutshane kweprojekthi ngexesha lokusasazwa.

Ziqhelise. Iingxaki ezinokwenzeka ngokuvalwa okuthandekayo

NGINX

Okokuqala, kuluncedo ukukhumbula: ukongeza ekusebenziseni umyalelo nginx -s quit Kukho elinye inqanaba elifunekayo ukunikela ingqalelo. Siye sadibana nomcimbi apho i-NGINX isaya kuthumela i-SIGTERM endaweni yesignali ye-SIGQUIT, ebangela ukuba izicelo zingazaliseki ngokuchanekileyo. Iimeko ezifanayo zinokufunyanwa, umzekelo, apha. Ngelishwa, asikwazanga ukufumanisa isizathu esithile sokuziphatha: kwakukho ukukrokra malunga nenguqulo ye-NGINX, kodwa ayizange iqinisekiswe. Uphawu yayikukuba imiyalezo yajongwa kwizigodo ze-NGINX: "vula isokethi #10 ekhohlo kuqhagamshelo 5", emva koko i-pod yayeka.

Sinokuyibona ingxaki enjalo, umzekelo, kwiimpendulo kwi-Ingress esiyifunayo:

Izibonakaliso zeekhowudi zesimo ngexesha lokuthunyelwa

Kule meko, sifumana nje ikhowudi yephutha ye-503 evela kwi-Ingress ngokwayo: ayikwazi ukufikelela kwi-container ye-NGINX, ekubeni ingasafikeleleki. Ukuba ujonga iilogi zesikhongozeli nge-NGINX, ziqulethe oku kulandelayo:

[alert] 13939#0: *154 open socket #3 left in connection 16
[alert] 13939#0: *168 open socket #6 left in connection 13

Emva kokutshintsha umqondiso wokumisa, isitya siqala ukuyeka ngokuchanekileyo: oku kuqinisekiswa kukuba impazamo ye-503 ayisabonwa.

Ukuba udibana nengxaki efanayo, iyavakala ukufumanisa ukuba yeyiphi na isignali yokuyeka esetyenziswa kwisikhongozeli kwaye ijongeka njani i-preStop hook. Kunokwenzeka ukuba isizathu silele ngokuchanekileyo koku.

PHP-FPM... kunye nokunye

Ingxaki nge-PHP-FPM ichazwa ngendlela encinci: ayilindi ukugqitywa kweenkqubo zomntwana, iyabaphelisa, yingakho iimpazamo ze-502 zenzeka ngexesha lokuthunyelwa kunye neminye imisebenzi. Kukho iingxelo ezininzi zebug kwi-bugs.php.net ukusukela ngo-2005 (umz apha и apha), echaza le ngxaki. Kodwa ngokuqinisekileyo awuyi kubona nantoni na kwiilogi: i-PHP-FPM iya kubhengeza ukugqitywa kwenkqubo yayo ngaphandle kweempazamo okanye izaziso zeqela lesithathu.

Kuyafaneleka ukucacisa ukuba ingxaki ngokwayo inokuxhomekeka kumlinganiselo omncinci okanye omkhulu kwisicelo ngokwawo kwaye ingabonakali, umzekelo, ekubekeni iliso. Ukuba uyadibana nayo, indlela elula yokulungisa iza engqondweni kuqala: yongeza i-preStop hook nayo sleep(30). Iya kukuvumela ukuba ugcwalise zonke izicelo ebezikho ngaphambili (kwaye asamkeli ezintsha, kuba i-pod sele sele ndiyakwazi u Ukupheliswa), kwaye emva kwemizuzwana engama-30 i-pod ngokwayo iya kugqiba ngomqondiso SIGTERM.

Kuvela oko lifecycle kuba isitya siya kujongeka ngolu hlobo:

    lifecycle:
      preStop:
        exec:
          command:
          - /bin/sleep
          - "30"

Nangona kunjalo, ngenxa ye-30-yesibini sleep sikho kakhulu siya kwandisa ixesha lokuthunyelwa, ekubeni i-pod nganye iya kupheliswa ubuncinci Imizuzwana engama-30, into embi. Yintoni enokwenziwa ngale nto?

Makhe siphendukele kwiqela elijongene nokuphunyezwa ngokuthe ngqo kwesicelo. Kwimeko yethu kunjalo I-PHP-FPM, yintoni ngokungagqibekanga ayijongi ukuphunyezwa kweenkqubo zomntwana wayo: Inkqubo yenkosi iphela ngokukhawuleza. Ungayitshintsha le ndlela yokuziphatha usebenzisa isikhokelo process_control_timeout, echaza imida yexesha leenkqubo zomntwana zokulinda iimpawu ezivela kumphathi. Ukuba ubeka ixabiso kwimizuzwana engama-20, oku kuya kugubungela uninzi lwemibuzo ebaleka kwisikhongozeli kwaye iya kumisa inkqubo yenkosi xa sele igqityiwe.

Ngolu lwazi, masibuyele kwingxaki yethu yokugqibela. Njengoko kukhankanyiwe, i-Kubernetes ayilona iqonga le-monolithic: unxibelelwano phakathi kwamacandelo ayo ahlukeneyo kuthatha ixesha elithile. Oku kuyinyani ngakumbi xa siqwalasela ukusebenza kwe-Ingresses kunye nezinye izinto ezinxulumene nazo, kuba ngenxa yokulibaziseka okunjalo ngexesha lokuthunyelwa kulula ukufumana i-surge of 500 iimpazamo. Ngokomzekelo, impazamo inokuthi yenzeke kwinqanaba lokuthumela isicelo kumlambo onyukayo, kodwa "ixesha laxa" lokusebenzisana phakathi kwamacandelo lifutshane kakhulu - ngaphantsi kwesibini.

Ngoko ke, Zizonke ngomyalelo osele ukhankanyiwe process_control_timeout ungasebenzisa olu lwakhiwo lulandelayo ukuze lifecycle:

lifecycle:
  preStop:
    exec:
      command: ["/bin/bash","-c","/bin/sleep 1; kill -QUIT 1"]

Kule meko, siya kuhlawulela ukulibaziseka ngomyalelo sleep kwaye ungonyusi kakhulu ixesha lokuthunyelwa: ngaba kukho umahluko obonakalayo phakathi kwemizuzwana engama-30 kunye nenye?.. process_control_timeout, kwaye lifecycle isetyenziswe kuphela "njengomnatha wokhuseleko" xa kukho i-lag.

Ngokubanzi indlela yokuziphatha echaziweyo kunye nomsebenzi ohambelanayo awusebenzi kwi-PHP-FPM kuphela. Imeko efanayo inokuvela ngenye indlela okanye enye xa kusetyenziswa ezinye iilwimi/isakhelo. Ukuba awukwazi ukulungisa ukuvala ngobubele ngezinye iindlela - umzekelo, ngokubhala kwakhona ikhowudi ukuze isicelo siqhube kakuhle izibonakaliso zokuphelisa - ungasebenzisa indlela echazwe. Isenokungabi yeyona nto intle, kodwa iyasebenza.

Ziqhelise. Uvavanyo lomthwalo ukujonga ukusebenza kwepod

Ukuvavanywa komthwalo yenye yeendlela zokujonga ukuba isitya sisebenza njani, kuba le nkqubo iyisondeza kwiimeko zokwenyani zokulwa xa abasebenzisi bendwendwela indawo. Ukuvavanya ezi ngcebiso zingentla, ungasebenzisa Yandex.Tankom: Igubungela zonke iimfuno zethu ngokugqibeleleyo. Ezi zilandelayo ziingcebiso kunye neengcebiso zokuqhuba uvavanyo ngomzekelo ocacileyo kumava ethu sibonga kwiigrafu zeGrafana kunye neYandex.Tank ngokwayo.

Eyona nto ibalulekileyo apha khangela utshintsho step by step. Emva kokongeza ukulungiswa okutsha, sebenzisa uvavanyo kwaye ubone ukuba iziphumo zitshintshile xa kuthelekiswa nokubaleka kokugqibela. Ngaphandle koko, kuya kuba nzima ukuchonga izisombululo ezingasebenziyo, kwaye ekuhambeni kwexesha kunokwenza umonakalo kuphela (umzekelo, ukwandisa ixesha lokuthunyelwa).

Enye i-nuance kukujonga izigodo zesikhongozeli ngexesha lokupheliswa kwayo. Ngaba ulwazi malunga nokuvalwa okuthandekayo lurekhodiwe apho? Ngaba kukho naziphi na iimpazamo kwiilogi xa ufikelela kwezinye izibonelelo (umzekelo, kwisikhongozeli se-PHP-FPM esingummelwane)? Iimpazamo kwisicelo ngokwaso (njengokuba kunjalo nge-NGINX echazwe ngasentla)? Ndiyathemba ukuba ulwazi oluyintshayelelo oluphuma kweli nqaku luya kukunceda uqonde ngcono okwenzekayo kwisitya ngexesha lokupheliswa kwayo.

Ke, uvavanyo lokuqala lwenzeka ngaphandle lifecycle kwaye ngaphandle kwezikhokelo ezongezelelweyo zomncedisi wesicelo (process_control_timeout kwi-PHP-FPM). Injongo yolu vavanyo yayikukubona inani leempazamo (nokuba zikhona na). Kwakhona, ukusuka kulwazi olongezelelweyo, kufuneka wazi ukuba ixesha eliqhelekileyo lokuthunyelwa kwi-pod nganye yayimalunga ne-5-10 imizuzwana de ibe ilungile ngokupheleleyo. Iziphumo zezi:

Iphaneli yolwazi ye-Yandex.Tank ibonisa i-spike yeempazamo ze-502, ezenzeke ngexesha lokuthunyelwa kwaye zahlala kumyinge ukuya kwimizuzwana emi-5. Kucingelwa ukuba oku kwakungenxa yokuba izicelo esele zikho kwipod endala zaziyekiswa xa yayiyekiswa. Emva koko, iimpazamo ze-503 zavela, eziye zaba ngumphumo we-container ye-NGINX emisiwe, ebuye yehla uxhulumaniso ngenxa ye-backend (eyayithintela i-Ingress ukuba idibanise kuyo).

Makhe sibone ukuba njani process_control_timeout kwi-PHP-FPM iya kusinceda silinde ukugqitywa kweenkqubo zomntwana, okt. lungisa iimpazamo ezinjalo. Sebenzisa kwakhona lo myalelo:

Azisekho iimpazamo ngexesha lokuthunyelwa kwe-500! Ukusasazwa kuphumelele, ukuvalwa okuthandekayo kuyasebenza.

Nangona kunjalo, kufanelekile ukukhumbula umba kunye nezikhongozeli ze-Ingress, ipesenti encinci yeempazamo esinokuzifumana ngenxa yexesha. Ukuziphepha, konke okuseleyo kukongeza isakhiwo kunye sleep kwaye phinda ukuthunyelwa. Nangona kunjalo, kwimeko yethu ethile, akukho zinguqu zibonakalayo (kwakhona, akukho ziphoso).

isiphelo

Ukuphelisa inkqubo ngobubele, silindele le ndlela yokuziphatha ilandelayo kwisicelo:

1. Yima imizuzwana embalwa kwaye uyeke ukwamkela imidibaniso emitsha.
2. Linda zonke izicelo zigqibezele kwaye uvale lonke uqhagamshelwano olugcina uphila olungazenzi izicelo.
3. Phelisa inkqubo yakho.

Nangona kunjalo, ayizizo zonke izicelo ezinokusebenza ngolu hlobo. Esinye isisombululo kwingxaki kwi-Kubernetes realities:

• ukongeza i-hook yokumisa kwangaphambili eya kulinda imizuzwana embalwa;
• ukufunda ifayile yoqwalaselo ye-backend yethu yeeparamitha ezifanelekileyo.

Umzekelo kunye ne-NGINX yenza kucace ukuba nokuba isicelo ekufuneka siqale sisebenze iimpawu zokuphelisa ngokuchanekileyo asinakwenza njalo, ngoko ke kubalulekile ukujonga iimpazamo ze-500 ngexesha lokufakwa kwesicelo. Oku kwakhona kukuvumela ukuba ujonge ingxaki ngokubanzi kwaye ungagxininisi kwipod enye okanye isitya, kodwa jonga iziseko zophuhliso ngokubanzi.

Njengesixhobo sokuvavanya, ungasebenzisa i-Yandex.Tank ngokubambisana nayo nayiphi na inkqubo yokubeka iliso (kwimeko yethu, idatha ithathwe kwiGrafana kunye ne-Prometheus backend yovavanyo). Iingxaki ngokuvalwa okuthandekayo zibonakala ngokucacileyo phantsi kwemithwalo enzima enokuvelisa i-benchmark, kwaye ukubeka iliso kunceda ukuhlalutya imeko ngokubanzi ngexesha okanye emva kovavanyo.

Ekuphenduleni impendulo kwinqaku: kuyafaneleka ukukhankanya ukuba iingxaki kunye nezisombululo zichazwe apha ngokubhekiselele kwi-NGINX Ingress. Kwezinye iimeko, kukho ezinye izisombululo, esinokuziqwalasela kwezi zinto zilandelayo zoluhlu.

PS

Okunye kwi K8s iingcebiso & tricks series:

• «Amaphepha eempazamo zesiko kwi-NGINX Ingress";
• «Malunga nolwabiwo lweenodi kunye nemithwalo kwisicelo sewebhu";
• «Ukufikelela kwiziza ze-dev";
• «Ukukhawulezisa i-bootstrap yogcino-lwazi olukhulu».

umthombo: www.habr.com