Amathiphu namaqhinga we-Kubernetes: izici zokuvala shaqa okuhle ku-NGINX naku-PHP-FPM

Isimo esijwayelekile lapho kusetshenziswa i-CI/CD ku-Kubernetes: isicelo kufanele sikwazi ukungamukeli izicelo zeklayenti ezintsha ngaphambi kokuma ngokuphelele, futhi okubaluleke kakhulu, ukuqedela ezikhona ngempumelelo.

Ukuhambisana nalesi simo kukuvumela ukuthi uzuze isikhathi sokuphumula esiziro ngesikhathi sokuthunyelwa. Kodwa-ke, noma usebenzisa izinqwaba ezidume kakhulu (njenge-NGINX ne-PHP-FPM), ungahlangabezana nobunzima obuzoholela ekuqubukeni kwamaphutha ngokusetshenziswa ngakunye...

Ithiyori. Iphila kanjani i-pod

Sesivele sishicilele ngokuningiliziwe mayelana nomjikelezo wempilo ye-pod le ndatshana. Kumongo wesihloko esicatshangelwayo, sinentshisekelo kulokhu okulandelayo: okwamanje lapho i-pod ingena embusweni Ukuqeda, izicelo ezintsha ziyeka ukuthunyelwa kuyo (pod kususiwe ohlwini lwezindawo zokugcina zesevisi). Ngakho-ke, ukugwema isikhathi sokuphumula ngesikhathi sokuthunyelwa, kwanele ngathi ukuxazulula inkinga yokumisa uhlelo ngendlela efanele.

Kufanele futhi ukhumbule ukuthi isikhathi somusa esimisiwe siyi 30 imizuzwana: ngemva kwalokhu, i-pod izonqanyulwa futhi isicelo kufanele sibe nesikhathi sokucubungula zonke izicelo ngaphambi kwalesi sikhathi. Ukubhala: nakuba noma yisiphi isicelo esithatha ngaphezu kwamasekhondi angu-5-10 sesivele siyinkinga, futhi ukuvala shaqa okuhle ngeke kusasiza...

Ukuze uqonde kangcono ukuthi kwenzekani lapho i-pod iphela, vele ubheke umdwebo olandelayo:

A1, B1 - Ithola izinguquko mayelana nesimo seziko
A2 - Ukuhamba SIGTERM
B2 - Ukukhipha i-pod ezindaweni zokugcina
B3 - Ukuthola izinguquko (uhlu lwamaphoyinti okugcina lushintshile)
B4 - Buyekeza imithetho ye-iptables

Sicela uqaphele: ukususa iphodi yephoyinti lokugcina nokuthumela i-SIGTERM akwenzeki ngokulandelana, kodwa ngokuhambisana. Futhi ngenxa yokuthi i-Ingress ayitholi ngokushesha uhlu olusha lwe-Endpoints, izicelo ezintsha ezivela kumakhasimende zizothunyelwa ku-pod, okuzodala iphutha le-500 ngesikhathi sokuqedwa kwe-pod. (ukuthola imininingwane eyengeziwe ngalolu daba, si kuhunyushwe). Le nkinga idinga ukuxazululwa ngalezi zindlela ezilandelayo:

• Thumela Ukuxhumana: vala izihloko zempendulo (uma lokhu kuthinta uhlelo lokusebenza lwe-HTTP).
• Uma kungenakwenzeka ukwenza izinguquko kukhodi, isihloko esilandelayo sichaza isisombululo esizokuvumela ukuthi ucubungule izicelo kuze kube sekupheleni kwesikhathi somusa.

Ithiyori. I-NGINX ne-PHP-FPM ziqeda kanjani izinqubo zabo

NGINX

Ake siqale nge-NGINX, njengoba yonke into isobala ngayo. Ngokungena emcabangweni, sifunda ukuthi i-NGINX inenqubo eyodwa eyinhloko kanye “nezisebenzi” ezimbalwa - lezi yizinqubo zezingane ezicubungula izicelo zamaklayenti. Inketho elula inikezwa: usebenzisa umyalo nginx -s <SIGNAL> nqamula izinqubo ngokuvala shaqa okusheshayo noma kumodi yokuvala shaqa enomusa. Ngokusobala, inketho yokugcina esithakaselayo.

Khona-ke konke kulula: udinga ukungeza i-preStop-hook umyalo ozothumela isignali yokuvala shaqa enomusa. Lokhu kungenziwa ku-Deployment, ku-container block:

       lifecycle:
          preStop:
            exec:
              command:
              - /usr/sbin/nginx
              - -s
              - quit

Manje, lapho i-pod ivala, sizobona okulandelayo kulogi yesitsha se-NGINX:

2018/01/25 13:58:31 [notice] 1#1: signal 3 (SIGQUIT) received, shutting down
2018/01/25 13:58:31 [notice] 11#11: gracefully shutting down

Futhi lokhu kuzosho esikudingayo: I-NGINX ilinda ukuthi izicelo ziqedwe, bese ibulala inqubo. Nokho, ngezansi sizophinde sicabangele inkinga evamile ngenxa yalokho, ngisho nomyalo nginx -s quit inqubo iphela ngokungalungile.

Futhi kulesi sigaba senziwe nge-NGINX: okungenani kusukela ezingodweni ungaqonda ukuthi konke kusebenza ngendlela efanele.

Iyini inkontileka nge-PHP-FPM? Ikuphatha kanjani ukuvala shaqa okuhle? Ake sikuthole.

I-PHP-FPM

Endabeni ye-PHP-FPM, kunolwazi oluncane oluncane. Uma ugxila imanuwali esemthethweni ngokusho kwe-PHP-FPM, izothi amasiginali we-POSIX alandelayo amukelwe:

1. SIGINT, SIGTERM - ukuvala ngokushesha;
2. SIGQUIT - ukuvala ucingo ngomusa (esikudingayo).

Izimpawu ezisele azidingeki kulo msebenzi, ngakho-ke sizokuyeka ukuhlaziya kwazo. Ukuze unqamule inqubo ngendlela efanele, uzodinga ukubhala i-preStop hook elandelayo:

        lifecycle:
          preStop:
            exec:
              command:
              - /bin/kill
              - -SIGQUIT
              - "1"

Uma uthi nhlá, yilokhu kuphela okudingekayo ukuze kuvalwe kahle kuzo zombili iziqukathi. Nokho, umsebenzi unzima kakhulu kunalokho okubonakala. Ngezansi kunezimo ezimbili lapho ukuvala shaqa okuhle kungazange kusebenze futhi kwabangela ukungatholakali kwephrojekthi okwesikhashana ngesikhathi sokuthunyelwa.

Prakthiza. Izinkinga ezingaba khona ngokuvala shaqa okuhle

NGINX

Okokuqala, kuyasiza ukukhumbula: ngaphezu kokwenza umyalo nginx -s quit Kunesinye futhi isigaba okufanele sinakwe. Sihlangabezane nenkinga lapho i-NGINX isazothumela khona i-SIGTERM esikhundleni sesignali ye-SIGQUIT, okubangele ukuthi izicelo zingaqedi kahle. Amacala afanayo angatholakala, isibonelo, lapha. Ngeshwa, asikwazanga ukucacisa isizathu esiqondile salokhu kuziphatha: kwakukhona izinsolo mayelana nenguqulo ye-NGINX, kodwa ayizange iqinisekiswe. Uphawu bekungukuthi imilayezo ibonwe kumalogi esiqukathi se-NGINX: "vula isokhethi #10 esele ekuxhumekeni 5", emva kwalokho i-pod yama.

Singabona inkinga enjalo, ngokwesibonelo, ezimpendulweni ze-Ingress esizidingayo:

Izinkomba zamakhodi esimo ngesikhathi sokuthunyelwa

Kulokhu, sithola ikhodi yephutha engu-503 kuphela evela ku-Ingress ngokwayo: ayikwazi ukufinyelela isitsha se-NGINX, ngoba ayisatholakali. Uma ubheka izingodo zesitsha nge-NGINX, ziqukethe okulandelayo:

[alert] 13939#0: *154 open socket #3 left in connection 16
[alert] 13939#0: *168 open socket #6 left in connection 13

Ngemva kokushintsha isignali yokumisa, isitsha siqala ukuma ngendlela efanele: lokhu kuqinisekiswa ukuthi iphutha le-503 alisabonwa.

Uma uhlangabezana nenkinga efanayo, kunengqondo ukuthola ukuthi iyiphi isignali yokumisa esetshenziswa esitsheni nokuthi ihuku le-preStop libukeka kanjani. Kungenzeka ukuthi isizathu silele kulokhu.

PHP-FPM... nokunye

Inkinga nge-PHP-FPM ichazwa ngendlela encane: ayilindi ukuqedwa kwezinqubo zengane, iyawaqeda, yingakho amaphutha angu-502 eyenzeka ngesikhathi sokuthunyelwa kanye neminye imisebenzi. Kunemibiko embalwa yeziphazamisi ku-bugs.php.net kusukela ngo-2005 (isb lapha и lapha), echaza le nkinga. Kodwa cishe ngeke ubone lutho kulogi: I-PHP-FPM izomemezela ukuqedwa kwenqubo yayo ngaphandle kwamaphutha noma izaziso ezivela eceleni.

Kuyafaneleka ukucacisa ukuthi inkinga ngokwayo ingase incike ezingeni elincane noma elikhulu kuhlelo lokusebenza ngokwalo futhi ingase ingazibonakalisi, isibonelo, ekuqapheni. Uma uhlangabezana nakho, indlela elula yokulungisa ifika engqondweni kuqala: engeza ihuku ye-preStop nayo sleep(30). Izokuvumela ukuthi ugcwalise zonke izicelo ebezingaphambili (futhi asamukeli ezintsha, kusukela ku-pod vele ekwaziyo Ukuqeda), futhi ngemva kwemizuzwana engu-30 i-pod ngokwayo izophela ngesignali SIGTERM.

Kuvela lokho lifecycle ngoba isitsha sizobukeka kanje:

    lifecycle:
      preStop:
        exec:
          command:
          - /bin/sleep
          - "30"

Nokho, ngenxa 30-yesibili sleep thina okuningi sizokwandisa isikhathi sokuthunyelwa, njengoba i-pod ngayinye izonqanyulwa ubuncane imizuzwana engu-30, okuyinto embi. Yini engenziwa ngalokhu?

Ake siphendukele eqenjini elibhekele ukwenziwa kwesicelo ngokuqondile. Esimeni sethu kunjalo I-PHP-FPM, okuyinto ngokuzenzakalelayo ayiqapheli ukwenziwa kwezinqubo zengane yayo: Inqubo eyinhloko inqanyulwa ngokushesha. Ungashintsha lokhu kuziphatha usebenzisa isiyalelo process_control_timeout, ecacisa imikhawulo yesikhathi yezinqubo zengane zokulinda amasignali avela kunkosi. Uma usetha inani libe imizuzwana engu-20, lokhu kuzovala imibuzo eminingi egijima esitsheni futhi kuzomisa inqubo eyinhloko uma isiqediwe.

Ngalolu lwazi, ake sibuyele enkingeni yethu yokugcina. Njengoba kushiwo, i-Kubernetes ayiyona inkundla ye-monolithic: ukuxhumana phakathi kwezingxenye zayo ezihlukene kuthatha isikhathi. Lokhu kuyiqiniso ikakhulukazi uma sicabangela ukusebenza kwe-Ingresses nezinye izingxenye ezihlobene, ngoba ngenxa yokubambezeleka okunjalo ngesikhathi sokuthunyelwa kulula ukuthola ukukhuphuka kwamaphutha angu-500. Isibonelo, iphutha lingase lenzeke esigabeni sokuthumela isicelo endaweni engenhla nomfula, kodwa "isikhathi esinqunyiwe" sokusebenzelana phakathi kwezingxenye sifushane kakhulu - ngaphansi kwesekhondi.

Ngakho-ke, Sekukonke ngomyalelo osuvele ushiwo process_control_timeout ungasebenzisa ukwakhiwa okulandelayo ukuze lifecycle:

lifecycle:
  preStop:
    exec:
      command: ["/bin/bash","-c","/bin/sleep 1; kill -QUIT 1"]

Kulokhu, sizonxephezela ukubambezeleka ngomyalo sleep futhi ungakhulisi kakhulu isikhathi sokuthunyelwa: ingabe ukhona umehluko obonakalayo phakathi kwemizuzwana engu-30 neyodwa?.. Eqinisweni, yiyona process_control_timeout, futhi lifecycle isetshenziswa kuphela “njengenethi yokuphepha” uma kwenzeka kubanekezeka.

Sengisho nje, ukuziphatha okuchaziwe kanye nendlela yokusebenza ehambisanayo akusebenzi ku-PHP-FPM kuphela. Isimo esifanayo singavela ngendlela eyodwa noma enye lapho kusetshenziswa ezinye izilimi/izinhlaka. Uma ungakwazi ukulungisa ukuvala shaqa okuhle ngezinye izindlela - ngokwesibonelo, ngokubhala kabusha ikhodi ukuze uhlelo lokusebenza lucubungule kahle izimpawu zokunqanyulwa - ungasebenzisa indlela echaziwe. Kungase kungabi kuhle kakhulu, kodwa kuyasebenza.

Prakthiza. Ukuhlolwa komthwalo ukuhlola ukusebenza kwe-pod

Ukuhlolwa komthwalo kungenye yezindlela zokuhlola ukuthi isiqukathi sisebenza kanjani, njengoba le nqubo isisondeza ezimweni zangempela zokulwa lapho abasebenzisi bevakashela isayithi. Ukuze uhlole izincomo ezingenhla, ungasebenzisa I-Yandex.Tankom: Ihlanganisa zonke izidingo zethu ngokuphelele. Okulandelayo ngamathiphu nezincomo zokwenza ukuhlolwa ngesibonelo esicacile esivela kokuhlangenwe nakho kwethu ngenxa yamagrafu e-Grafana ne-Yandex.I-Tank ngokwayo.

Okubaluleke kakhulu lapha hlola izinguquko isinyathelo ngesinyathelo. Ngemva kokwengeza ukulungiswa okusha, qalisa ukuhlola futhi ubone ukuthi ingabe imiphumela ishintshile uma kuqhathaniswa nokugijima kokugcina. Uma kungenjalo, kuyoba nzima ukukhomba izixazululo ezingasebenzi, futhi ngokuhamba kwesikhathi kungenza umonakalo kuphela (isibonelo, ukwandisa isikhathi sokuthunyelwa).

Enye i-nuance ukubheka izingodo zesitsha ngesikhathi sokunqanyulwa kwayo. Ingabe ulwazi mayelana nokuvalwa komusa lurekhodiwe lapho? Ingabe akhona amaphutha kulogi lapho ufinyelela ezinye izinsiza (isibonelo, kwisitsha esingumakhelwane se-PHP-FPM)? Amaphutha kuhlelo lokusebenza ngokwalo (njengoba kwenzeka nge-NGINX echazwe ngenhla)? Ngithemba ukuthi ulwazi oluyisingeniso oluvela kulesi sihloko luzokusiza ukuthi uqonde kangcono ukuthi kwenzekani esitsheni ngesikhathi sokunqanyulwa kwaso.

Ngakho, ukuhlolwa kokuqala kwenziwa ngaphandle lifecycle futhi ngaphandle kweziqondiso ezengeziwe zeseva yohlelo (process_control_timeout ku-PHP-FPM). Inhloso yalokhu kuhlola bekuwukubona inani elilinganiselwe lamaphutha (nokuthi akhona yini). Futhi, kusukela olwazini olwengeziwe, kufanele wazi ukuthi isikhathi esimaphakathi sokuphakelwa kwe-pod ngayinye sasicishe sibe amasekhondi angu-5-10 size silunge ngokugcwele. Imiphumela yilena:

Iphaneli yolwazi ye-Yandex.Tank ibonisa ukwanda kwamaphutha angu-502, enzeke ngesikhathi sokuphakwa futhi ahlala ngokwesilinganiso kufika kumasekhondi angu-5. Ngokunokwenzeka lokhu kwakungenxa yokuthi izicelo ezikhona ku-pod endala zazinqanyulwa ngesikhathi kupheliswa. Ngemva kwalokhu, amaphutha angu-503 avela, okwakuwumphumela wesitsha esimisiwe se-NGINX, esiphinde sehlise ukuxhumeka ngenxa ye-backend (evimbele i-Ingress ekuxhumekeni kuyo).

Ake sibone ukuthi kanjani process_control_timeout ku-PHP-FPM izosisiza ukuthi silinde ukuqedwa kwezinqubo zezingane, i.e. lungisa amaphutha anjalo. Khipha kabusha usebenzisa lesi siqondiso:

Awasekho amaphutha ngesikhathi sokuthunyelwa kwe-500! Ukuthunyelwa kuphumelele, ukuvala shaqa okuhle kuyasebenza.

Kodwa-ke, kufanelekile ukukhumbula udaba ngeziqukathi ze-Ingress, iphesenti elincane lamaphutha esingase siwathole ngenxa yesikhathi eside. Ukuze uzigweme, okusele nje ukwengeza isakhiwo nge sleep bese uphinda ukuthunyelwa. Kodwa-ke, esimweni sethu esithile, azikho izinguquko ezibonakalayo (futhi, awekho amaphutha).

isiphetho

Ukuze sinqamule inqubo ngomusa, silindele ukuziphatha okulandelayo esicelweni:

1. Linda imizuzwana embalwa bese uyeka ukwamukela ukuxhumana okusha.
2. Lindela zonke izicelo ukuthi ziqedele futhi uvale zonke izixhumanisi ezigcinayo ezingasebenzisi izicelo.
3. Qeda inqubo yakho.

Nokho, akuzona zonke izinhlelo zokusebenza ezingasebenza ngale ndlela. Isixazululo esisodwa senkinga kumaqiniso e-Kubernetes sithi:

• ukwengeza i-hook yangaphambi kokumisa ezolinda imizuzwana embalwa;
• ukufunda ifayela lokucushwa le-backend yethu kumapharamitha afanelekile.

Isibonelo nge-NGINX senza kucace ukuthi ngisho nesicelo okufanele siqale sicubungule izimpawu zokunqanyulwa ngendlela efanele angeke yenze kanjalo, ngakho-ke kubalulekile ukuhlola amaphutha we-500 ngesikhathi sokuthunyelwa kwesicelo. Lokhu futhi kukuvumela ukuthi ubheke inkinga ngokubanzi futhi ungagxili ku-pod eyodwa noma isitsha, kodwa ubheke yonke ingqalasizinda iyonke.

Njengethuluzi lokuhlola, ungasebenzisa i-Yandex.Tank ngokuhambisana nanoma iyiphi isistimu yokuqapha (kithi, idatha ithathwe ku-Grafana nge-backend ye-Prometheus ukuze kuhlolwe). Izinkinga ngokuvala shaqa okuhle zibonakala ngokucacile ngaphansi kwemithwalo esindayo engenziwa ibhentshimakhi, futhi ukuqapha kusiza ukuhlaziya isimo ngokuningiliziwe phakathi noma ngemva kokuhlolwa.

Ekuphenduleni impendulo esihlokweni: kufanelekile ukusho ukuthi izinkinga nezixazululo zichazwe lapha maqondana ne-NGINX Ingress. Kwezinye izimo, kunezinye izixazululo, esingase sizicabangele ezintweni ezilandelayo zochungechunge.

PS

Okunye okuvela ochungechungeni lwamathiphu namaqhinga we-K8s:

• «Amakhasi Ephutha Ngokwezifiso ku-NGINX Ingress";
• «Mayelana nokwabiwa kwamanodi kanye nemithwalo kuhlelo lokusebenza lwewebhu";
• «Ukufinyelela kumasayithi we-dev";
• «Ukusheshisa i-bootstrap yedathabheyisi enkulu".

Source: www.habr.com