Makhe sijonge ingqikelelo ye-Kubernetes esweni, siqhelane nesixhobo sePrometheus, kwaye sithethe malunga nokulumkisa.
Isihloko sokubeka iliso sininzi; Injongo yesi sicatshulwa kukubonelela ngesishwankathelo sezixhobo, iikhonsepthi kunye neendlela.
Inqaku lezinto eziphathekayo - isicatshulwa ukusuka . Ukuba ufuna ukufumana uqeqesho olupheleleyo, bhalisela ikhosi kwi .
Yintoni ebekwe esweni kwiqela leKubernetes
Iiseva eziphathekayo. Ukuba iqela le-Kubernetes libekwe kwiiseva zalo, kufuneka ubeke iliso kwimpilo yabo. I-Zabbix iphethe lo msebenzi; ukuba usebenza naye, ngoko akukho mfuneko yokwala, akuyi kubakho zingxabano. YiZabbix ebeka esweni imeko yeeseva zethu.
Masiqhubele phambili ekubekeni iliso kwinqanaba leqela.
Amacandelo ePlanethi yokulawula: I-API, uMcwangcisi kunye nabanye. Ubuncinci, kufuneka ubeke iliso ukuba i-API yomncedisi okanye i-etcd inkulu kuno-0. I-Etcd inokubonelela ngee-metrics ezininzi: kwiidiski apho ijikeleza khona, kwimpilo ye-cluster yayo etcd, kunye nabanye.
Docker yavela kudala kwaye wonke umntu uyazi kakuhle iingxaki zayo: izikhongozeli ezininzi zibangela umkhenkce kunye nezinye iingxaki. Ke ngoko, iDocker ngokwayo, njengenkqubo, kufuneka ibekwe iliso, ubuncinci ukufumaneka.
I-DNS. Ukuba i-DNS iyasilela kwiqela, ngoko ke yonke inkonzo yokuFumana nayo iya kusilela, kwaye iifowuni ezivela kwiipod ukuya kwiipods ziya kuyeka ukusebenza. Kwimeko yam, kwakungekho ngxaki, kodwa oku akuthethi ukuba imeko ye-DNS ayifuni kubekwa iliso. Cela ukubambezeleka kunye nezinye iimetrics zinokulandelelwa kwi-CoreDNS.
Ingress. Kuyimfuneko ukulawula ukufumaneka kwe-ingresses (kubandakanywa noMlawuli we-Ingress) njengeendawo zokungena kwiprojekthi.
Amacandelo aphambili e-cluster adityanisiwe - ngoku makhe siye ezantsi, ukuya kwinqanaba lokukhupha.
Kubonakala ngathi izicelo ziqhutywa kwiipods, okuthetha ukuba kufuneka zilawulwe, kodwa eneneni azinjalo. Iipods zi-ephemeral: namhlanje zisebenza kwiseva enye, ngomso kwenye; Namhlanje kukho i-10 kubo, ngomso 2. Yingakho kungekho mntu ubeka iliso kwiipods. Kwi-architecture ye-microservice, kubaluleke kakhulu ukulawula ukufumaneka kwesicelo ngokubanzi. Ngokukodwa, jonga ukufumaneka kokuphela kwenkonzo: ngaba kukho into esebenzayo? Ukuba isicelo siyafumaneka, ngoko kwenzeka ntoni emva kwayo, zingaphi ii-replicas ezikhoyo ngoku - le yimibuzo ye-oda lesibini. Akukho mfuneko yokubeka iliso kwiziganeko zomntu ngamnye.
Kwinqanaba lokugqibela, kufuneka ubeke iliso ekusebenzeni kwesicelo ngokwalo, thatha iimethrikhi zoshishino: inani lee-odolo, ukuziphatha komsebenzisi, njl.
Prometheus
Eyona nkqubo ilungileyo yokubeka iliso kwiqela . Andazi nasiphi na isixhobo esinokuthelekisa nePrometheus ngokomgangatho kunye nokusebenziseka ngokulula. Ilungile kwiziseko ezingundoqo, ke xa abantu besithi "Kubernetes iliso" bahlala bethetha iPrometheus.
Kukho iinketho ezimbalwa zokuqalisa ngePrometheus: usebenzisa iHelm, ungafaka iPrometheus eqhelekileyo okanye iPrometheus Operator.
- Prometheus rhoqo. Yonke into ilungile ngayo, kodwa kufuneka uqwalasele iConfigMap - ngokusisiseko, bhala iifayile zoqwalaselo lombhalo, njengoko besenzile ngaphambili, phambi koyilo lwemicroservice.
- I-Prometheus Operator ibanzi ngakumbi, intsonkothe ngakumbi kwingqiqo yayo yangaphakathi, kodwa kulula ukusebenza nayo: kukho izinto ezahlukileyo, izinto ezithathwayo zongezwa kwiqela, ngoko ke kulula kakhulu ukulawula nokuqwalasela.
Ukuqonda imveliso, ndincoma ukufaka iPrometheus rhoqo kuqala. Kuya kufuneka uqwalasele yonke into ngoqwalaselo, kodwa oku kuya kuba luncedo: uya kuqonda ukuba yeyiphi kwaye iqwalaselwe njani. Kwi-Prometheus Operator, ngoko nangoko uphakama uye kwindawo ephezulu, nangona ukuba uyafuna, unokude ungene enzulwini.
I-Prometheus ihlanganiswe kakuhle kunye ne-Kubernetes: inokufikelela kunye nokusebenzisana ne-API Server.
I-Prometheus iyathandwa kwaye ixhaswa yinani elikhulu lezicelo kunye neelwimi zeprogram. Inkxaso iyafuneka kuba i-Prometheus inefomathi yayo yeemetriki, kwaye ukuyihambisa kufuneka ilayibrari ngaphakathi kwesicelo okanye umthengisi osele ekulungele. Kwaye baninzi kakhulu abo bathumela ngaphandle. Ngokomzekelo, kukho i-PostgreSQL Exporter: ithatha idatha kwi-PostgreSQL kwaye iyiguqule kwifomathi ye-Prometheus ukuze i-Prometheus isebenze nayo.
Uyilo lwePrometheus
Umncedisi wePrometheus - le yinxalenye yeseva, ingqondo kaPrometheus. Apha kulapho iimetrics zigcinwa kwaye zisetyenzwe.
Iimetrics zigcinwa kuluhlu lwedatha yedatha (TSDB). I-TSDB ayisosiseko sedatha esahlukileyo, kodwa iphakheji ye-Go eyakhelwe kwi-Prometheus. Ngokucacileyo, yonke into ikwi-binary enye.
Sukugcina idatha kwi-TSDB ixesha elide
Isiseko sePrometheus asifanelekanga ukugcinwa kwexesha elide leemethrikhi. Ixesha elimiselweyo lokugcinwa ziintsuku ezili-15. Ungawugqitha lo mda, kodwa kufuneka uhlale ukhumbula: okukhona ugcina idatha kwi-TSDB kwaye xa uyenza ixesha elide, kokukhona izibonelelo ziya kusebenzisa. Ukugcina idatha yembali kwiPrometheus kuthathwa njengento embi.
Ukuba unetrafikhi enkulu, inani leemethrikhi likhulu lamawaka ngomzuzwana, ke kungcono ukunciphisa ukugcinwa kwazo ngendawo yediski okanye ubude bexesha. Ngokuqhelekileyo, i-TSDB igcina "idatha eshushu", iimethrikhi ukusuka ngokoqobo kwiiyure ezimbalwa. Ukugcinwa kwexesha elide, ukugcinwa kwangaphandle kusetyenziswe kwezo nkcukacha ezifanelekileyo ngokwenene kule nto, umzekelo, i-InfluxDB, i-ClickHouse, njalo njalo. Ndabona uphononongo oluhle ngakumbi malunga neClickHouse.
Iseva yePrometheus isebenza ngokwemodeli Tsala: yena ngokwakhe uya kwi-metrics kwii-endpoints esimnike zona. Bathe: "yiya kwi-API Server," kwaye iya apho yonke inombolo ye-nth yemizuzwana kwaye ithatha i-metrics.
Kwizinto ezinobomi obufutshane (umsebenzi okanye umsebenzi wecron) ezinokuvela phakathi kwamaxesha okukhuhla, kukho icandelo lePushgateway. Iimethrikhi ezivela kwizinto zexesha elifutshane zityhalwa kuyo: umsebenzi wavuka, wagqiba isenzo, wathumela i-metrics kwiPushgateway kwaye yagqitywa. Emva kwexesha elithile, uPrometheus uya kuhamba ngesantya sakhe kwaye athathe ezi metrics kwiPushgateway.
Ukuqwalasela izaziso kwi-Prometheus kukho icandelo elahlukileyo - Umphathi weAlert. Kwaye imithetho yokulumkisa. Umzekelo, kufuneka udale isilumkiso ukuba i-API yomncedisi ngu-0. Xa isiganeko siqaliswa, isilumkiso sigqithiselwe kumphathi we-alert ukuze uthumele ngakumbi. Umphathi wesilumkiso unesetingi zendlela ebhetyebhetye: elinye iqela lezilumkiso linokuthunyelwa kwincoko yetelegram ye-admins, elinye kwincoko yabaphuhlisi, kunye nesithathu kwincoko yabasebenzi beziseko zophuhliso. Izaziso zinokuthunyelwa kwi-Slack, iTelegram, i-imeyile kunye namanye amajelo.
Kwaye ekugqibeleni, ndiza kukuxelela malunga nenqaku elibulalayo likaPrometheus - Ukufumanisa. Xa usebenza ne-Prometheus, awudingi ukucacisa iidilesi ezithile zezinto zokubeka iliso kwanele ukucacisa uhlobo lwazo. Oko kukuthi, akukho mfuneko yokubhala "nantsi idilesi ye-IP, nantsi i-port - monitor", endaweni yoko kufuneka unqume ukuba yeyiphi imigaqo yokufumana ezi zinto (iithagethi - iinjongo). I-Prometheus ngokwayo, kuxhomekeke ekubeni zeziphi izinto ezisebenzayo ngoku, ikhupha ezo ziyimfuneko kwaye izongeze ekubekeni iliso.
Le ndlela ihambelana kakuhle nesakhiwo se-Kubernetes, apho yonke into iphinda ihambe: namhlanje kukho iiseva ze-10, kusasa 3. Ukuze ungabonakali idilesi ye-IP yomncedisi rhoqo, sake sabhala indlela yokuyifumana - kwaye i-Discovering iya kuyenza.
Ulwimi lwePrometheus lubizwa ngokuba I-PromQL. Ukusebenzisa olu lwimi, unokufumana amaxabiso eemethrikhi ezithile kwaye emva koko uwaguqule kwaye wakhe izibalo zohlalutyo ezisekwe kuwo.
https://prometheus.io/docs/prometheus/latest/querying/basics/
Простой запрос
container_memory_usage_bytes
Математические операции
container_memory_usage_bytes / 1024 / 1024
Встроенные функции
sum(container_memory_usage_bytes) / 1024 / 1024
Уточнение запроса
100 - avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m]) * 100)Ujongano lwewebhu lwePrometheus
I-Prometheus inendawo yayo, kunokuba i-minimalistic web interface. Ilungele kuphela ukulungisa ingxaki okanye umboniso.
Ungabhala umbuzo kwi-PromQL kwi-Expression line.
Izaziso isithuba iqulathe imigaqo yokwazisa, kwaye zinemigangatho emithathu:
- engasebenziyo - ukuba isilumkiso asisebenzi okwangoku, oko kukuthi, yonke into ilungile nayo, kwaye ayizange isebenze;
- ilindile - oku ukuba isilumkiso siqalisiwe, kodwa uthumelo akukenzeki. Ukulibaziseka kusetelwe ukuhlawulela ukuqhwanyaza kwenethiwekhi: ukuba inkonzo echaziweyo iphakame ngaphakathi komzuzu, akukho mfuneko yokukhalisa i-alamu okwangoku;
- ukudubula linqanaba lesithathu, xa isivuseleli sikhanyisa kwaye sithumela imiyalezo.
KwiMenu yeSimo uya kufumana ukufikelela kulwazi malunga nokuba yintoni iPrometheus. Kukho kwakhona inguqu kwiinjongo esithethe ngazo ngasentla.
Ukuze ufumane inkcazelo engakumbi yojongano lwePrometheus, bona .
Ukudityaniswa neGrafana
Kwi-interface yewebhu ye-Prometheus awuyi kufumana iigrafu ezintle neziqondakalayo apho unokuthatha khona izigqibo malunga nemeko yeqela. Ukuzakha, uPrometheus udibanisa noGrafana. Ezi ziideshibhodi esizifumanayo.
Ukuseta ukudityaniswa kwePrometheus kunye neGrafana akunzima konke konke; , kulungile, ndiza kugqiba ngale nto.
Kumanqaku alandelayo siza kuqhubeka nesihloko sokubeka iliso: siza kuthetha ngokuqokelela nokuhlalutya iinkuni sisebenzisa iGrafana Loki kunye nezinye izixhobo.
Umbhali: UMarcel Ibraev, umlawuli oqinisekisiweyo we-Kubernetes, uqeqeshe injineli kwinkampani , isithethi kunye nomphuhlisi wekhosi ye-Slurm.
umthombo: www.habr.com
