Ndikupangira kuti muwerenge zolembedwa za lipoti la Roman Khavronenko "ExtendedPromQL"
Mwachidule za ine. Dzina langa ndine Roman. Ndimagwira ntchito ku CloudFlare ndipo ndimakhala ku London. Koma ndinenso wosamalira VictoriaMetrics.
Ndipo ndine wolemba za Grafana ndi ndi projekiti yaying'ono ya ClickHouse.
Tiyamba ndi gawo loyamba, lomwe limatchedwa "Zovuta Zomasulira" ndipo m'menemo ndilankhula za mfundo yakuti chinenero chilichonse kapena chinenero cholankhulana ndi chofunika kwambiri. Chifukwa umu ndi momwe mumaperekera malingaliro anu kwa munthu wina kapena dongosolo, momwe mumapangira pempho. Anthu pa intaneti amakangana za chilankhulo chomwe chili chabwino - java kapena china. Kwa ine ndekha, ndinaganiza kuti ndiyenera kusankha molingana ndi ntchitoyo, chifukwa zonsezi ndizokhazikika.
Tiyeni tiyambe kuyambira pachiyambi. Kodi PromQL ndi chiyani? PromQL ndi Prometheus Query Language. Umu ndi momwe timapangira mafunso ku Prometheus kuti tipeze zambiri zanthawi.
Kodi mndandanda wa nthawi ndi chiyani? Kwenikweni, awa ndi magawo atatu.
Izi ndi:
- Kodi tikuyang'ana chiyani?
- Pamene ife tiyang'ana pa izo.
- Ndipo limasonyeza phindu lotani?
Ngati muyang'ana tchatichi (tchatichi chikuchokera ku foni yanga chomwe chikuwonetsa masitepe anga), akhoza kuyankha mafunso awa mwamsanga.
Timayang'ana masitepe. Timawona tanthauzo ndipo timawona nthawi yomwe timayang'ana. Ndiko kuti, poyang'ana chithunzichi, mutha kunena mosavuta kuti Lamlungu ndinayenda masitepe pafupifupi 15. Izi ndi data zanthawi.
Tsopano tiyeni "tigawike" (titembenuzire) kukhala chitsanzo china cha data mu mawonekedwe a tebulo. Pano tilinso ndi zomwe tikuyang'ana. Pano ndinawonjezera deta yowonjezera pang'ono, yomwe tidzatcha meta-data, mwachitsanzo, sindinali ine amene ndinadutsamo, koma anthu awiri, mwachitsanzo, Jay ndi Silent Bob. Izi ndi zomwe tikuyang'ana; chimasonyeza chiyani ndipo chimasonyeza liti mtengo wake.
Tsopano tiyeni tiyese kusunga deta yonseyi mu nkhokwe. Mwachitsanzo, ndinatenga mawu a ClickHouse. Ndipo apa timapanga tebulo limodzi lotchedwa "Masitepe", mwachitsanzo, zomwe tikuyang'ana. Pali nthawi yomwe timayang'ana; zomwe zikuwonetsa ndi meta data komwe tidzasunga kuti ndi ndani: Jay ndi Silent Bob.
Ndipo kuyesa kuwona m'maganizo mwathu zonsezi, tidzagwiritsa ntchito Grafana chifukwa, choyamba, ndi yokongola.
Tidzagwiritsanso ntchito pulogalamu yowonjezerayi. Pali zifukwa ziwiri. Choyamba ndi chifukwa ndinachilemba. Ndipo ndikudziwa ndendende momwe zimavutira kukokera nthawi kuchokera ku ClickHouse kuti muwonetse ku Grafana.
Tiziwonetsa mu Gulu la Graph. Ili ndilo gulu lodziwika kwambiri ku Grafana, lomwe limasonyeza kudalira kwa mtengo pa nthawi, choncho timangofunika magawo awiri okha.
Tiyeni tilembe funso losavuta kwambiri - momwe mungasonyezere ziwerengero za masitepe ku Grafana, kusunga izi mu ClickHouse, patebulo lomwe tidapanga. Ndipo timalemba pempho losavutali. Timasankha masitepe. Timasankha mtengo ndikusankha nthawi yazinthu izi, mwachitsanzo magawo atatu omwe tidakambirana.
Ndipo zotsatira zake, tidzapeza graph ngati iyi. Ndani akudziwa chifukwa chake ali wachilendo?
Ndiko kulondola, tiyenera kusanja ndi nthawi.
Ndipo pamapeto tidzapeza bwino, koma ndondomeko yachilendo. Ndani akudziwa chifukwa chake? Ndiko kulondola, pali awiri omwe atenga nawo mbali, ndipo ife ku Grafana timapereka mndandanda wa nthawi ziwiri, chifukwa ngati muyang'ananso chitsanzo cha deta, ndiye kuti nthawi iliyonse mndandanda ndi kuphatikiza kwapadera kwa dzina ndi zolemba zonse zamtengo wapatali.
Choncho, tiyenera kusankha munthu wapadera. Timasankha Jay.
Ndipo tiyeni tijambulenso. Tsopano graph ikuwoneka ngati chowonadi. Tsopano iyi ndi ndandanda wamba ndipo zonse zikuyenda bwino.
Ndipo mwina mukudziwa momwe mungachitire zomwezo, koma ku Prometheus kudzera pa PromQL. Chinachake chonga ichi. Zosavuta pang'ono. Ndipo tiyeni tiphwanye zonse. Tinatenga Masitepe. Ndipo sefa ndi Jay. Sitikunena pano kuti tifunika kupeza mtengo ndipo sitikusankha nthawi.
Tsopano tiyeni tiyese kuwerengera kuthamanga kwa Jay kapena Silent Bob. Mu ClickHouse tifunika kuchita runningDifference, mwachitsanzo, kuwerengera kusiyana pakati pa mfundo ziwiri ndikuzigawa ndi nthawi kuti tipeze liwiro lenileni. Pempho lidzawoneka chonchi.
Ndipo iwonetsa pafupifupi izi, mwachitsanzo, Silent Bob kapena Jay amatenga pafupifupi masitepe 1,8 pamphindikati.
Ndipo ku Prometheus mukudziwa momwe mungachitire izi. Zosavuta kuposa kale.
Ndipo kuti zikhale zosavuta kuchita ku Grafana, ndidawonjezera chopukutira ichi, chomwe chimawoneka chofanana kwambiri ndi PromQL. Imatchedwa Rate Macros kapena chilichonse chomwe mukufuna kuchitcha. Ku Grafana mumangolemba "mlingo", koma penapake pansi zimasintha kukhala pempho lalikululi. Ndipo simuyeneranso kuyang'ana, kuli kwinakwake, koma mumasunga nthawi yochuluka, chifukwa kulemba mafunso akuluakulu a SQL nthawi zonse kumakhala okwera mtengo. Mutha kulakwitsa mosavuta kenako osamvetsetsa zomwe zikuchitika kwa nthawi yayitali.
Ndipo ili ndi pempho lomwe silinagwirizane ndi slide imodzi ndipo ndinayenera kuigawa m'zaza ziwiri. Ilinso ndi pempho ku ClickHouse, lomwe limapanganso mtengo womwewo, koma pazotsatira zonse ziwiri: Silent Bob ndi Jay, kuti tikhale ndi nthawi ziwiri pagawo. Ndipo izi ndizovuta kale, mwa lingaliro langa.
Ndipo malinga ndi Prometheus idzakhala ndalama (mulingo). Kwa ClickHouse, ndidapanga macro osiyana otchedwa RateColumns, omwe amawoneka ngati funso ku Prometheus.
Tidaziwona ndipo zikuwoneka ngati PromQL ndiyabwino kwambiri, koma ili ndi malire.
Izi ndi:
- Zochepa SINANI.
- Borderline JOINs.
- Palibe thandizo.
Ndipo ngati mwagwira nawo ntchito kwa nthawi yayitali, ndiye kuti mumadziwa kuti nthawi zina zimakhala zovuta kuchita zinthu mu PromQL, koma mu SQL mutha kuchita pafupifupi chilichonse, chifukwa zonse zomwe takambiranazi zitha kuchitika mu SQL. . Koma kodi zingakhale bwino kuzigwiritsa ntchito? Ndipo izi zimandipangitsa kuganiza kuti chilankhulo champhamvu kwambiri sichingakhale chothandiza nthawi zonse.
Choncho, nthawi zina muyenera kusankha chinenero ntchito. Zili ngati Batman akumenyana ndi Superman. Zikuwonekeratu kuti Superman ndi wamphamvu, koma Batman adatha kumugonjetsa chifukwa ndi wothandiza komanso amadziwa zomwe akuchita.
Ndipo gawo lotsatira ndi Extending PromQL.
Apanso za VictoriaMetrics. Kodi VictoriaMetrics ndi chiyani? Iyi ndi nkhokwe yanthawi yayitali, ili mu OpenSource, timagawa mitundu yake imodzi ndi magulu. Malingana ndi zizindikiro zathu, ndi mofulumira kuposa chirichonse chomwe chiri pamsika tsopano ndipo kupanikizika kuli kofanana, mwachitsanzo, anthu enieni amafotokoza kupanikizika kwa pafupifupi 0,4 bytes pa mfundo, pamene Prometheus ndi 1,2-1,4.
Timathandizira kuposa Prometheus. Timathandizira InfluxDB, Graphite, OpenTSDB.
Mutha "kulembera" kwa ife, ndiko kuti, mutha kusamutsa deta yakale.
Ndipo timagwiranso ntchito bwino ndi Prometheus ndi Grafana, mwachitsanzo, timathandizira injini ya PromQL. Ndipo ku Grafana mutha kungosintha Prometheus endpoint kukhala VictoriaMetrics ndipo ma dashboard anu onse azigwira ntchito momwe amachitira.
Koma mutha kugwiritsanso ntchito zina zowonjezera zomwe VictoriaMetrics imapereka.
Tidutsa mwachangu zomwe tawonjezera.
Omit interval param - mutha kusiya magawo apakati ku Grafana. Pamene simukufuna kupeza ma graph achilendo mukamayandikira / kunja mu gulu, tikulimbikitsidwa kugwiritsa ntchito kusintha. $__interval. Uku ndikusintha kwa Grafana mkati ndipo imasankha mtundu wa data womwe. Ndipo VictoriaMetrics palokha imatha kumvetsetsa zomwe izi ziyenera kukhala. Ndipo simuyenera kusintha zopempha zanu zonse. Zidzakhala zosavuta.
Ntchito yachiwiri ndikulozera kwapakati. Mutha kugwiritsa ntchito nthawiyi m'mawu anu. Mutha kuchulukitsa, kugawa, kusamutsa, kulozera kwa izo.
Chotsatira ndi banja la rollup. Ntchito ya Rollup imasintha mndandanda wanu wanthawi zonse kukhala magawo atatu osiyana. Izi ndi min, max ndi avg. Ndimaona kuti izi ndizothandiza kwambiri chifukwa nthawi zina zimatha kuwonetsa zina zakunja ndi zolakwika.
Ndipo ngati mukungokwiya kapena kuwongolera, ndiye kuti mudzaphonya nthawi zina pomwe mndandanda wanthawi sizikuyenda momwe mumayembekezera. Ndi ntchitoyi ndizosavuta kuwona, tinene kuti max ndiambiri kuchokera ku avg.
Chotsatira ndicho kusintha kosasintha. Zosasintha - izi zikutanthauza mtengo womwe tikufunika kujambula ku Grafana ngati tilibe mndandanda wanthawi pakadali pano. Kodi izi zimachitika liti? Tiyerekeze kuti mukutumiza zolakwika zina. Ndipo muli ndi pulogalamu yabwino kotero kuti mukangoyamba, mulibe zolakwika ndipo mulibe zolakwika kwa maola atatu otsatira kapena tsiku. Ndipo muli ndi ma dashboards omwe amawonetsa ubale kuchokera pa kupambana kupita ku cholakwika. Ndipo sakuwonetsani kalikonse chifukwa mulibe metric yolakwika. Ndipo mosakhazikika mutha kufotokoza chilichonse.
Keep_last_Value - imasunga mtengo womaliza wa metric ngati palibe. Ngati Prometheus sanaipeze mkati mwa mphindi 5 pambuyo pa kukwapula kotsatira, ndiye apa tidzakumbukira mtengo wake wotsiriza ndipo ma graph anu sangaswekenso.
Scrape_interval - ikuwonetsa kangati Prometheus amasonkhanitsa deta pa metric yanu, komanso pafupipafupi. Apa mutha kuwona chiphaso, mwachitsanzo.
Kusintha kwa zilembo ndi chinthu chodziwika bwino. Koma tikuganiza kuti ndizovuta pang'ono chifukwa zimatengera mikangano yonse. Ndipo simuyenera kungokumbukira mikangano 5, komanso kukumbukira kutsatizana kwawo.
Choncho, bwanji osawapanga kukhala osavuta? Ndiko kuti, gawanitseni kukhala ntchito zazing'ono ndi mawu omveka bwino.
Ndipo tsopano gawo losangalatsa. Chifukwa chiyani tikuganiza kuti PromQL yowonjezera? Chifukwa timathandizira Common Table Expressions. Mutha kutsatira QR code (), onani maulalo okhala ndi zitsanzo, kuchokera pabwalo lamasewera, pomwe mutha kuyendetsa mafunso mwachindunji mu VictoriaMetrics osayiyika mu msakatuli chabe.
Ndipo ichi ndi chiyani? Pempho ili pamwambapa ndi lotchuka kwambiri. Ndikuganiza mu dashboard iliyonse m'makampani ambiri mumagwiritsa ntchito fyuluta yomweyo pachilichonse. Nthawi zambiri zimatero. Koma mukafuna kuwonjezera fyuluta yatsopano, muyenera kusintha gulu lililonse, kapena kutsitsa dashboard, tsegulani mu JSON, pezani m'malo, zomwe zimatenganso nthawi. Bwanji osasunga mtengowu muzosintha ndikuzigwiritsanso ntchito? Izi zikuwoneka, mwa lingaliro langa, zosavuta komanso zomveka bwino.
Mwachitsanzo, ndikafunika kusintha zosefera ku Grafana pazopempha zonse, ndipo dashboard ikhoza kukhala yayikulu kapena pangakhale angapo. Ndipo ndingakonde bwanji kuthetsa vutoli ku Grafana?
Ndimathetsa vutoli motere: Ndimapanga commonFilter ndikutanthauzira fyulutayi mmenemo, ndikuigwiritsanso ntchito pofunsanso. Koma ngati muchita zomwezo tsopano, sizigwira ntchito chifukwa Grafana samakulolani kugwiritsa ntchito zosinthika mkati mwamafunso. Ndipo ndi zachilendo pang'ono.
Ndipo kotero ine ndinapanga njira yomwe imakulolani kuti muchite izi. Ndipo ngati muli ndi chidwi kapena mukufuna chinthu choterocho, ndiye chichirikizeni kapena musachikonde ngati simukukonda lingaliro ili.
Zambiri za PromQL yowonjezera. Apa sitikutanthauza kusinthasintha kokha, koma ntchito yonse. Ndipo timachitcha kuti ru (kugwiritsa ntchito zida). Ndipo ntchitoyi imavomereza zothandizira zaulere, malire azinthu ndi fyuluta. Mawuwa akuwoneka kuti ndi osavuta. Ndipo ndizosavuta kugwiritsa ntchito ntchitoyi ndikuwerengera kuchuluka kwa kukumbukira kwaulere komwe tili. Ndiko kuti, kuchuluka kwa kukumbukira komwe tili, komwe kuli malire komanso momwe tingasefe. Zimawoneka zosavuta ngati mutazilemba zonse, kugwiritsa ntchito zosefera zomwezo, chifukwa zitha kukhala funso lalikulu.
Ndipo apa pali chitsanzo cha pempho lalikulu chotere. Zachokera ku dashboard yovomerezeka ya NodeExporter ya Grafana. Koma sindikumvetsa zomwe zikuchitika pano. Izi ndizo, ndithudi, ndikumvetsa ngati muyang'anitsitsa, koma chiwerengero cha mabatani chikhoza kuchepetsa nthawi yomweyo chilimbikitso kuti mumvetse zomwe zikuchitika pano. Ndipo bwanji osapangitsa kuti zikhale zosavuta komanso zomveka bwino?
Mwachitsanzo, monga chonchi, kulekanitsa zinthu zofunika kapena zigawo kukhala zosinthika. Ndiyeno chitani masamu anu oyambirira. Izi zili kale ngati mapulogalamu, izi ndizomwe ndikufuna kuziwona m'tsogolomu ku Grafana.
Nachi chitsanzo chachiwiri cha momwe tingapangire izi kukhala zosavuta ngati tinali ndi ntchito iyi, ndipo ilipo kale ku VictoriaMetrics. Ndipo mumangodutsa mtengo wosungidwa womwe mudalengeza mu CTE.
Ndanena kale za kufunika kogwiritsa ntchito chilankhulo choyenera. Ndipo, mwina, kampani iliyonse ku Grafana ili ndi china chake chomwe chikuchitika. Ndipo mwina mumapatsanso mwayi wofikira ku Grafana kwa omwe akukutukutsani, ndipo omwe akutukula amachita zawozawo. Ndipo iwo onse amachita izo mwanjira ina. Koma ndinkafuna kuti zikhale zofanana, ndiko kuti, kuzichepetsa kukhala zofanana.
Tiyerekeze kuti mulibe mainjiniya okha, mwina muli ndi akatswiri, ma devops kapena SRE. Mwinamwake muli ndi akatswiri omwe amadziwa kuti kuyang'anira ndi chiyani, omwe amadziwa zomwe Grafana ali, ndiye kuti, akhala akugwira nawo ntchito kwa zaka zambiri ndipo amadziwa bwino momwe angachitire. Ndipo adalemba kale izi nthawi 100 ndikuzifotokozera aliyense, koma pazifukwa zina palibe amene amamvera.
Nanga bwanji ngati akanayika chidziwitsochi mwachindunji ku Grafana kuti ogwiritsa ntchito ena agwiritsenso ntchito mawonekedwewo? Ndipo ngati angafunikire kuwerengera kuchuluka kwa kukumbukira kwaulere, amangogwiritsa ntchito ntchitoyi. Nanga bwanji ngati omwe amapanga otumiza kunja, limodzi ndi zinthu zawo, adaperekanso magawo amomwe angagwiritsire ntchito ndi ma metric awo, chifukwa amadziwa bwino lomwe ma metricwa ndi momwe angawerengere molondola?
Izi kulibe kwenikweni. Izi ndi zomwe ndinachita ndekha. Ichi ndi chithandizo cha library ku Grafana. Tinene kuti anyamata omwe adapanga NodeExporter adachita zomwe ndidalankhula. Ndipo adaperekanso gulu la ntchito.
Ndiko kuti, zikuwoneka ngati izi. Mumalumikiza laibulale iyi ku Grafana, mumayamba kusintha ndipo zalembedwa mophweka mu JSON momwe mungagwirire ntchito ndi metric iyi. Ndiko kuti, magawo ena a ntchito, kufotokozera kwawo ndi zomwe amasintha.
Ndikuganiza kuti izi zitha kukhala zothandiza, chifukwa ku Grafana mungalembe motere. Ndipo Grafana "amakuuzani" kuti pali ntchito zina kuchokera ku laibulale yotere - tiyeni tigwiritse ntchito. Ndikuganiza kuti zingakhale bwino kwambiri.
Zambiri za VictoriaMetrics. Timachita zinthu zambiri zosangalatsa. Werengani nkhani zathu zokhudzana ndi kuponderezana, za mpikisano wathu ndi mapulogalamu ena a nthawi, ndondomeko yathu ya momwe tingagwiritsire ntchito ndi PromQL, chifukwa pali oyambitsa ambiri mu izi, komanso za scalability yowongoka komanso kulimbana ndi Thanos.
Mafunso:
Ndiyamba funso langa ndi nkhani yosavuta ya moyo. Nditayamba kugwiritsa ntchito Grafana, ndidalemba funso lokakamiza kwambiri lomwe linali lalitali mizere 5. Chotsatira chake ndi graph yokhutiritsa kwambiri. Ndondomekoyi yatsala pang'ono kuyamba kupanga. Koma poyang'anitsitsa, zidapezeka kuti chithunzichi chikuwonetsa zopanda pake zomwe sizikugwirizana ndi zenizeni, ngakhale kuti ziwerengerozo zimagwera m'kati mwazomwe timayembekezera. Ndipo funso langa. Tili ndi malaibulale, tili ndi ntchito, koma timalemba bwanji mayeso a Grafana? Mwalemba pempho lovuta lomwe chisankho cha bizinesi chimadalira - kuyitanitsa chidebe chenicheni cha ma seva kapena osayitanitsa. Ndipo monga tikudziwira, ntchito iyi yomwe imakoka graph ndi yofanana ndi chowonadi. Zikomo.
Zikomo chifukwa cha funso. Pali magawo awiri. Choyamba, ndimakhala ndi chidwi, kutengera zomwe ndakumana nazo, kuti ogwiritsa ntchito ambiri, akayang'ana ma chart awo, samamvetsetsa zomwe akuwawonetsa. Pazifukwa zina, anthu ndiabwino kwambiri kubwera ndi chowiringula cha vuto lililonse lomwe limapezeka mu ma graph, ngakhale ndi cholakwika mkati mwa ntchito. Ndipo gawo lachiwiri - zikuwoneka kwa ine kuti kugwiritsa ntchito ntchito zoterezi kungakhale njira yabwino kwambiri yothetsera vuto lanu, m'malo moti aliyense wa omwe akutukula adzipangire yekha kukonzekera ndi kupanga zolakwika ndi mwayi wina.
Momwe mungayang'anire?
Kodi kufufuza? Mwina ayi.
Monga mayeso ku Grafana.
Kodi Grafana ndi chiyani? Grafana amamasulira pempholi mwachindunji ku DataSource.
Kuwonjezera pang'ono ku magawo.
Ayi, palibe chomwe chikuwonjezeredwa ku Grafana. Pakhoza kukhala magawo a GET, monga, kunena, sitepe. Sizinatchulidwe momveka bwino, koma mutha kuzilemba, kapena simungathe kuzilemba, koma zimangowonjezeredwa zokha. Simulemba mayeso apa. Sindikuganiza kuti tiyenera kudalira Grafana ngati gwero la chowonadi pano.
Zikomo chifukwa cha lipoti! Zikomo chifukwa chakupanikizana! Munatchulapo kupanga mapu osinthika mu graph, kuti ku Grafana simungagwiritse ntchito kusintha mkati mwa kusintha. Kodi mukudziwa zomwe ndikutanthauza?
Inde.
Izi poyamba zinkandipweteka mutu pamene ndinkafuna kupanga chenjezo ku Grafana. Ndipo pamenepo muyenera kuchita chenjezo kwa wolandira aliyense payekhapayekha. Zomwe mudapangazi, kodi zimagwira ntchito pazidziwitso ku Grafana?
Ngati Grafana sapeza zosinthika mosiyana, ndiye inde, zigwira ntchito. Koma upangiri wanga ndikuti musagwiritse ntchito kuchenjeza ku Grafana konse, ndibwino kugwiritsa ntchito alertmanager.
Inde, ndimagwiritsa ntchito, koma zimangowoneka zosavuta kukhazikitsa ku Grafana, koma zikomo chifukwa cha upangiri!
Source: www.habr.com