ProHoster > Блог > Maamulka > Loki - ururinta log iyadoo la adeegsanayo habka Prometheus

Loki - ururinta log iyadoo la adeegsanayo habka Prometheus

Salaan, dadka deggan Khabrovsk! Habeenka bilawga diiwaangelinta cusub ee koorsada "Dhaqanka iyo qalabka DevOps" Waxaan kuu diyaarinay tarjumaad waxyaabo xiiso leh.

Maqaalkani waa hordhac gaaban oo Loki ah. Mashruuca Loki waxaa taageeray Grafana waxaana loogu talagalay ururinta logyada dhexe (laga soo bilaabo seerfarada ama weelasha).

Dhiirigelinta ugu weyn ee Loki waxay ahayd Prometheus iyadoo fikradda ah adeegsiga hababkiisa maaraynta log:

  • adigoo isticmaalaya calaamado lagu kaydiyo xogta
  • isticmaalka kheyraad yar

Waxaan ku laaban doonaa sida Prometheus u shaqeeyo oo aan ku siin doono tusaalooyin isticmaalkeeda marka la eego macnaha Kubernetes.

Dhowr eray oo ku saabsan Prometheus

Si aad si buuxda u fahamto sida Loki u shaqeeyo, waa muhiim inaad dib u qaado oo aad xasuusato wax yar oo ku saabsan Prometheus.

Mid ka mid ah sifooyinka gaarka ah ee Prometheus waa ka-soo-saarka mitirka ee dhibcaha aruurinta (iyada oo loo sii marayo dhoofiyeyaasha) iyo ku kaydinta TSDB (Saldhigga Xogta Taxanaha Waqtiga), iyadoo lagu daro metadata qaab calaamado ah.

Maxaa loogu baahan yahay

Dhawaan, Prometheus wuxuu noqday heerka dhabta ah ee adduunka weelasha iyo Kubernetes: rakibadeedu waa mid aad u fudud, iyo kooxda Kubernetes waxay la socotaa dhammaadka asalka ah ee Prometheus. Prometheus sidoo kale wuxuu ka soo saari karaa cabbirada codsiyada lagu shubay weelka iyadoo lagu kaydinayo calaamado gaar ah. Sidaa darteed, la socodka codsiga waa mid aad u fudud in la fuliyo.

Nasiib darro, weli ma jiro xal furaha maaraynta log, oo waa inaad adigu xal u heshaa:

  • Adeegga daruuriga ee la maareeyay ee xarun dhexe looggu sameeyay (AWS, Azure ama Google)
  • adeegga la socodka "kormeerka adeeg ahaan" (tusaale ahaan, Datadog)
  • abuurista adeeg ururinta logaga gaarka ah.

Doorashada saddexaad, waxaan dhaqan ahaan u isticmaali jiray Elasticsearch, in kasta oo xaqiiqda ah inaanan had iyo jeer ku faraxsanayn (gaar ahaan culeyskeeda iyo kakanaanta qaabeynta).

Loki waxaa loo qorsheeyay in lagu fududeeyo fulinta iyadoo la raacayo mabaadi'da soo socota:

  • si fudud u bilow
  • ku isticmaal kheyraad yar
  • si madax-bannaan u shaqeeyaan iyada oo aan wax dayactir ah oo gaar ah lahayn
  • u adeega sidii wax dheeraad ah oo ku saabsan Prometheus si uu uga caawiyo baaritaanka cayayaanka

Si kastaba ha ahaatee, fududayntan waxay ku timaadaa kharashka tanaasulyada qaarkood. Mid ka mid ah ma aha in la tix-geliyo waxa ku jira. Sidaa darteed, raadinta qoraalku maaha mid aad waxtar u leh ama qani ah mana ogola tirakoobyada nuxurka qoraalka. Laakiin maadaama Loki uu doonayo inuu u dhigmo grep iyo dhammaystirka Prometheus, tani maahan khasaare.

Baaritaanka Dhacdada

Si aad si fiican u fahamto sababta Loki aanu ugu baahnayn tusmaynta, aynu ku noqono habka baadhista shilka ee ay horumariyayaasha Loki adeegsadeen:

Loki - ururinta log iyadoo la adeegsanayo habka Prometheus
1 Digniin → 2 Dashboard → 3 Adhoc Su'aal → 4 Isku darka Log → 5 Baafin Qaybsan → 6 Hagaajin!
(1 Digniin → 2 Dashboard → 3 Adhoc Su'aal → 4 Isku darka Log → 5 Baafin Qaybsan → 6 Hagaajinta!)

Fikradda ayaa ah inaan helno nooc ka mid ah digniinta (Ogaysiinta caajiska ah, SMS, iwm.) iyo intaas ka dib:

  • fiiri Grafana dashboards
  • eeg cabbirrada adeegga (tusaale ahaan, Prometheus)
  • Eeg gelinta logyada (tusaale, Elasticsearch)
  • laga yaabaa in aad eegto raad qaybsan (Jaeger, Zipkin, iwm.)
  • iyo ugu dambeyntii hagaajinta dhibaatada asalka ah.

Halkan, marka laga hadlayo Grafana + Prometheus + Elasticsearch + Zipkin, waa inaad isticmaashaa afar qalab oo kala duwan. Si loo dhimo wakhtiga, way fiicnaan lahayd in la awoodo in la sameeyo dhammaan tillaabooyinkan iyada oo la adeegsanayo hal qalab: Grafana. Waxaa xusid mudan in habkan cilmi-baarista lagu hirgeliyay Grafana tan iyo nooca 6. Sidaa darteed, waxay suurtogal u tahay in si toos ah xogta Prometheus laga helo Grafana.

Loki - ururinta log iyadoo la adeegsanayo habka Prometheus
Shaashadda Explorer ayaa kala qaybsantay Prometheus iyo Loki

Shaashadan waxaad ka arki kartaa Loki Loki ee la xidhiidha metrics Prometheus adoo isticmaalaya fikradda shaashadda kala qaybsan. Laga soo bilaabo nooca 6.5, Grafana waxay kuu ogolaanaysaa inaad kala qaybiso aqoonsiga raadraaca ee Loki Loki logins si aad u raacdo xiriirinta aaladaha baafinta ee aad ugu jeceshahay (Jaeger).

Imtixaanka Loki ee deegaanka

Sida ugu fudud ee Loki lagu tijaabiyo gudaha waa in la isticmaalo docker-compose. Faylka docker-compose wuxuu ku yaalaa kaydka Loki. Waxaad ku heli kartaa kaydka adoo isticmaalaya amarka soo socda git:

$ git clone https://github.com/grafana/loki.git

Markaa waxaad u baahan tahay inaad aado tusaha wax soo saarka:

$ cd production

Tan ka dib waxaad heli kartaa muuqaalkii ugu dambeeyay ee sawirada Docker:

$ docker-compose pull

Ugu dambeyntii, xirmada Loki waxaa lagu bilaabay amarka soo socda:

$ docker-compose up

Loki architecture

Halkan waxaa ah jaantus yar oo leh qaab dhismeedka Loki:

Loki - ururinta log iyadoo la adeegsanayo habka Prometheus
Loki Architecture Principles

Macmiilka webka wuxuu ku shaqeeyaa arjiyada server-ka, Promtail wuxuu aruuriyaa logu wuxuuna u diraa Loki, macmiilka webka wuxuu kaloo u soo diraa xogta badan Loki. Loki wax walba ayuu isu geeyaa oo u gudbiyaa Grafana.
Loki waa la bilaabay. Si aad u aragto qaybaha la heli karo, socodsii amarka soo socda:

$ docker ps

Xaaladda Docker cusub oo la rakibay, amarku waa inuu soo celiyaa natiijada soo socota:

IMAGE               PORTS                  NAMES
grafana/promtail:                          production_promtail_1
grafana/grafana: m  0.0.0.0:3000->3000/tcp production_grafana_1
grafana/loki: late  80/tcp,0.0.0.0:3100... production_loki_1

Waxaan aragnaa qaybaha soo socda:

  • Promtail: wakiilka mas'uul ka ah dhexeynta logu
  • Grafana: qalab dashboard caan ah
  • Loki: Data Centralization Daemon

Iyada oo qayb ka ah kaabayaasha caadiga ah (tusaale, ku salaysan mashiinnada farsamada gacanta), wakiilka Promtail waa in lagu rakibaa mishiin kasta. Grafana iyo Loki waxaa lagu rakibi karaa isla mashiinka.

Loo dirayo Kubernetes

Ku rakibida qaybaha Loki ee Kubernetes waxay noqon doontaa sida soo socota:

  • daemonSet si loo geeyo wakiilka Promtail mid kasta oo ka mid ah mashiinada ku jira kooxda serverka
  • Gelida Loki
  • kan ugu dambeeyana waa hawlgelinta Grafana.

Nasiib wanaag, Loki waxaa loo heli karaa sidii xirmo Helm ah, taasoo sahlaysa in la geeyo.

Ku rakibida iyada oo loo marayo Heml

Waa inaad mar hore ku rakibtay Heml. Waxaa laga soo dejisan karaa kaydka GitHub ee mashruuca. Waxa lagu rakibay iyada oo la furayo kaydka u dhigma dhismahaaga oo lagu daro koofiyadda $PATH.

Fiiro gaar ah: nooca 3.0.0 ee Helm ayaa la sii daayay dhawaan. Maadaama ay jiraan isbeddelo badan oo ku yimid, akhristaha waxaa lagula talinayaa in uu sugo xoogaa ka hor intaadan isticmaalin..

Ku darida isha Helm

Talaabada ugu horeysa waa in lagu daro kaydka "loki" adoo isticmaalaya amarka soo socda:

$ helm add loki https://grafana.github.io/loki/charts

Taas ka dib, waxaad raadin kartaa baakadaha lagu magacaabo "loki":

$ helm search loki

Natiijada:

loki/loki       0.17.2 v0.4.0 Loki: like Prometheus, but for logs.
loki/loki-stack 0.19.1 v0.4.0 Loki: like Prometheus, but for logs.
loki/fluent-bit 0.0.2  v0.0.1 Uses fluent-bit Loki go plugin for...
loki/promtail   0.13.1 v0.4.0 Responsible for gathering logs and...

Xirmooyinkani waxay leeyihiin sifooyinka soo socda:

  • xirmada loki/loki u dhigma server-ka Loki kaliya
  • xirmada loki/filan-bit wuxuu kuu ogolaanayaa inaad geyso DaemonSet adoo isticmaalaya si fiican u-biin si aad u soo ururiso diiwaanka halkii aad ka isticmaali lahayd Promtail
  • xirmada loki/promtail waxaa ku jira wakiilka aruurinta faylka log
  • xirmada loki/loki-xirmo, wuxuu kuu ogolaanayaa inaad isla markiiba geyso Loki si wadajir ah Promtail.

Ku rakibida Loki

Si aad Loki u dirto Kubernetes, ku socodsii amarkan soo socda meesha magaca “kormeerka”:

$ helm upgrade --install loki loki/loki-stack --namespace monitoring

Si aad u kaydiso saxanka, ku dar cabbirka --set loki.persistence.enabled = true:

$ helm upgrade --install loki loki/loki-stack 
              --namespace monitoring 
              --set loki.persistence.enabled=true

Fiiro gaar ah: Haddii aad rabto in aad geyso Grafana isku mar, ka dibna ku dar parameterka --set grafana.enabled = true

Markaad socodsiiso amarkan waa inaad heshaa soo-saarka soo socda:

LAST DEPLOYED: Tue Nov 19 15:56:54 2019
NAMESPACE: monitoring
STATUS: DEPLOYED
RESOURCES:
==> v1/ClusterRole
NAME AGE
loki-promtail-clusterrole 189d
…
NOTES:
The Loki stack has been deployed to your cluster. Loki can now be added as a datasource in Grafana.
See <a href="http://docs.grafana.org/features/datasources/loki/">http://docs.grafana.org/features/datasources/loki/</a> for more details.

Markaan eegno xaaladda boodhadhka ee goobta magaca "kormeerka", waxaan arki karnaa in wax walba la geeyay:

$ kubectl -n monitoring get pods -l release=loki

Natiijada:

NAME                 READY  STATUS   RESTARTS  AGE
loki-0               1/1    Running  0         147m
loki-promtail-9zjvc  1/1    Running  0         3h25m
loki-promtail-f6brf  1/1    Running  0         11h
loki-promtail-hdcj7  1/1    Running  0         3h23m
loki-promtail-jbqhc  1/1    Running  0         11h
loki-promtail-mj642  1/1    Running  0         62m
loki-promtail-nm64g  1/1    Running  0         24m

Dhammaan boodhadhku way socdaan. Hadda waa waqtigii la samayn lahaa baaritaanno!

Ku xidhida Grafana

Si aad ugu xidho Grafana oo hoos timaada Kubernetes, waxaad u baahan tahay inaad u furto tunnel godkeeda. Hoos waxaa ah amarka lagu furayo dekedda 3000 ee Grafana pod:

$ kubectl -n port-forward monitoring svc/loki-grafana 3000:80

Qodob kale oo muhiim ah ayaa ah baahida loo qabo soo celinta erayga sirta ah ee maamulaha Grafana. Furaha sirta ah ayaa loo hayaa loki-grafana duurka ku jira .data.admin-user qaab base64 ah.

Si loo soo celiyo, waxaad u baahan tahay inaad socodsiiso amarka soo socda:

$ kubectl -n monitoring get secret loki-grafana 
 --template '{{index .data "admin-password" | base64decode}}'; echo

Isticmaal erayga sirta ah ee la socda koontada maamulaha ee caadiga ah (admin).

Qeexida isha xogta Loki ee Grafana

Marka hore, hubi in isha xogta Loki la abuuray
Halkan waxaa ah tusaale:

Loki - ururinta log iyadoo la adeegsanayo habka Prometheus
Tusaale ahaan dejinta isha xogta Loki

Markaad gujiso "Tijaabi" waxaad hubin kartaa xidhiidhka Loki.

Codsiyada Loki

Hadda u gudub Grafana qaybta "Explore". Markaad ka helayso diiwaannada weelasha, Loki wuxuu ku darayaa xogta badan ee Kubernetes. Sidaas awgeed, waxay noqonaysaa suurtogal in la eego diiwaannada weel gaar ah.

Tusaale ahaan, si aad u doorato diiwaannada weelka promtail, waxaad isticmaali kartaa su'aalaha soo socda: {container_name = "promtail"}.
Halkan sidoo kale xasuuso inaad doorato isha xogta Loki.

Weydiintani waxay ku soo celin doontaa hawsha weelka sida soo socota:

Loki - ururinta log iyadoo la adeegsanayo habka Prometheus
Natiijada weydiinta Grafana

Ku dar dashboard-ka

Laga bilaabo Grafana 6.4, waxaad si toos ah ugu dhejin kartaa macluumaadka log ee dashboard-ka. Taas ka dib, isticmaaluhu wuxuu awood u yeelan doonaa inuu si dhakhso ah u kala beddelo inta u dhaxaysa tirada codsiyada boggiisa iyo raadraaca codsiga.

Hoos waxaa ku yaal tusaale dashboard-ka oo hirgeliya is-dhexgalkan:

Loki - ururinta log iyadoo la adeegsanayo habka Prometheus
Tusaalaha dashboard-ka leh Prometheus metrics iyo Loki logs

Mustaqbalka Loki

Waxaan bilaabay isticmaalka Loki May/June oo wata nooca 0.1. Maanta nooca 1, iyo xitaa 1.1 iyo 1.2 mar hore ayaa la sii daayay.

Waa in la qiraa in nooca 0.1 aanu ahayn mid deggan oo ku filan. Laakiin 0.3 waxay horeyba u muujisay calaamadaha dhabta ah ee qaan-gaarnimada, iyo noocyada xiga (0.4, ka dibna 1.0) kaliya waxay xoojiyeen aragtidan.

Ka dib 1.0.0, qofna ma heli karo wax cudur daar ah si uusan u isticmaalin qalabkan cajiibka ah.

Horumarin dheeri ah waa inaysan ku saabsan Loki, laakiin waa ku biirinteeda Grafana heer sare ah. Dhab ahaantii, Grafana 6.4 waxay horey u leedahay is dhexgalka wanaagsan ee muraayadaha.

Grafana 6.5, oo dhawaan la sii daayay, ayaa sii wanaajisay is dhexgalkan iyadoo si toos ah u aqoonsanaysa waxyaabaha ku jira qaabka JSON.

Fiidiyowga hoose wuxuu muujinayaa tusaale yar oo habkan ah:

Loki - ururinta log iyadoo la adeegsanayo habka Prometheus
Isticmaalka xadhkaha Loki ee lagu soo bandhigay Grafana

Waxa suurtogal noqonaysa in loo isticmaalo mid ka mid ah goobaha JSON, tusaale ahaan,

  • isku xirka qalabka dibadda
  • shaandhaynta waxa ku jira log

Tusaale ahaan, waxaad riixi kartaa traceId si aad u aaddo Zipkin ama Jaeger.

Sida caadada ah, waxaan rajaynaynaa faallooyinkaaga oo aan kugu martiqaadno webinar furan, halkaas oo aan uga hadli doono sida warshadaha DevOps ay u horumareen inta lagu guda jiro 2019 oo aan ka wada hadalno waddooyinka horumarka ee suurtagalka ah ee 2020.

Source: www.habr.com