ProHoster > Блог > Gudanarwa > Loki - tarin log ta amfani da tsarin Prometheus

Loki - tarin log ta amfani da tsarin Prometheus

Gaisuwa, mazauna Khabrovsk! A jajibirin fara sabon rajista don kwas "Ayyukan DevOps da kayan aikin" Mun shirya muku fassarar abu mai ban sha'awa.

Wannan labarin gajeriyar gabatarwa ce ga Loki. Project Loki da Grafana kuma an yi niyya ne don tarin rajistan ayyukan (daga sabar ko kwantena).

Babban abin burge Loki shine Prometheus tare da ra'ayin yin amfani da hanyoyinsa don sarrafa log:

  • amfani da labels don adana bayanai
  • amfani da 'yan albarkatu

Za mu koma yadda Prometheus ke aiki kuma mu ba da wasu misalan amfani da shi a cikin mahallin Kubernetes.

'Yan kalmomi game da Prometheus

Don cikakken fahimtar yadda Loki ke aiki, yana da mahimmanci a ɗauki mataki baya kuma ku tuna kaɗan game da Prometheus.

Ɗaya daga cikin keɓancewar fasalin Prometheus shine haɓakar ma'auni daga wuraren tattarawa (ta hanyar masu fitar da kaya) da adana su a cikin TSDB (Time Series Data Base), tare da ƙari na metadata a cikin nau'ikan lakabi.

Me yasa ya zama dole

Kwanan nan, Prometheus ya zama ma'auni na gaskiya a cikin duniyar kwantena da Kubernetes: shigarwar sa yana da sauƙi, kuma gungu na Kubernetes ya zo tare da asalin ƙarshen Prometheus. Prometheus kuma yana iya cire ma'auni daga aikace-aikacen da aka tura a cikin akwati yayin da ake adana takamaiman tambari. Saboda haka, saka idanu aikace-aikace yana da sauƙin aiwatarwa.

Abin takaici, har yanzu ba a sami mafita don sarrafa log ɗin ba, kuma dole ne ku nemo mafita da kanku:

  • sabis na girgije da aka gudanar don daidaita rajistan ayyukan (AWS, Azure ko Google)
  • Sabis na saka idanu "sa idanu azaman sabis" (misali, Datadog)
  • ƙirƙirar sabis ɗin tarin log ɗin ku.

Don zaɓi na uku, na yi amfani da Elasticsearch a al'ada, duk da cewa ba koyaushe nake farin ciki da shi ba (musamman nauyinsa da sarƙar sanyi).

An tsara Loki don sauƙaƙe aiwatarwa bisa ga ka'idoji masu zuwa:

  • zama mai sauki don farawa
  • cinye albarkatu kaɗan
  • aiki da kansa ba tare da wani kulawa na musamman ba
  • yi aiki azaman ƙari ga Prometheus don taimakawa tare da binciken kwaro

Duk da haka, wannan sauƙi yana zuwa ne ta hanyar biyan kuɗi na wasu sasantawa. Ɗayan su shine ba don ba da lissafin abun ciki ba. Don haka, binciken rubutu ba shi da inganci ko wadata kuma baya bada izinin kididdiga akan abubuwan da ke cikin rubutun. Amma tun da Loki yana so ya zama daidai da grep kuma mai dacewa ga Prometheus, wannan ba hasara ba ne.

Binciken Farko

Don ƙarin fahimtar dalilin da yasa Loki baya buƙatar firikwensin, bari mu koma ga hanyar binciken lamarin da masu haɓaka Loki suka yi amfani da su:

Loki - tarin log ta amfani da tsarin Prometheus
1 Faɗakarwa → 2 Dashboard → 3 Adhoc Query → 4 Tarin Log → 5 Rarraba Binciko → 6 Gyara!
(1 Gargaɗi → 2 Dashboard → 3 Adhoc Query → 4 Log Aggregation → 5 Rarraba Binciko → 6 Gyarawa!)

Manufar ita ce cewa muna karɓar wani nau'in faɗakarwa (Slack Notification, SMS, da dai sauransu) kuma bayan haka:

  • duba dashboards na Grafana
  • duba awo na sabis (misali, a cikin Prometheus)
  • duba shigarwar log (misali, a cikin Elasticsearch)
  • kila ku kalli alamun da aka rarraba (Jaeger, Zipkin, da sauransu)
  • kuma a karshe gyara matsalar asali.

Anan, a cikin yanayin Grafana + Prometheus + Elasticsearch + Zipkin stack, dole ne ku yi amfani da kayan aiki daban-daban guda huɗu. Don rage lokaci, zai yi kyau a sami damar yin duk waɗannan matakan ta amfani da kayan aiki guda ɗaya: Grafana. Yana da kyau a lura cewa an aiwatar da wannan hanyar bincike a Grafana tun daga sigar 6. Don haka, yana yiwuwa a sami damar bayanan Prometheus kai tsaye daga Grafana.

Loki - tarin log ta amfani da tsarin Prometheus
Rarraba allon Explorer tsakanin Prometheus da Loki

Daga wannan allon zaku iya duba rajistan ayyukan Loki masu alaƙa da ma'aunin Prometheus ta amfani da ra'ayin tsagaggen allo. Tun daga sigar 6.5, Grafana yana ba ku damar tantance id ɗin a cikin shigarwar log Loki don bin hanyoyin haɗin kai zuwa kayan aikin ganowa da kuka fi so (Jaeger).

Gwajin Loki na gida

Hanya mafi sauƙi don gwada Loki a cikin gida shine amfani da docker-compose. Fayil ɗin docker-compose yana cikin ma'ajiyar Loki. Kuna iya samun ma'ajiyar ta amfani da umarni mai zuwa git:

$ git clone https://github.com/grafana/loki.git

Sa'an nan kana bukatar ka je zuwa samar directory:

$ cd production

Bayan wannan zaku iya samun sabon sigar Hotunan Docker:

$ docker-compose pull

A ƙarshe, an ƙaddamar da tarin Loki tare da umarni mai zuwa:

$ docker-compose up

Loki gine-gine

Ga ƙaramin zane tare da gine-ginen Loki:

Loki - tarin log ta amfani da tsarin Prometheus
Ka'idodin Architecture na Loki

Abokin yanar gizon yana gudanar da aikace-aikace akan sabar, Promtail yana tattara rajistan ayyukan ya aika su zuwa Loki, abokin ciniki na yanar gizo kuma yana aika metadata zuwa Loki. Loki ya tattara komai ya aika zuwa Grafana.
An ƙaddamar da Loki. Don duba abubuwan da aka samu, gudanar da umarni mai zuwa:

$ docker ps

Game da sabon shigar Docker, umarnin yakamata ya dawo da sakamako mai zuwa:

IMAGE               PORTS                  NAMES
grafana/promtail:                          production_promtail_1
grafana/grafana: m  0.0.0.0:3000->3000/tcp production_grafana_1
grafana/loki: late  80/tcp,0.0.0.0:3100... production_loki_1

Muna ganin abubuwa masu zuwa:

  • Promtail: wakili da ke da alhakin daidaita rajistan ayyukan
  • Grafana: sanannen kayan aikin dashboard
  • Loki: Data Centralization Daemon

A matsayin wani ɓangare na kayan aikin yau da kullun (misali, dangane da injunan kama-da-wane), dole ne a tura wakilin Promtail akan kowace na'ura. Ana iya shigar da Grafana da Loki akan na'ura ɗaya.

Aika zuwa Kubernetes

Shigar da abubuwan Loki akan Kubernetes zai kasance kamar haka:

  • daemonSet don tura wakili na Promtail akan kowane injina a cikin gungun uwar garken
  • Ƙaddamar da Loki
  • sannan na karshe shine tura Grafana.

An yi sa'a, Loki yana samuwa azaman kunshin Helm, yana sauƙaƙa turawa.

Shigarwa ta hanyar Heml

Ya kamata ka riga an shigar da Heml. Ana iya sauke shi daga ma'ajin GitHub na aikin. An shigar da shi ta hanyar buɗe kayan tarihin da ke daidai da gine-ginen ku da ƙara helm zuwa $PATH.

Note: An saki sigar 3.0.0 na Helm kwanan nan. Tun da an sami sauye-sauye da yawa a kansa, ana ba mai karatu shawarar ya ɗan jira kafin amfani da shi..

Ƙara tushen don Helm

Mataki na farko shine ƙara ma'ajiyar “loki” ta amfani da umarni mai zuwa:

$ helm add loki https://grafana.github.io/loki/charts

Bayan haka, zaku iya nemo fakiti masu suna "loki":

$ helm search loki

Sakamako:

loki/loki       0.17.2 v0.4.0 Loki: like Prometheus, but for logs.
loki/loki-stack 0.19.1 v0.4.0 Loki: like Prometheus, but for logs.
loki/fluent-bit 0.0.2  v0.0.1 Uses fluent-bit Loki go plugin for...
loki/promtail   0.13.1 v0.4.0 Responsible for gathering logs and...

Waɗannan fakitin suna da fasali masu zuwa:

  • kunshin loki/loki ya dace da uwar garken Loki kawai
  • kunshin loki/sauki-bit yana ba ku damar tura DaemonSet ta amfani da fluent-bin don tattara rajistan ayyukan maimakon Promtail
  • kunshin loki/promtail ya ƙunshi wakilin tattara fayil ɗin log
  • kunshin loki/loki-tari, yana ba ku damar tura Loki nan da nan tare da Promtail.

Sanya Loki

Don tura Loki zuwa Kubernetes, gudanar da umarni mai zuwa a cikin sunan "sa idanu":

$ helm upgrade --install loki loki/loki-stack --namespace monitoring

Don ajiyewa zuwa faifai, ƙara siga --set loki.persistence.enabled = true:

$ helm upgrade --install loki loki/loki-stack 
              --namespace monitoring 
              --set loki.persistence.enabled=true

Note: idan kuna son tura Grafana a lokaci guda, to ƙara ma'auni --set grafana.enabled = true

Lokacin da kuke gudanar da wannan umarni ya kamata ku sami fitarwa mai zuwa:

LAST DEPLOYED: Tue Nov 19 15:56:54 2019
NAMESPACE: monitoring
STATUS: DEPLOYED
RESOURCES:
==> v1/ClusterRole
NAME AGE
loki-promtail-clusterrole 189d
…
NOTES:
The Loki stack has been deployed to your cluster. Loki can now be added as a datasource in Grafana.
See <a href="http://docs.grafana.org/features/datasources/loki/">http://docs.grafana.org/features/datasources/loki/</a> for more details.

Idan muka kalli yanayin kwas ɗin a cikin sunan “sa ido”, zamu iya ganin cewa an tura komai:

$ kubectl -n monitoring get pods -l release=loki

Sakamako:

NAME                 READY  STATUS   RESTARTS  AGE
loki-0               1/1    Running  0         147m
loki-promtail-9zjvc  1/1    Running  0         3h25m
loki-promtail-f6brf  1/1    Running  0         11h
loki-promtail-hdcj7  1/1    Running  0         3h23m
loki-promtail-jbqhc  1/1    Running  0         11h
loki-promtail-mj642  1/1    Running  0         62m
loki-promtail-nm64g  1/1    Running  0         24m

Duk kwas ɗin suna gudana. Yanzu ya yi da za a yi wasu gwaje-gwaje!

Haɗa zuwa Grafana

Don haɗawa zuwa Grafana a ƙarƙashin Kubernetes, kuna buƙatar buɗe rami zuwa kwafsa. Da ke ƙasa akwai umarnin buɗe tashar jiragen ruwa 3000 don kwaf ɗin Grafana:

$ kubectl -n port-forward monitoring svc/loki-grafana 3000:80

Wani muhimmin batu shine buƙatar dawo da kalmar wucewa ta mai gudanarwa na Grafana. Kalmar sirrin sirri ce loki-grafana a cikin filin .data.admin-user a cikin tsarin tushe64.

Don mayar da shi, kuna buƙatar gudanar da umarni mai zuwa:

$ kubectl -n monitoring get secret loki-grafana 
 --template '{{index .data "admin-password" | base64decode}}'; echo

Yi amfani da wannan kalmar sirri tare da tsohuwar asusun mai gudanarwa (admin).

Ƙayyade tushen bayanan Loki a Grafana

Da farko, tabbatar da cewa an ƙirƙiri tushen bayanan Loki (Configuration/Datasource).
Ga misali:

Loki - tarin log ta amfani da tsarin Prometheus
Misali na kafa tushen bayanai don Loki

Ta danna kan "Gwaji" za ku iya duba haɗin gwiwa tare da Loki.

Yin buƙatu zuwa Loki

Yanzu je zuwa Grafana zuwa sashin "Explore". Lokacin karɓar rajistan ayyukan daga kwantena, Loki yana ƙara metadata daga Kubernetes. Don haka, yana yiwuwa a duba rajistan ayyukan wani takamaiman akwati.

Misali, don zaɓar rajistan ayyukan kwantena, zaku iya amfani da tambaya mai zuwa: {container_name = "promtail"}.
Anan kuma ku tuna don zaɓar tushen bayanan Loki.

Wannan tambayar za ta dawo da aikin kwantena kamar haka:

Loki - tarin log ta amfani da tsarin Prometheus
Sakamakon tambaya a Grafana

Ƙara zuwa dashboard

Fara da Grafana 6.4, zaku iya sanya bayanan log kai tsaye akan dashboard. Bayan haka, mai amfani zai iya canzawa da sauri tsakanin adadin buƙatun da ke kan rukunin yanar gizonsa da alamun aikace-aikacen.

A ƙasa akwai misalin dashboard ɗin da ke aiwatar da wannan hulɗar:

Loki - tarin log ta amfani da tsarin Prometheus
Samfurin dashboard tare da ma'aunin Prometheus da Loki logs

Makomar Loki

Na fara amfani da Loki baya a watan Mayu/Yuni tare da sigar 0.1. Yau sigar 1, har ma da 1.1 da 1.2 an riga an sake su.

Dole ne a yarda cewa sigar 0.1 ba ta da ƙarfi sosai. Amma 0.3 ya riga ya nuna alamun balaga, kuma sigogin da suka biyo baya (0.4, sannan 1.0) kawai sun ƙarfafa wannan ra'ayi.

Bayan 1.0.0, babu wanda zai iya samun uzuri don kada ya yi amfani da wannan kayan aiki mai ban mamaki.

Ƙarin haɓakawa bai kamata ya kasance game da Loki ba, amma haɗin kai tare da kyakkyawan Grafana. A zahiri, Grafana 6.4 ya riga ya sami kyakkyawar haɗin kai tare da dashboards.

Grafana 6.5, wanda aka saki kwanan nan, yana ƙara inganta wannan haɗin kai ta hanyar gane abubuwan da ke ciki ta atomatik a cikin tsarin JSON.

Bidiyon da ke ƙasa yana nuna ƙaramin misali na wannan tsarin:

Loki - tarin log ta amfani da tsarin Prometheus
An fallasa amfani da igiyoyin Loki a Grafana

Zai yiwu a yi amfani da ɗayan filayen JSON, misali, don:

  • hanyoyin haɗi zuwa kayan aiki na waje
  • tace abun ciki

Misali, zaku iya danna traceId don zuwa Zipkin ko Jaeger.

Kamar yadda muka saba, muna jiran ra'ayoyin ku kuma muna gayyatar ku zuwa bude webinar, Inda za mu yi magana game da yadda masana'antar DevOps ta haɓaka a lokacin 2019 kuma mu tattauna hanyoyin ci gaba mai yiwuwa don 2020.

source: www.habr.com