ProHoster > Блог > Nchịkwa > Loki - nchịkọta ndekọ site na iji ụzọ Prometheus

Loki - nchịkọta ndekọ site na iji ụzọ Prometheus

Daalụ, ndị bi na Khabrovsk! N'abalị nke mmalite nke ndebanye aha ọhụrụ maka usoro ahụ "omume na ngwaọrụ DevOps" Anyị akwadola ntụgharị asụsụ nke ihe na-atọ ụtọ maka gị.

Edemede a bụ obere mmalite nke Loki. Project Loki nke Grafana kwadoro na ezubere ya na centralized nchịkọta ndekọ (si sava ma ọ bụ containers).

Isi mkpali maka Loki bụ Prometheus n'echiche nke itinye usoro ya na njikwa log:

  • iji akara iji chekwaa data
  • oriri nke ihe onwunwe ole na ole

Anyị ga-alaghachi na ka Prometheus si arụ ọrụ wee nye ụfọdụ ọmụmaatụ nke ojiji ya na ọnọdụ Kubernetes.

Okwu ole na ole gbasara Prometheus

Iji ghọta nke ọma ka Loki si arụ ọrụ, ọ dị mkpa iwere nzọụkwụ azụ wee cheta ntakịrị gbasara Prometheus.

Otu n'ime ihe dị iche iche nke Prometheus bụ mwepụta nke metrics site na nchịkọta nchịkọta (site na ndị na-ebupụ) na ịchekwa ha na TSDB (Time Series Data Base), yana mgbakwunye nke metadata n'ụdị akara.

Kedu ihe kpatara o ji dị mkpa

N'oge na-adịbeghị anya, Prometheus aghọwo ọkọlọtọ de facto na ụwa nke arịa na Kubernetes: ntinye ya dị nnọọ mfe, na ụyọkọ Kubernetes na-abịa na njedebe njedebe maka Prometheus. Prometheus nwekwara ike wepụ metrics na ngwa etinyere n'ime akpa ka ọ na-echekwa akara ụfọdụ. Ya mere, ngwa nlekota bụ nnọọ mfe mejuputa.

N'ụzọ dị mwute, enweghị ngwọta ntụgharị maka njikwa log, ị ga-achọtarịrị ihe ngwọta maka onwe gị:

  • Ọrụ igwe ojii ejirila maka ịhazi ndekọ (AWS, Azure ma ọ bụ Google)
  • Ọrụ nlekota oru "nleba anya dị ka ọrụ" (dịka ọmụmaatụ, Datadog)
  • ịmepụta ọrụ nchịkọta log nke gị.

Maka nhọrọ nke atọ, ejiri m Elasticsearch na-emekarị, n'agbanyeghị na anaghị m enwe obi ụtọ na ya (karịsịa ịdị arọ ya na mgbagwoju anya nke nhazi).

Emebere Loki iji mee ka mmejuputa ya dị mfe dịka ụkpụrụ ndị a si dị:

  • dị mfe ịmalite
  • eri ihe ole na ole
  • na-arụ ọrụ n'adabereghị na ya na-enweghị mmezi pụrụ iche
  • jee ozi dị ka mgbakwunye na Prometheus iji nyere aka na nyocha ahụhụ

Otú ọ dị, nke a dị mfe na-abịa na-efu nke ụfọdụ nkwekọrịta. Otu n'ime ha abụghị ịkọwapụta ọdịnaya. Ya mere, ọchụchọ ederede adịghị arụ ọrụ nke ọma ma ọ bụ ọgaranya yana ọ naghị ekwe ka ọnụ ọgụgụ dị na ọdịnaya nke ederede. Mana ebe ọ bụ na Loki chọrọ ka ọ bụrụ grep na ihe nkwado na Prometheus, nke a abụghị ihe ọjọọ.

Nchọpụta ihe merenụ

Iji ghọta nke ọma ihe kpatara na Loki anaghị achọ ndenye aha, ka anyị laghachi azụ na usoro nyocha ihe mere ndị mmepe Loki ji:

Loki - nchịkọta ndekọ site na iji ụzọ Prometheus
1 Alert → 2 Dashboard → 3 Adhoc Query → 4 Log Aggregation → 5 Ekesara nchụso → 6 Idozi!
(1 Ịdọ aka ná ntị → 2 Dashboard → 3 Adhoc Query → 4 Log Aggregation → 5 Ekesara ịchụso → 6 Idozi!)

Echiche a bụ na anyị na-enweta ụdị njikere (Slack Notification, SMS, wdg) na mgbe nke ahụ gasịrị:

  • lee anya na grafana dashboards
  • lelee metrik ọrụ (dịka ọmụmaatụ, na Prometheus)
  • lelee ndenye ndekọ (dịka ọmụmaatụ, na Elasticsearch)
  • eleghị anya, lelee akara ndị ekesara (Jaeger, Zipkin, wdg)
  • na n'ikpeazụ idozi nsogbu mbụ ahụ.

N'ebe a, n'ihe banyere Grafana + Prometheus + Elasticsearch + Zipkin stack, ị ga-eji ngwaọrụ anọ dị iche iche. Iji belata oge, ọ ga-adị mma inwe ike iji otu ngwaọrụ rụọ usoro ndị a niile: Grafana. Ọ dị mma ịmara na usoro nyocha a ka etinyere na Grafana kemgbe ụdị 6. Ya mere, ọ ga-ekwe omume ịnweta data Prometheus ozugbo site na Grafana.

Loki - nchịkọta ndekọ site na iji ụzọ Prometheus
Ihuenyo Explorer kewara n'etiti Prometheus na Loki

Site na ihuenyo a ị nwere ike ịlele ndekọ Loki metụtara Prometheus metrics site na iji echiche ihuenyo kewara. Site na ụdị 6.5, Grafana na-enye gị ohere inyocha trace id na Loki log ndenye ka soro njikọ nke ọkacha mmasị gị ngwaọrụ nchụso ekesa (Jaeger).

Nnwale Loki mpaghara

Ụzọ kachasị mfe iji nwalee Loki na mpaghara bụ iji docker-dede. Faịlụ docker-dede dị na ebe nchekwa Loki. Ị nwere ike nweta ebe nchekwa ahụ site na iji iwu a git:

$ git clone https://github.com/grafana/loki.git

Mgbe ahụ ịkwesịrị ịga na ndekọ mmepụta ihe:

$ cd production

Mgbe nke a gasịrị, ị nwere ike nweta ụdị onyonyo Docker kachasị ọhụrụ:

$ docker-compose pull

N'ikpeazụ, a na-eji iwu a malite nchịkọta Loki:

$ docker-compose up

Loki architecture

Nke a bụ obere eserese nwere ihe owuwu Loki:

Loki - nchịkọta ndekọ site na iji ụzọ Prometheus
Ụkpụrụ ihe owuwu Loki

Onye ahịa webụ na-eme ngwa na ihe nkesa, Promtail na-anakọta ndekọ wee ziga ha na Loki, onye ahịa webụ na-ezigakwa metadata na Loki. Loki na-achịkọta ihe niile wee bufee ya na Grafana.
A tọhapụrụ Loki. Ka ilele ihe ndị dịnụ, mee iwu a:

$ docker ps

N'ihe banyere Docker arụnyere ọhụrụ, iwu ahụ kwesịrị iweghachi nsonaazụ a:

IMAGE               PORTS                  NAMES
grafana/promtail:                          production_promtail_1
grafana/grafana: m  0.0.0.0:3000->3000/tcp production_grafana_1
grafana/loki: late  80/tcp,0.0.0.0:3100... production_loki_1

Anyị na-ahụ ihe ndị a:

  • Promtail: onye ọrụ na-ahụ maka ịhazi ndekọ ndekọ
  • Grafana: ngwa dashboard ama ama
  • Loki: Data Centralization Daemon

Dịka akụkụ nke akụrụngwa kpochapụwo (dịka ọmụmaatụ, dabere na igwe mebere), a ga-ebunye onye ọrụ Promtail na igwe ọ bụla. Enwere ike itinye Grafana na Loki n'otu igwe.

Nkwanye na Kubernetes

Ịwụnye akụrụngwa Loki na Kubernetes ga-abụ nke a:

  • daemonSet maka ibuga onye nnọchi anya Promtail na igwe ọ bụla dị na ụyọkọ nkesa
  • Nkwanye Loki
  • na nke ikpeazụ bụ mbugharị nke Grafana.

Ọ dabara nke ọma, Loki dị ka ngwugwu Helm, na-eme ka ọ dị mfe ibuga.

Ntinye site na Heml

Ị ga-etinyerịrị Heml. Enwere ike ibudata ya na ebe nchekwa GitHub nke ọrụ ahụ. Awụnyere ya site n'iwepụ akwụkwọ ndekọ aha kwekọrọ na ihe owuwu gị yana ịgbakwunye helm na ya $PATH.

Cheta na: E wepụtara ụdị 3.0.0 nke Helm n'oge na-adịbeghị anya. Ebe ọ bụ na e nwewo ọtụtụ mgbanwe na ya, a na-adụ onye na-agụ ọdụ ka ọ chere ntakịrị tupu iji ya..

Na-agbakwụnye isi iyi maka Helm

Nzọụkwụ mbụ bụ ịgbakwunye ebe nchekwa "loki" site na iji iwu a:

$ helm add loki https://grafana.github.io/loki/charts

Mgbe nke a gasịrị, ị nwere ike ịchọ ngwugwu akpọrọ "loki":

$ helm search loki

Nsonaazụ:

loki/loki       0.17.2 v0.4.0 Loki: like Prometheus, but for logs.
loki/loki-stack 0.19.1 v0.4.0 Loki: like Prometheus, but for logs.
loki/fluent-bit 0.0.2  v0.0.1 Uses fluent-bit Loki go plugin for...
loki/promtail   0.13.1 v0.4.0 Responsible for gathering logs and...

Ngwungwu ndị a nwere atụmatụ ndị a:

  • ngwugwu loki/loki dakọtara naanị sava Loki
  • ngwugwu loki/fluent-bit na-enye gị ohere ibuga DaemonSet site na iji fluent-bin ịnakọta ndekọ kama Promtail
  • ngwugwu loki/promtail nwere onye na-anakọta faịlụ log
  • ngwugwu loki/loki-nkpokọta, na-enye gị ohere ibuga Loki ozugbo yana Promtail.

Ịwụnye Loki

Iji bufee Loki na Kubernetes, mee iwu a na oghere aha "nleba anya":

$ helm upgrade --install loki loki/loki-stack --namespace monitoring

Ka ịchekwaa na diski, tinye oke --set loki.persistence.enabled = true:

$ helm upgrade --install loki loki/loki-stack 
              --namespace monitoring 
              --set loki.persistence.enabled=true

Cheta na: Ọ bụrụ na ịchọrọ ibugharị Grafana n'otu oge, tinyezie paramita ahụ --set grafana.enabled = true

Mgbe ị na-eme iwu a, ị ga-enweta nsonaazụ a:

LAST DEPLOYED: Tue Nov 19 15:56:54 2019
NAMESPACE: monitoring
STATUS: DEPLOYED
RESOURCES:
==> v1/ClusterRole
NAME AGE
loki-promtail-clusterrole 189d
…
NOTES:
The Loki stack has been deployed to your cluster. Loki can now be added as a datasource in Grafana.
See <a href="http://docs.grafana.org/features/datasources/loki/">http://docs.grafana.org/features/datasources/loki/</a> for more details.

N'ileghachi anya na ọnọdụ nke pọd na oghere aha "nleba anya", anyị nwere ike ịhụ na etinyere ihe niile:

$ kubectl -n monitoring get pods -l release=loki

Nsonaazụ:

NAME                 READY  STATUS   RESTARTS  AGE
loki-0               1/1    Running  0         147m
loki-promtail-9zjvc  1/1    Running  0         3h25m
loki-promtail-f6brf  1/1    Running  0         11h
loki-promtail-hdcj7  1/1    Running  0         3h23m
loki-promtail-jbqhc  1/1    Running  0         11h
loki-promtail-mj642  1/1    Running  0         62m
loki-promtail-nm64g  1/1    Running  0         24m

Pods niile na-agba ọsọ. Ugbu a ọ bụ oge ime ụfọdụ ule!

Jikọọ na Grafana

Iji jikọọ na Grafana n'okpuru Kubernetes, ị ga-emepe oghere na pọd ya. N'okpuru bụ iwu imepe ọdụ ụgbọ mmiri 3000 maka Pod Grafana:

$ kubectl -n port-forward monitoring svc/loki-grafana 3000:80

Isi ihe ọzọ dị mkpa bụ mkpa ị nwetaghachi paswọọdụ nchịkwa nke Grafana. A na-edobe paswọọdụ nzuzo loki-grafana n'ubi .data.admin-user na usoro base64.

Iji weghachi ya, ị ga-agbarịrị iwu a:

$ kubectl -n monitoring get secret loki-grafana 
 --template '{{index .data "admin-password" | base64decode}}'; echo

Jiri okwuntughe a na njikọ akaụntụ nchịkwa (admin).

Na-akọwapụta isi iyi data Loki na Grafana

Nke mbụ, gbaa mbọ hụ na emepụtara isi iyi data Loki (Nhazi/Datasource).
Lee otu ihe atụ:

Loki - nchịkọta ndekọ site na iji ụzọ Prometheus
Ọmụmaatụ nke ịtọlite ​​​​isi iyi data maka Loki

Site na ịpị "Nnwale" ị nwere ike ịlele njikọ na Loki.

Na-eme arịrịọ na Loki

Ugbu a gaa Grafana na ngalaba "Explore". Mgbe ị na-anata ndekọ sitere na arịa, Loki na-agbakwunye metadata sitere na Kubernetes. Ya mere, ọ ga-ekwe omume ịlele ndekọ nke otu akpa.

Ọmụmaatụ, ka ịhọrọ ndekọ akpa ahịa promtail, ị nwere ike iji ajụjụ a: {container_name = "promtail"}.
Ebe a chetakwa ịhọrọ ebe data Loki.

Ajụjụ a ga-eweghachi ọrụ akpa dịka ndị a:

Loki - nchịkọta ndekọ site na iji ụzọ Prometheus
Nsonaazụ ajụjụ na Grafana

Tinye na dashboard

Malite na Grafana 6.4, ị nwere ike idowe ozi ndekọ ozugbo na dashboard. Mgbe nke a gasịrị, onye ọrụ ga-enwe ike ịgbanwe ngwa ngwa n'etiti ọnụ ọgụgụ nke arịrịọ na saịtị ya na ngwa ngwa.

N'okpuru bụ ọmụmaatụ nke dashboard na-arụ ọrụ mmekọrịta a:

Loki - nchịkọta ndekọ site na iji ụzọ Prometheus
Sample dashboard nwere Prometheus metrics na Loki logs

Ọdịnihu nke Loki

Amalitere m iji Loki laa azụ na Mee/June nwere ụdị 0.1. Taa ụdị 1, na ọbụna 1.1 na 1.2 ewepụtalarị.

Ekwesịrị ịkwenye na ụdị 0.1 akwụsịbeghị nke ọma. Ma 0.3 egosilarị ezigbo ihe ịrịba ama nke ntozu okè, na nsụgharị ndị na-esote (0.4, mgbe ahụ 1.0) na-ewusi echiche a ike.

Mgbe 1.0.0 gasịrị, ọ dịghị onye nwere ike inwe ihe ngọpụ ọ bụla ịghara iji ngwá ọrụ a magburu onwe ya.

Nkwalite ọzọ ekwesịghị ịbụ maka Loki, kama ọ bụ njikọ ya na Grafana mara mma. N'ezie, Grafana 6.4 enweelarị njikọ dị mma na dashboards.

Grafana 6.5, nke ewepụtara n'oge na-adịbeghị anya, na-eme ka njikọ a dịkwuo mma site n'ịghọta ọdịnaya ndekọ na-akpaghị aka na usoro JSON.

Vidiyo dị n'okpuru na-egosi obere ihe atụ nke usoro a:

Loki - nchịkọta ndekọ site na iji ụzọ Prometheus
Iji eriri Loki kpughere na Grafana

Ọ ga-ekwe omume iji otu n'ime mpaghara JSON, dịka ọmụmaatụ, maka:

  • njikọ na ngwá ọrụ mpụga
  • nzacha ndekọ ọdịnaya

Dịka ọmụmaatụ, ị nwere ike pịa traceId ka ịga na Zipkin ma ọ bụ Jaeger.

Dị ka ọ dị na mbụ, anyị na-atụ anya okwu gị ma kpọọ gị òkù ka ị bịa mepee webinar, ebe anyị ga-ekwu maka otu ụlọ ọrụ DevOps si etolite n'oge 2019 wee kparịta ụzọ mmepe maka 2020.

isi: www.habr.com