Lumelisang, baahi ba Khabrovsk! Bosiung ba ho qala ho ngolisoa ho hocha bakeng sa thupelo Re u lokiselitse phetolelo ea boitsebiso bo thahasellisang bakeng sa hau.
Sengoliloeng sena ke kenyelletso e khuts'oane ea Loki. Morero oa Loki 'me e reretsoe ho bokella li-logs tse bohareng (ho tsoa ho li-server kapa lijana).
Tšusumetso e kholo ea Loki e ne e le ka mohopolo oa ho sebelisa mekhoa ea hae ea taolo ea log:
- ho sebelisa lileibole ho boloka data
- tšebeliso ea lisebelisoa tse fokolang
Re tla khutlela ho kamoo Prometheus e sebetsang kateng 'me re fane ka mehlala ea tšebeliso ea eona ho latela moelelo oa Kubernetes.
Mantsoe a seng makae ka Prometheus
Ho utloisisa ka botlalo hore na Loki o sebetsa joang, ho bohlokoa ho khutlela morao 'me u hopole hanyane ka Prometheus.
E 'ngoe ea likarolo tse ikhethang tsa Prometheus ke ho ntšoa ha metrics ho tsoa libakeng tsa pokello (ka barekisi ba kantle ho naha) le ho li boloka ho TSDB (Time Series Data Base), ka tlatsetso ea metadata ka mokhoa oa lileibole.
Hobaneng e hlokahala
Haufinyane tjena, Prometheus e se e le maemo a lefats'e la lijana le Kubernetes: ho kengoa ha eona ho bonolo haholo, 'me sehlopha sa Kubernetes se tla le pheletso ea tlhaho ea Prometheus. Prometheus e ka boela ea ntša metrics ho tsoa lits'ebetsong tse kentsoeng ka har'a setshelo ha e ntse e boloka lileibole tse itseng. Ka hona, ts'ebeliso ea ts'ebeliso e bonolo haholo ho e sebelisa.
Ka bomalimabe, ha ho na tharollo ea turnkey bakeng sa taolo ea log, 'me u tlameha ho iphumanela tharollo:
- ts'ebeletso e laoloang ea leru bakeng sa ho kenya li-log (AWS, Azure kapa Google)
- tšebeletso ea ho beha leihlo "ho beha leihlo joalo ka ts'ebeletso" (mohlala, Datadog)
- ho iketsetsa tšebeletso ea pokello ea li-log.
Bakeng sa khetho ea boraro, ke ne ke tloaetse ho sebelisa Elasticsearch, leha ke ne ke sa thabele eona kamehla (haholo-holo boima ba eona le ho rarahana ha eona).
Loki e etselitsoe ho nolofatsa ts'ebetsong ho latela melao-motheo e latelang:
- ho be bonolo ho qala
- sebelisa lisebelisoa tse fokolang
- sebetsa ka boikemelo ntle le tlhokomelo e khethehileng
- sebetsa joalo ka tlatsetso ho Prometheus ho thusa ka lipatlisiso tsa likokoana-hloko
Leha ho le joalo, mokhoa ona o bonolo o tla ka litšenyehelo tsa ho sekisetsa ho itseng. E 'ngoe ea tsona ke ho se ngolise litaba. Ka hona, patlisiso ea mongolo ha e sebetse hantle kapa e ruile ebile ha e lumelle lipalo-palo tsa litaba tsa mongolo. Empa kaha Loki o batla ho lekana le grep le motlatsi oa Prometheus, sena ha se phoso.
Patlisiso ea Ketsahalo
Ho utloisisa hamolemo hore na ke hobane'ng ha Loki a sa hloke indexing, ha re khutleleng mokhoeng oa lipatlisiso oa liketsahalo oo baetsi ba Loki ba o sebelisitseng:
1 Tlhokomeliso → 2 Dashboard → 3 Adhoc Potso → 4 Log Aggregation → 5 Distributed Tracing → 6 Lokisa!
(Temoso e 1 → 2 Dashboard → 3 Adhoc Query → 4 Log Aggregation → 5 Distributed Tracing → 6 Lokisa!)
Taba ke hore re fumana tlhokomeliso ea mofuta o itseng (Tsebiso ea Slack, SMS, joalo-joalo) mme kamora moo:
- sheba li-dashboard tsa Grafana
- sheba metrics ea litšebeletso (mohlala, ho Prometheus)
- sheba likenyo tsa log (mohlala, ho Elasticsearch)
- mohlomong u ka sheba mesaletsa e ajoang (Jaeger, Zipkin, joalo-joalo)
- mme qetellong ho lokisa bothata ba pele.
Mona, tabeng ea Grafana + Prometheus + Elasticsearch + Zipkin stack, u tla tlameha ho sebelisa lisebelisoa tse 'nè tse fapaneng. Ho fokotsa nako, ho ka ba monate ho khona ho etsa mehato ena kaofela ka sesebelisoa se le seng: Grafana. Ke habohlokoa ho hlokomela hore mokhoa ona oa ho etsa lipatlisiso o sebelisitsoe Grafana ho tloha phetolelong ea 6. Kahoo, hoa khoneha ho fumana lintlha tsa Prometheus ka ho toba ho tloha Grafana.
Ho aroloa ha skrine ea Explorer pakeng tsa Prometheus le Loki
Ho tloha skrineng sena u ka sheba li-log tsa Loki tse amanang le metrics ea Prometheus u sebelisa mohopolo oa skrine o arohaneng. Ho tloha ka mofuta oa 6.5, Grafana e u fa monyetla oa ho hlahlobisisa id ea trace ho li-log tsa Loki ho latela likhokahano tsa lisebelisoa tseo u li ratang haholo tsa ho latella (Jaeger).
Teko ea sebaka sa Loki
Mokhoa o bonolo oa ho leka Loki sebakeng sa heno ke ho sebelisa docker-compose. Faele ea "docker-compose" e fumaneha sebakeng sa polokelo ea Loki. U ka fumana polokelo u sebelisa taelo e latelang git:
$ git clone https://github.com/grafana/loki.gitEbe o hloka ho ea bukeng ea tlhahiso:
$ cd productionKamora sena o ka fumana mofuta oa morao-rao oa litšoantšo tsa Docker:
$ docker-compose pullQetellong, stack ea Loki e qalisoa ka taelo e latelang:
$ docker-compose upMehaho ea Loki
Mona ke setšoantšo se senyenyane se nang le meralo ea Loki:
Loki Architecture Melao-motheo
Moreki oa marang-rang o tsamaisa lits'ebetso ho seva, Promtail e bokella lits'oants'o ebe e li romella ho Loki, moreki oa webo o boetse o romella metadata ho Loki. Loki o kopanya ntho e 'ngoe le e' ngoe ebe o e fetisetsa Grafana.
Loki e qala. Ho bona likarolo tse fumanehang, tsamaisa taelo e latelang:
$ docker psTabeng ea Docker e sa tsoa kenngoa, taelo e lokela ho khutlisa sephetho se latelang:
IMAGE PORTS NAMES
grafana/promtail: production_promtail_1
grafana/grafana: m 0.0.0.0:3000->3000/tcp production_grafana_1
grafana/loki: late 80/tcp,0.0.0.0:3100... production_loki_1Re bona likarolo tse latelang:
- Promtail: moemeli ea ikarabellang bakeng sa ho kenya likutung bohareng
- Grafana: sesebelisoa se tsebahalang sa dashboard
- Loki: Data Centralization Daemon
E le karolo ea lisebelisoa tsa khale (mohlala, tse ipapisitseng le mechini ea sebele), moemeli oa Promtail o tlameha ho sebelisoa mochining o mong le o mong. Grafana le Loki li ka kenngoa mochining o le mong.
Ho tsamaisoa ho Kubernetes
Ho kenya likarolo tsa Loki ho Kubernetes ho tla ba ka tsela e latelang:
- daemonSet bakeng sa ho tsamaisa moemeli oa Promtail mochining o mong le o mong o ka har'a sehlopha sa seva
- Deployment Loki
- 'me ea ho qetela ke ho romelloa ha Grafana.
Ka lehlohonolo, Loki e fumaneha e le sephutheloana sa Helm, se etsang hore ho be bonolo ho se tsamaisa.
Ho kenya ka Heml
U tlameha ho ba le Heml e kentsoeng. E ka khoasolloa sebakeng sa polokelo ea GitHub ea projeke. E kentsoe ka ho notlolla polokelo ea litaba e tsamaellanang le meaho ea hau le ho eketsa helm ho $PATH.
Ela hloko: mofuta oa 3.0.0 oa Helm o ile oa lokolloa haufinyane. Kaha ho bile le liphetoho tse ngata ho eona, 'mali o eletsoa ho ema hanyane pele a e sebelisa..
Ho eketsa mohloli bakeng sa Helm
Mohato oa pele ke ho eketsa polokelo ea "loki" u sebelisa taelo e latelang:
$ helm add loki https://grafana.github.io/loki/chartsKamora moo, o ka batla liphutheloana tse bitsoang "loki":
$ helm search lokiSephetho:
loki/loki 0.17.2 v0.4.0 Loki: like Prometheus, but for logs.
loki/loki-stack 0.19.1 v0.4.0 Loki: like Prometheus, but for logs.
loki/fluent-bit 0.0.2 v0.0.1 Uses fluent-bit Loki go plugin for...
loki/promtail 0.13.1 v0.4.0 Responsible for gathering logs and...Liphutheloana tsena li na le likarolo tse latelang:
- sephutheloana loki/loki e tsamaisana le seva sa Loki feela
- sephutheloana loki/fluent-bit e o lumella ho tsamaisa DaemonSet o sebelisa bin e bonolo ho bokella li-log ho fapana le Promtail
- sephutheloana loki/promtail e na le moemeli oa pokello ea lifaele tsa log
- sephutheloana loki/loki-stack, e u lumella ho tsamaisa Loki hang-hang hammoho le Promtail.
Ho kenya Loki
Ho isa Loki ho Kubernetes, tsamaisa taelo e latelang sebakeng sa mabitso sa "monitoring":
$ helm upgrade --install loki loki/loki-stack --namespace monitoringHo boloka ho disk, eketsa parameter --set loki.persistence.enabled = true:
$ helm upgrade --install loki loki/loki-stack
--namespace monitoring
--set loki.persistence.enabled=trueEla hloko: haeba u batla ho kenya Grafana ka nako e le 'ngoe, joale eketsa parameter
--set grafana.enabled = true
Ha o tsamaisa taelo ena o lokela ho fumana tlhahiso e latelang:
LAST DEPLOYED: Tue Nov 19 15:56:54 2019
NAMESPACE: monitoring
STATUS: DEPLOYED
RESOURCES:
==> v1/ClusterRole
NAME AGE
loki-promtail-clusterrole 189d
…
NOTES:
The Loki stack has been deployed to your cluster. Loki can now be added as a datasource in Grafana.
See <a href="http://docs.grafana.org/features/datasources/loki/">http://docs.grafana.org/features/datasources/loki/</a> for more details.Ha re sheba boemo ba li-pods sebakeng sa mabitso sa "ho beha leihlo", re ka bona hore ntho e ngoe le e ngoe e kentsoe:
$ kubectl -n monitoring get pods -l release=lokiSephetho:
NAME READY STATUS RESTARTS AGE
loki-0 1/1 Running 0 147m
loki-promtail-9zjvc 1/1 Running 0 3h25m
loki-promtail-f6brf 1/1 Running 0 11h
loki-promtail-hdcj7 1/1 Running 0 3h23m
loki-promtail-jbqhc 1/1 Running 0 11h
loki-promtail-mj642 1/1 Running 0 62m
loki-promtail-nm64g 1/1 Running 0 24mLi-pods tsohle lia sebetsa. Joale ke nako ea ho etsa liteko!
Ho kopana le Grafana
Ho hokela Grafana tlasa Kubernetes, o hloka ho bula kotopo ho pod ea eona. Ka tlase ke taelo ea ho bula port 3000 bakeng sa Grafana pod:
$ kubectl -n port-forward monitoring svc/loki-grafana 3000:80Ntlha e 'ngoe ea bohlokoa ke tlhoko ea ho khutlisa password ea motsamaisi oa Grafana. Lekunutu le bolokoa e le lekunutu loki-grafana tšimong .data.admin-user ka sebopeho sa base64.
Ho e khutlisa, o hloka ho tsamaisa taelo e latelang:
$ kubectl -n monitoring get secret loki-grafana
--template '{{index .data "admin-password" | base64decode}}'; echoSebelisa phasewete ena hammoho le akhaonto ea kamehla ea molaoli (admin).
Ho hlalosa mohloli oa data oa Loki Grafana
Pele ho tsohle, etsa bonnete ba hore mohloli oa data oa Loki o entsoe (Configuration/Datasource).
Mohlala ke ona:
Mohlala oa ho theha mohloli oa data bakeng sa Loki
Ka ho tobetsa "Test" u ka sheba khokahano le Loki.
Ho etsa likopo ho Loki
Joale e-ea Grafana ho karolo ea "Explore". Ha a amohela li-logs ho tsoa lijaneng, Loki o eketsa metadata ho tsoa ho Kubernetes. Ka hona, hoa khoneha ho sheba likutu tsa setshelo se itseng.
Ka mohlala, ho khetha li-logs tsa promtail, u ka sebelisa potso e latelang: {container_name = "promtail"}.
Mona hape hopola ho khetha mohloli oa data oa Loki.
Potso ena e tla khutlisa tšebetso ea linkho ka tsela e latelang:
Sephetho sa potso ho Grafana
Kenya ho dashboard
Ho qala ka Grafana 6.4, u ka beha tlhahisoleseling ea log ka kotloloho ho dashboard. Kamora sena, mosebelisi o tla khona ho fetola kapele lipakeng tsa palo ea likopo sebakeng sa hae sa marang-rang le lits'ebetso tsa kopo.
Ka tlase ke mohlala oa dashboard e sebelisang puisano ena:
Mohlala oa dashboard o nang le metrics ea Prometheus le li-log tsa Loki
Bokamoso ba Loki
Ke qalile ho sebelisa Loki morao ka May/June ka mofuta oa 0.1. Kajeno mofuta oa 1, esita le 1.1 le 1.2 li se li lokollotsoe.
E tlameha ho lumeloa hore mofuta oa 0.1 o ne o sa tsitsa ka ho lekana. Empa 0.3 e se e bontšitse matšoao a sebele a khōlo, 'me liphetolelo tse latelang (0.4, joale 1.0) li matlafalitse maikutlo ana feela.
Kamora 1.0.0, ha ho motho ea ka bang le lebaka la ho se sebelise sesebelisoa sena se setle.
Lintlafatso tse ling ha lia lokela ho ba mabapi le Loki, empa ho e-na le hoo, ho kopanngoa ha eona le Grafana e babatsehang. Ebile, Grafana 6.4 e se e ntse e na le kopanyo e ntle le li-dashboards.
Grafana 6.5, e lokollotsoeng haufinyane, e ntlafatsa ho kopanngoa hona ka ho tseba litaba tsa log ka mokhoa oa JSON.
Video e ka tlase e bonts'a mohlala o monyane oa mochini ona:
Ho sebelisa likhoele tsa Loki tse pepesitsoeng Grafana
Hoa khoneha ho sebelisa e 'ngoe ea masimo a JSON, mohlala, bakeng sa:
- likhokahano ho sesebelisoa sa kantle
- ho tlhotla dikahare tsa log
Mohlala, o ka tobetsa ho traceId ho ea Zipkin kapa Jaeger.
Joalo ka tloaelo, re lebelletse maikutlo a hau ka thabo 'me re u mema hore u etse joalo , moo re tla bua ka hore na indasteri ea DevOps e ntlafalitse joang nakong ea 2019 le ho buisana ka litsela tse ka khonehang tsa nts'etsopele ea 2020.
Source: www.habr.com