Ho bokella lifate ho tsoa Loki

Rona ba Badoo re lula re beha leihlo litheknoloji tse ncha le ho lekola hore na re li sebelise tsamaisong ea rona kapa che. Re batla ho arolelana e 'ngoe ea lithuto tsena le sechaba. E nehetsoe ho Loki, mokhoa oa ho kopanya li-log.

Loki ke tharollo ea ho boloka le ho shebella likutu, 'me stack ena e boetse e fana ka tsamaiso e feto-fetohang bakeng sa ho e hlahloba le ho romela lintlha ho Prometheus. Ka May, ntlafatso e 'ngoe e ile ea lokolloa, e khothalletsoang ka mafolofolo ke baqapi. Re ne re thahasella seo Loki a ka se etsang, hore na e fana ka menyetla efe, le hore na e ka sebetsa ho isa bohōleng bofe e le mokhoa o mong oa ELK, stack eo re e sebelisang hona joale.

Loki ke eng

Grafana Loki ke sete ea likarolo bakeng sa sistimi e felletseng ea ho rema lifate. Ho fapana le lits'ebetso tse ling tse ts'oanang, Loki e ipapisitse le mohopolo oa ho supa metadata ea log feela - lileibole (joalo ka ho Prometheus), le ho hatella lifate ka botsona hore li be likotoana tse arohaneng.

Leqephe la lehae, GitHub

Pele ke kena ho seo u ka se etsang ka Loki, ke batla ho hlakisa se boleloang ke "khopolo ea ho supa metadata feela". Ha re bapise mokhoa oa Loki le mokhoa oa indexing litharollong tsa setso, joalo ka Elasticsearch, re sebelisa mohlala oa mola o tsoang ho log nginx:

172.19.0.4 - - [01/Jun/2020:12:05:03 +0000] "GET /purchase?user_id=75146478&item_id=34234 HTTP/1.1" 500 8102 "-" "Stub_Bot/3.0" "0.001"

Sistimi ea khale e arola mola kaofela, ho kenyeletsoa libaka tse nang le lintlha tse ngata tse ikhethang tsa user_id le item_id, 'me li boloke tsohle ka li-index tse kholo. Molemo oa mokhoa ona ke hore o ka khona ho botsa lipotso tse rarahaneng ka potlako, kaha hoo e batlang e le boitsebiso bohle bo teng ho index. Empa o tlameha ho lefa bakeng sa sena ka hore index e ba kholo, e fetolelang litlhoko tsa mohopolo. Ka lebaka leo, index ea mongolo o felletseng oa li-log e ka bapisoa ka boholo le li-log ka botsona. E le ho batla ka potlako ho eona, index e tlameha ho kenngoa mohopolong. 'Me ha li-log li le ngata, index e eketseha ka potlako' me e sebelisa mohopolo haholoanyane.

Mokhoa oa Loki o hloka hore data e hlokahalang feela e ntšoe khoeleng, palo ea boleng ba eona e nyane. Ka tsela ena re fumana index e nyane mme re ka batlisisa data ka ho e sefa ka nako le masimo a ngolisitsoeng, ebe re hlahloba tse ling kaofela ka mantsoe a tloaelehileng kapa lipatlisiso tse tlase. Ts'ebetso ha e bonahale e potlakile ka ho fetesisa, empa Loki o arola kopo ka likarolo tse 'maloa ebe o li etsa ka ho ts'oana, ho sebetsana le data e ngata ka nako e khuts'oane. Palo ea li-shards le likopo tse tšoanang ho tsona li ka lokisoa; ka hona, palo ea data e ka sebetsoang ka yuniti ea nako e ipapisitse le palo ea lisebelisoa tse fanoeng.

Khoebo ena pakeng tsa index e kholo e potlakileng le index e nyane ea parallel brute-force e lumella Loki ho laola litšenyehelo tsa sistimi. E ka hlophisoa ka mokhoa o bonolo le ho atolosoa ho latela litlhoko tsa hau.

Loki stack e na le likarolo tse tharo: Promtail, Loki, Grafana. Promtail e bokella lits'oants'o, e li sebetsa ebe e li romela ho Loki. Loki oa li boloka. Mme Grafana a ka kopa data ho Loki mme a e bonts'e. Ka kakaretso, Loki e ka sebelisoa eseng feela bakeng sa ho boloka lifate le ho li batla. Sebaka sohle se fana ka menyetla e metle ea ho sebetsana le ho hlahloba lintlha tse kenang ka tsela ea Prometheus.
Tlhaloso ea ts'ebetso ea ho kenya e ka fumanoa mona.

Log Search

U ka batla likutung ka e khethehileng segokanyimmediamentsi sa sebolokigolo Grafana - Explorer. Lipotso li sebelisa puo ea LogQL, e ts'oanang haholo le PromQL e sebelisoang ke Prometheus. Ha e le hantle, e ka nkoa e le grep e abuoang.

Sehokelo sa ho batla se shebahala tjena:

Potso ka boeona e na le likarolo tse peli: selector le sefe. Selector ke patlo ka metadata e nang le indexed (li-labels) e abetsoeng lits'oants'o, 'me filthara ke khoele ea ho batla kapa regexp e sefang lirekoto tse hlalosoang ke mokhethoa. Mohlaleng o fanoeng: Ka har'a masakaneng a curly - mokhethoa, ntho e 'ngoe le e' ngoe ka mor'a - sefa.

{image_name="nginx.promtail.test"} |= "index"

Ka lebaka la tsela eo Loki e sebetsang ka eona, u ke ke ua etsa likopo ntle le mokhethi, empa lileibole li ka etsoa ka mokhoa o akaretsang.

Mokhethi ke bohlokoa-boleng ba boleng ba litšepe tse kobehileng. O ka kopanya bakgethi mme wa hlakisa maemo a fapaneng a patlo o sebedisa =, != basebedisi kapa dipolelo tse tlwaelehileng:

{instance=~"kafka-[23]",name!="kafka-dev"} 
// Найдёт логи с лейблом instance, имеющие значение kafka-2, kafka-3, и исключит dev 

Sesefa ke mongolo kapa regexp e tla sefa lintlha tsohle tse amohetsoeng ke mokhethoa.

Hoa khoneha ho fumana li-graph tsa ad-hoc ho latela data e amohetsoeng ka mokhoa oa metrics. Ka mohlala, u ka fumana hore na ho etsahala hangata hakae ka har'a li-logs tsa nginx tsa keno e nang le khoele ea index:

Tlhaloso e feletseng ea likarolo e ka fumanoa litokomaneng LogQL.

Phatlalatso ea logong

Ho na le mekhoa e mengata ea ho bokella li-log:

• Ka thuso ea Promtail, karolo e tloaelehileng ea stack bakeng sa ho bokella lintlha.
• Ka ho toba ho tloha setshelo sa docker se sebelisa Mokhanni oa ho rema lifate oa Loki Docker.
• Sebelisa Fluentd kapa Fluent Bit e ka romellang data ho Loki. Ho fapana le Promtail, ba na le li-parser tse lokiselitsoeng hoo e batlang e le mofuta ofe kapa ofe oa logong 'me li khona ho sebetsana le lits'oants'o tse ngata.

Hangata Promtail e sebelisetsoa ho qhekella. E etsa lintho tse tharo:

• E fumana mehloli ea data.
• Kenya li-label ho tsona.
• E romella data ho Loki.

Hajoale Promtail e khona ho bala lingoloa ho tsoa lifaeleng tsa lehae le ho tsoa ho systemd koranta. E tlameha ho kenngoa mochining o mong le o mong oo likutu li bokelloang ho oona.

Ho na le kopanyo le Kubernetes: Promtail e iphumanela boemo ba sehlopha ka bo eona ka Kubernetes REST API mme e bokella lits'oants'o ho tloha node, tšebeletso kapa pod, hang-hang e romella mangolo a thehiloeng ho metadata ho tloha Kubernetes (lebitso la pod, lebitso la faele, joalo-joalo).

U ka boela ua fanyeha li-labels ho latela data e tsoang ho log u sebelisa Pipeline. Pipeline Promtail e ka ba le mefuta e mene ea mekhahlelo. Lintlha tse ling - ho litokomane tsa molao, hang-hang ke tla hlokomela tse ling tsa li-nuances.

1. Mekhahlelo ea ho parola. Ena ke sethala sa RegEx le JSON. Mothating ona, re ntša data ho tsoa ho li-log ho ea ho seo ho thoeng ke 'mapa o ntšitsoeng. U ka ntša ho JSON ka ho kopitsa masimo ao re a hlokang 'mapeng o ntšitsoeng, kapa ka lipolelo tse tloaelehileng (RegEx), moo lihlopha tse bitsoang li "mapa"' mapeng o nkiloeng. 'Mapa o ntšitsoeng ke polokelo ea bohlokoa, moo senotlolo e leng lebitso la tšimo,' me boleng ke boleng ba eona ho tsoa ho li-log.
2. Fetola mekhahlelo. Sethala sena se na le likhetho tse peli: ho fetola, moo re behang melao ea phetoho, le mohloli - mohloli oa data bakeng sa phetoho ho tloha 'mapeng o ntšitsoeng. Haeba ho se na tšimo e joalo 'mapeng o ntšitsoeng, joale e tla etsoa. Kahoo, hoa khoneha ho etsa lileibole tse sa ipapisang le 'mapa o ntšitsoeng. Mothating ona, re ka theola lintlha tse 'mapeng o ntšitsoeng re sebelisa matla a lekaneng template ea golang. Ho feta moo, re tlameha ho hopola hore 'mapa o ntšitsoeng o tlatsitsoe ka botlalo nakong ea ho bala, e leng se etsang hore ho khonehe, ka mohlala, ho hlahloba boleng ba eona: "{{if .tag}tag value e teng{end}}". Template e ts'ehetsa maemo, loops, le lits'ebetso tse ling tsa likhoele joalo ka Replace le Trim.
3. Mekhahlelo ea tšebetso. Nakong ena, o ka etsa ho hong ka se ntšitsoeng:
   • Etsa lengolo ho tsoa ho data e ntšitsoeng, e tla hlahisoa ke Loki.
   • Fetola kapa beha nako ea ketsahalo ho tsoa ho log.
   • Fetola data (log text) e tla ea ho Loki.
   • Etsa metrics.
4. Mekhahlelo ea ho sefa. Boemo ba papali, moo re ka romellang lirekoto tseo re sa li hlokeng ho /dev/null, kapa re li romelle bakeng sa ts'ebetso e eketsehileng.

Ke sebelisa mohlala oa ho sebetsana le lits'oants'o tse tloaelehileng tsa nginx, ke tla bonts'a hore na u ka bapisa lifate u sebelisa Promtail joang.

Bakeng sa tlhahlobo, ha re nkeng e fetotsoeng ea nginx jwilder/nginx-proxy: alpine image le daemon e nyane e ka ipotsetsang ka HTTP joalo ka moemeli oa nginx. Daemon e na le liphetho tse 'maloa, tseo e ka fanang ka likarabo tsa boholo bo fapaneng, ka maemo a fapaneng a HTTP le ka tieho e fapaneng.

Re tla bokella lifate ho tsoa lijaneng tsa docker, tse ka fumanoang tseleng /var/lib/docker/containers/ / -json.log

Ho docker-compose.yml re theha Promtail mme re hlakisa tsela ea tlhophiso:

promtail:
  image: grafana/promtail:1.4.1
 // ...
 volumes:
   - /var/lib/docker/containers:/var/lib/docker/containers:ro
   - promtail-data:/var/lib/promtail/positions
   - ${PWD}/promtail/docker.yml:/etc/promtail/promtail.yml
 command:
   - '-config.file=/etc/promtail/promtail.yml'
 // ...

Eketsa tsela ea lits'oants'o ho promtail.yml (ho na le khetho ea "docker" ho config e etsang se tšoanang moleng o le mong, empa e ke ke ea totobala hakaalo):

scrape_configs:
 - job_name: containers

   static_configs:
       labels:
         job: containerlogs
         __path__: /var/lib/docker/containers/*/*log  # for linux only

Ha tlhophiso ena e nolofalitsoe, Loki e tla fumana lintlha tse tsoang lijaneng tsohle. Ho qoba sena, re fetola litlhophiso tsa test nginx ho docker-compose.yml - eketsa ho rema lifate tšimong ea tag:

proxy:
 image: nginx.test.v3
//…
 logging:
   driver: "json-file"
   options:
     tag: "{{.ImageName}}|{{.Name}}"

Fetola promtail.yml 'me u thehe Pipeline. Li-logs ke tse latelang:

{"log":"u001b[0;33;1mnginx.1    | u001b[0mnginx.test 172.28.0.3 - - [13/Jun/2020:23:25:50 +0000] "GET /api/index HTTP/1.1" 200 0 "-" "Stub_Bot/0.1" "0.096"n","stream":"stdout","attrs":{"tag":"nginx.promtail.test|proxy.prober"},"time":"2020-06-13T23:25:50.66740443Z"}
{"log":"u001b[0;33;1mnginx.1    | u001b[0mnginx.test 172.28.0.3 - - [13/Jun/2020:23:25:50 +0000] "GET /200 HTTP/1.1" 200 0 "-" "Stub_Bot/0.1" "0.000"n","stream":"stdout","attrs":{"tag":"nginx.promtail.test|proxy.prober"},"time":"2020-06-13T23:25:50.702925272Z"}

mekhahlelo ea liphaephe:

 - json:
     expressions:
       stream: stream
       attrs: attrs
       tag: attrs.tag

Re ntša stream, attrs, attrs.tag fields (haeba e teng) ho tsoa ho JSON e tlang ebe re e kenya 'mapeng o ntšitsoeng.

 - regex:
     expression: ^(?P<image_name>([^|]+))|(?P<container_name>([^|]+))$
     source: "tag"

Haeba ho ne ho khonahala ho beha lebala la tag 'mapeng o ntšitsoeng, ebe re sebelisa regexp re ntša mabitso a setšoantšo le setshelo.

 - labels:
     image_name:
     container_name:

Re fana ka li-labels. Haeba linotlolo image_name le container_name li fumanoa ho data e nkiloeng, boleng ba bona bo tla abeloa lileibole tse nepahetseng.

 - match:
     selector: '{job="docker",container_name="",image_name=""}'
     action: drop

Re lahla lintlha tsohle tse se nang mabitso image_name le container_name set.

  - match:
     selector: '{image_name="nginx.promtail.test"}'
     stages:
       - json:
           expressions:
             row: log

Bakeng sa lits'oants'o tsohle tseo image_name ea tsona e lekanang le nginx.promtail.test, re ntša tšimo ea log ho tloha mohloling oa mohloli ebe re e kenya 'mapeng o ntšitsoeng ka senotlolo sa mola.

  - regex:
         # suppress forego colors
         expression: .+nginx.+|.+[0m(?P<virtual_host>[a-z_.-]+) +(?P<nginxlog>.+)
         source: logrow

Re hlakola khoele ea ho kenya ka mantsoe a tloaelehileng ebe re ntša moamoheli oa nginx le mola oa log.

     - regex:
         source: nginxlog
         expression: ^(?P<ip>[w.]+) - (?P<user>[^ ]*) [(?P<timestamp>[^ ]+).*] "(?P<method>[^ ]*) (?P<request_url>[^ ]*) (?P<request_http_protocol>[^ ]*)" (?P<status>[d]+) (?P<bytes_out>[d]+) "(?P<http_referer>[^"]*)" "(?P<user_agent>[^"]*)"( "(?P<response_time>[d.]+)")?

Hlakola log ea nginx ka lipolelo tse tloaelehileng.

    - regex:
           source: request_url
           expression: ^.+.(?P<static_type>jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|pdf|txt|tar|wav|bmp|rtf|js|flv|swf|html|htm)$
     - regex:
           source: request_url
           expression: ^/photo/(?P<photo>[^/?.]+).*$
       - regex:
           source: request_url
           expression: ^/api/(?P<api_request>[^/?.]+).*$

Hlalosa kopo_url. Ka thuso ea regexp, re tseba sepheo sa kopo: ho statics, ho lifoto, ho API le ho beha senotlolo se lumellanang 'mapeng o nkiloeng.

       - template:
           source: request_type
           template: "{{if .photo}}photo{{else if .static_type}}static{{else if .api_request}}api{{else}}other{{end}}"

Re sebelisa li-operators tse nang le maemo ho Template, re sheba libaka tse kentsoeng 'mapeng o ntšitsoeng ebe re beha litekanyetso tse hlokahalang bakeng sa sebaka sa request_type: photo, static, API. Abela tse ling haeba li hlolehile. Hona joale request_type e na le mofuta oa kopo.

       - labels:
           api_request:
           virtual_host:
           request_type:
           status:

Re beha li-labels api_request, virtual_host, request_type le boemo (boemo ba HTTP) ho latela seo re khonneng ho se beha 'mapeng o ntšitsoeng.

       - output:
           source: nginx_log_row

Fetola tlhahiso. Hona joale lenane le hloekisitsoeng la nginx 'mapeng o ntšitsoeng le ea Loki.

Kamora ho tsamaisa tlhophiso e kaholimo, u ka bona hore keno e 'ngoe le e' ngoe e ngotsoe ho latela data e tsoang ho log.

Hopola hore ho hula lileibole tse nang le palo e kholo ea boleng (k'hadinale) ho ka liehisa Loki haholo. Ke hore, ha ua lokela ho kenya index, mohlala, user_id. Bala haholoanyane ka sena sehloohongJoang lileibole ho Loki li ka etsa lipotso tsa log kapele le ha bonolo". Empa sena ha se bolele hore u ke ke ua batla ka user_id ntle le li-index. Hoa hlokahala ho sebelisa li-filters ha u batla ("tšoara" ho latela data), 'me index mona e sebetsa e le sesupo sa molapo.

Pono ea logong

Loki e ka sebetsa e le mohloli oa data oa lichate tsa Grafana a sebelisa LogQL. Lintlha tse latelang lia tšehetsoa:

• sekhahla - palo ea lirekoto motsotsoana;
• bala ka nako - palo ea lirekoto sebakeng se fanoeng.

Ho boetse ho na le mesebetsi e kopanyang Sum, Avg le tse ling. O ka etsa li-graph tse rarahaneng haholo, mohlala, kerafo ea palo ea liphoso tsa HTTP:

Mohloli oa data oa Loki o batla o sa sebetse ho feta mohloli oa data oa Prometheus (mohlala, u ke ke ua fetola tšōmo), empa Loki e ka hokahanngoa e le mohloli oa mofuta oa Prometheus. Ha ke na bonnete ba hore na sena ke boitšoaro bo tlalehiloeng, empa ho latela karabelo e tsoang ho bahlahisi "Mokhoa oa ho hlophisa Loki joalo ka Prometheus datasource? · Khatiso ea #1222 · grafana/loki”, ka mohlala, e molaong ka ho phethahala ’me Loki e lumellana ka botlalo le PromQL.

Kenya Loki e le mohloli oa data ka mofuta oa Prometheus 'me u kenye URL / loki:

'Me u ka etsa li-graph, joalokaha eka re sebetsa ka metrics e tsoang ho Prometheus:

Ke nahana hore ho se tšoane ha tšebetsong ke ha nakoana 'me bahlahisi ba tla e lokisa nakong e tlang.

Melemo

Loki e fana ka bokhoni ba ho ntša metrics ea linomoro ho tsoa litsing ebe o li romella ho Prometheus. Ka mohlala, log log ea nginx e na le palo ea li-byte ka karabo, hape, ka phetoho e itseng ea mokhoa o tloaelehileng oa log, nako ea metsotsoana eo e e nkileng ho arabela. Lintlha tsena li ka ntšoa 'me tsa romeloa ho Prometheus.

Kenya karolo e 'ngoe ho promtail.yml:

- match:
   selector: '{request_type="api"}'
   stages:
     - metrics:
         http_nginx_response_time:
           type: Histogram
           description: "response time ms"
           source: response_time
           config:
             buckets: [0.010,0.050,0.100,0.200,0.500,1.0]
- match:
   selector: '{request_type=~"static|photo"}'
   stages:
     - metrics:
         http_nginx_response_bytes_sum:
           type: Counter
           description: "response bytes sum"
           source: bytes_out
           config:
             action: add
         http_nginx_response_bytes_count:
           type: Counter
           description: "response bytes count"
           source: bytes_out
           config:
             action: inc

Khetho e u lumella ho hlalosa le ho ntlafatsa metrics ho latela lintlha tse tsoang 'mapeng o ntšitsoeng. Lintlha tsena ha li rometsoe ho Loki - li hlaha ho Promtail /metrics endpoint. Prometheus e tlameha ho hlophisoa ho amohela data ho tloha sethaleng sena. Mohlala o ka holimo, bakeng sa kopo_type="api" re bokella metric ea histogram. Ka mofuta ona oa metrics ho bonolo ho fumana li-percentiles. Bakeng sa li-statics le linepe, re bokella kakaretso ea li-byte le palo ea mela eo re fumaneng li-byte ho eona ho bala karolelano.

Bala haholoanyane ka metrics mona.

Bula boema-kepe ho Promtail:

promtail:
     image: grafana/promtail:1.4.1
     container_name: monitoring.promtail
     expose:
       - 9080
     ports:
       - "9080:9080"

Re etsa bonnete ba hore metrics e nang le prefix ea promtail_custom e hlahile:

Ho theha Prometheus. Kenya tlhahiso ea mosebetsi:

- job_name: 'promtail'
 scrape_interval: 10s
 static_configs:
   - targets: ['promtail:9080']

Ebe u taka graph:

Ka tsela ena u ka fumana, ho etsa mohlala, likopo tse 'ne tse liehang ho feta. U ka boela ua etsa hore ho behoe leihlo bakeng sa metrics ena.

Katoloso

Loki e ka ba ka mokhoa o le mong oa binary mme e sharded (mokhoa o otlolohileng o otlolohileng). Boemong ba bobeli, e ka boloka data ho leru, 'me li-chunks le index li bolokiloe ka thoko. Phetolelong ea 1.5, bokhoni ba ho boloka sebakeng se le seng bo kentsoe ts'ebetsong, empa ha bo so khothaletsoe ho bo sebelisa tlhahiso.

Li-chunks li ka bolokoa ka polokelong e lumellanang le S3, 'me li-database tse otlolohileng li ka sebelisoa ho boloka li-index: Cassandra, BigTable kapa DynamoDB. Likarolo tse ling tsa Loki - Baphatlalatsi (bakeng sa ho ngola) le Querier (bakeng sa lipotso) - ha li na boemo ebile li lekana ka holimo.

Sebokeng sa DevOpsDays Vancouver 2019, e mong oa bankakarolo Callum Styan o phatlalalitse hore ka Loki projeke ea hae e na le li-petabytes tsa lifate tse nang le index e ka tlase ho 1% ea boholo bohle: "Kamoo Loki e Amanang le Metrics le Logs - 'me e U bolokela Chelete".

Papiso ea Loki le ELK

Boholo ba index

Ho leka boholo ba index e hlahisitsoeng, ke ile ka nka lits'oants'o tse tsoang ka har'a sets'oants'o sa nginx moo Pipeline e kaholimo e lokiselitsoeng eona. Faele ea log e ne e na le mela e 406 e nang le boholo ba 624 MB. Li-log li entsoe nakong ea hora, lirekoto tse ka bang 109 motsotsoana.

Mohlala oa mela e 'meli ho tsoa ho log:

Ha e ngotsoe ke ELK, sena se fane ka boholo ba index ea 30,3 MB:

Tabeng ea Loki, sena se fane ka hoo e ka bang 128 KB ea index le hoo e ka bang 3,8 MB ea data ka likotoana. Ke habohlokoa ho hlokomela hore log e entsoe ka maiketsetso 'me e ne e se na mefuta e mengata ea data. Gzip e bonolo ho log ea pele ea Docker JSON e nang le data e fane ka compression ea 95,4%, 'me e fanoe ka hore ke feela log e hloekisitsoeng ea nginx e rometsoeng ho Loki ka boeona, ho hatella ho 4 MB hoa utloahala. Palo eohle ea boleng bo ikhethang bakeng sa mangolo a Loki e ne e le 35, e hlalosang boholo bo nyane ba index. Bakeng sa ELK, log e boetse e hlakotsoe. Kahoo, Loki e hatelletse data ea mantlha ka 96%, le ELK ka 70%.

Tšebeliso ea memori

Haeba re bapisa palo eohle ea Prometheus le ELK, joale Loki "e ja" ka makhetlo a 'maloa ka tlase. Ho hlakile hore tšebeletso ea Go e sebelisa chelete e ka tlaase ho tšebeletso ea Java, 'me ho bapisa boholo ba Heap Elasticsearch JVM le memori e fanoeng bakeng sa Loki e fosahetse, empa leha ho le joalo, ke habohlokoa ho hlokomela hore Loki o sebelisa mohopolo o fokolang haholo. Molemo oa eona oa CPU ha o totobetse, empa o teng hape.

Lebelo

Loki "e ja" lifate ka potlako. Lebelo le itšetlehile ka lintlha tse ngata - ke mofuta ofe oa li-logs, hore na re rarahane hakae re li arola, marang-rang, disk, joalo-joalo - empa ka sebele e phahame ho feta ea ELK (tekong ea ka - hoo e ka bang ka makhetlo a mabeli). Sena se hlalosoa ke taba ea hore Loki o kenya data e nyane haholo ho index mme, ka hona, o qeta nako e nyane ho indexing. Tabeng ena, boemo bo khutlisetsoa morao ka lebelo la ho batla: Loki e bonahala e lieha ho fokotsa data e kholo ho feta li-gigabytes tse seng kae, ha ELK, lebelo la ho batla ha le itšetlehe ka boholo ba data.

Log Search

Loki e tlase haholo ho ELK mabapi le bokhoni ba ho batla log. Grep e nang le lipolelo tse tloaelehileng ke ntho e matla, empa e ka tlaase ho database ea batho ba baholo. Ho haella ha lipotso tse fapaneng, ho bokellana feela ka lileibole, ho se khone ho batla ntle le li-label - sena sohle se re sitisa ho batla tlhahisoleseling e khahlang Loki. Sena ha se bolele hore ha ho letho le ka fumanoang ho sebelisoa Loki, empa e hlalosa ho phalla ha ho sebetsa ka li-logs, ha u qala ho fumana bothata lichabeng tsa Prometheus, ebe u sheba se ileng sa etsahala ka li-logs u sebelisa mangolo ana.

segokanyimmediamentsi sa sebolokigolo

Taba ea pele, e ntle (ke masoabi, ha e khone ho hanela). Grafana e na le sebopeho se setle, empa Kibana e sebetsa haholoanyane.

Loki melemo le boiketlo

Har'a li-pluses, ho ka hlokomeloa hore Loki o kopanya le Prometheus, ka ho latellana, re fumana metrics le ho lemosa ka ntle ho lebokose. E loketse ho bokella lits'oants'o le ho li boloka ka Kubernetes Pods, kaha e na le tšibollo ea ts'ebeletso e futsitsoeng ho tsoa ho Prometheus mme e ikamahanya le mabitso.

Ea li-minus - litokomane tse futsanehileng. Lintho tse ling, tse kang likarolo le bokhoni ba Promtail, ke fumane feela ts'ebetsong ea ho ithuta khoutu, molemo oa mohloli o bulehileng. Phoso e 'ngoe ke bokhoni bo fokolang ba ho bala. Mohlala, Loki ha e khone ho arola likutu tsa mela e mengata. Hape, mefokolo e kenyelletsa taba ea hore Loki ke theknoloji e batlang e le nyane (tokoloho ea 1.0 e bile ka Pulungoana 2019).

fihlela qeto e

Loki ke theknoloji e thahasellisang ea 100% e loketseng merero e menyenyane le e mahareng, e u lumellang hore u rarolle mathata a mangata a ho kopanya li-log, ho batla li-log, ho beha leihlo le ho hlahloba lifate.

Ha re sebelise Loki ho Badoo, hobane re na le ELK stack e re loketseng 'me e' nile ea apareloa ke litharollo tse fapaneng tsa moetlo ho theosa le lilemo. Ho rona, khopiso ke ho batla ka har'a likutu. Ka hoo e ka bang 100 GB ea li-logs ka letsatsi, ke habohlokoa hore re khone ho fumana ntho e 'ngoe le e' ngoe le ho feta hanyane ebe re e etsa kapele. Bakeng sa ho etsa lichate le ho beha leihlo, re sebelisa litharollo tse ling tse etselitsoeng litlhoko tsa rona mme li kopantsoe hammoho. The Loki stack e na le melemo e bonahalang, empa e ke ke ea re fa ho feta seo re nang le sona, 'me melemo ea eona e ke ke ea feta hantle litšenyehelo tsa ho falla.

'Me le hoja ka mor'a lipatlisiso ho ile ha hlaka hore re ke ke ra sebelisa Loki, re tšepa hore poso ena e tla u thusa ho khetha.

Sebaka sa polokelo se nang le khoutu e sebelisitsoeng sengolong se teng mona.

Source: www.habr.com