🥇Aoina o ogalaau mai Loki

Matou i Badoo o loʻo mataʻituina pea tekinolosi fou ma iloilo pe faʻaaogaina pe leai foi i la matou polokalama. Matou te fia faʻasoa se tasi o nei suʻesuʻega i le alalafaga. O loʻo tuʻuina atu ia Loki, o se faiga faʻapipiʻi ogalaau.

Loki o se fofo mo le teuina ma le matamataina o ogalaau, ma o lenei faaputuga e maua ai foi se faiga fetuutuunai mo le auʻiliʻili ma le lafoina o faʻamatalaga i Prometheus. Ia Me, na faʻasalalau ai se isi faʻafouga, lea o loʻo faʻamalosia malosi e le au foafoa. Matou te fiafia i mea e mafai ona fai e Loki, o a mea e maua ai, ma pe o le a le tele e mafai ai ona fai ma sui i le ELK, le faʻaputu o loʻo matou faʻaogaina nei.

O le a le Loki

Grafana Loki o se seti o vaega mo se faiga faʻapipiʻi atoa. E le pei o isi faiga fa'apena, Loki e fa'avae i luga o le manatu o le fa'asinomaga na'o le log metadata - fa'ailoga (pei o le Prometheus), ma fa'apipi'i ogalaau i latou i autafa i ni pusi eseese.

Home page, GitHub

Ae ou te leʻi oʻo i le mea e mafai ona e faia ma Loki, ou te manaʻo e faʻamalamalama le uiga o le "le manatu o le faasinoupuina naʻo metadata". Sei o tatou faʻatusatusa le auala Loki ma le faʻasino igoa i fofo masani, pei ole Elasticsearch, faʻaaoga le faʻataʻitaʻiga o se laina mai le log nginx:

172.19.0.4 - - [01/Jun/2020:12:05:03 +0000] "GET /purchase?user_id=75146478&amp;item_id=34234 HTTP/1.1" 500 8102 "-" "Stub_Bot/3.0" "0.001"

O faiga fa'aleaganu'u e fa'avasega le laina atoa, e aofia ai ma fanua ma le tele o fa'aoga tulaga ese user_id ma item_id, ma teu mea uma i fa'asino igoa tetele. O le lelei o lenei faiga o le mafai lea ona e faʻatautaia fesili lavelave vave, talu ai e toetoe lava o faʻamaumauga uma o loʻo i totonu o le faasino igoa. Ae e tatau ona e totogia lenei mea ona o le faʻailoga e faʻateleina, lea e faʻaliliuina i manaʻoga manatua. O le i'uga, o le fa'asinomaga atoa o ogalaau e fa'atusatusa i le lapopo'a ma ogala'au lava ia. Ina ia vave su'esu'e i ai, e tatau ona uta le fa'asino i le manatua. Ma o le tele o ogalaau, o le vave foi lea o le fa'atupuina o le fa'asinomaga ma le tele o le manatua e fa'aaogaina.

O le Loki auala e manaʻomia ai naʻo faʻamatalaga talafeagai e maua mai le manoa, o le numera o tau e laʻititi. O le auala lea tatou te maua ai se fa'ailoga la'ititi ma e mafai ona su'e fa'amaumauga e ala i le fa'amama i le taimi ma fa'asino igoa, ona su'esu'e lea o le isi vaega i fa'amatalaga masani po'o su'esu'ega i lalo. O le faagasologa e foliga mai e le sili ona vave, ae vaevae e Loki le talosaga i ni vaega ma faʻatino tutusa, faʻatautaia le tele o faʻamaumauga i se taimi puupuu. Ole numera o shards ma talosaga tutusa i totonu e mafai ona faʻatulagaina; o le mea lea, o le aofaʻi o faʻamaumauga e mafai ona faʻatautaia i le iunite o le taimi e faʻalagolago i luga ole aofaʻi o punaoa ua tuʻuina atu.

O lenei fefaʻatauaʻiga i le va o se faʻailoga televave tele ma se faʻamaufaʻailoga faʻasolosolo faʻatasi e mafai ai e Loki ona pulea le tau o le faiga. E mafai ona fetuutuunai configured ma faalauteleina e tusa ai ma ou manaoga.

O le faaputuga Loki e aofia ai vaega e tolu: Promtail, Loki, Grafana. E aoina e le Promtail ogalaau, faʻagasolo ma lafo i Loki. Loki e tausia i latou. Ma e mafai e Grafana ona talosagaina faʻamatalaga mai Loki ma faʻaalia. I se tulaga lautele, e mafai ona faʻaaogaina Loki e le gata mo le teuina o ogalaau ma suʻesuʻeina i latou. O le faaputuga atoa e maua ai avanoa sili mo le gaosiga ma le suʻeina o faʻamatalaga o loʻo oʻo mai e faʻaaoga ai le auala Prometheus.
E mafai ona maua se faʻamatalaga o le faʻapipiʻiina iinei.

Su'e ogalaau

E mafai ona e suʻeina ogalaau i se faʻapitoa faʻapitoa Grafana - Explorer. O fesili e faʻaaoga ai le gagana LogQL, e tutusa lelei ma le PromQL na faʻaaogaina e Prometheus. I le mataupu faavae, e mafai ona manatu o se grep tufatufa.

O le fa'aoga su'esu'e e fa'apea:

O le fesili lava ia e lua vaega: filifili ma faamama. Selector o se suʻesuʻega e ala i faʻamaufaʻailoga metadata (igoa) o loʻo tuʻuina atu i ogalaau, ma o le faamama o se manoa suʻesuʻe poʻo le regexp e faʻamama ai faʻamaumauga o loʻo faʻamatalaina e le tagata filifilia. I le faʻataʻitaʻiga ua tuʻuina atu: I puipui faʻamalo - o le tagata filifilia, mea uma pe a uma - o le faamama.

{image_name="nginx.promtail.test"} |= "index"

Ona o le auala e galue ai Loki, e le mafai ona e faia ni talosaga e aunoa ma se tagata filifilia, ae o igoa e mafai ona faia i se tulaga lautele.

O le tagata filifilia o le ki-taua o le tau i fusi fa'aa'ai. E mafai ona e tu'ufa'atasia tagata filifilia ma fa'amaoti tulaga su'esu'e eseese e fa'aaoga ai le =, != fa'agaioiga po'o fa'amatalaga masani:

{instance=~"kafka-[23]",name!="kafka-dev"} 
// Найдёт логи с лейблом instance, имеющие значение kafka-2, kafka-3, и исключит dev

O se faamama o se tusitusiga poʻo le regexp o le a faʻamama uma faʻamaumauga na maua e le tagata filifilia.

E mafai ona maua kalafi faʻapitoa e faʻavae i luga o faʻamaumauga maua i le metrics mode. Mo se faʻataʻitaʻiga, e mafai ona e suʻeina le tele o mea e tupu i totonu o ogalaau nginx o se faʻamatalaga o loʻo i ai le manoa faʻasino:

O se faʻamatalaga atoa o foliga e mafai ona maua i totonu o faʻamaumauga LogQL.

Fa'asalalau ogalaau

E tele auala e aoina ai ogalaau:

Faatasi ai ma le fesoasoani a Promtail, o se vaega masani o le faaputuga mo le aoina o ogalaau.
Sa'o mai le koneteina o lo'o fa'aoga Loki Docker Logging Avetaavale.
Fa'aaoga le Fluentd po'o le Fluent Bit e mafai ona lafo fa'amatalaga i Loki. E le pei o le Promtail, o lo'o i ai a latou fa'asalalauga ua saunia mo toetoe lava o so'o se ituaiga ogalaau ma e mafai fo'i ona fa'atautaia fa'amaumauga e tele laina.

E masani ona fa'aogaina le Promtail mo le fa'avasegaina. E tolu mea e fai:

Su'e puna'oa fa'amaumauga.
Fa'apipi'i igoa ia i latou.
Auina atu faʻamatalaga i Loki.

I le taimi nei e mafai e Promtail ona faitau ogalaau mai faila i le lotoifale ma mai le systemd journal. E tatau ona fa'apipi'i i masini uma e aoina mai ai ogalaau.

O loʻo i ai le tuʻufaʻatasia ma Kubernetes: E otometi lava ona maua e le Promtail le tulaga o le fuifui e ala i le Kubernetes REST API ma aoina mai ogalaau mai se node, auaunaga poʻo le pod, vave faʻapipiʻi igoa e faʻatatau i metadata mai Kubernetes (igoa pod, igoa faila, ma isi).

E mafai fo'i ona e tautau fa'ailoga e fa'atatau i fa'amaumauga mai le ogalaau e fa'aaoga ai le Pipeline. Pipeline Promtail e mafai ona aofia ai ituaiga e fa o laasaga. Fa'amatalaga atili - i pepa aloaia, O le a vave ona ou matauina nisi o nuances.

Fa'asologa o vaega. Ole tulaga lea ole RegEx ma le JSON. I le taimi nei, matou te faʻapipiʻiina faʻamaumauga mai ogalaau i totonu o le faʻafanua o loʻo taʻua. E mafai ona e maua mai le JSON e ala i le na o le kopiina o fanua matou te manaʻomia i totonu o le faʻafanua o loʻo maua mai, poʻo faʻamatalaga masani (RegEx), lea o loʻo faʻaigoaina ai vaega o loʻo "faʻafanua" i totonu o le faʻafanua na maua mai. O le fa'afanua o lo'o fa'aaogaina o se fa'aputuga taua, lea o le ki o le igoa o le fanua, ma le tau o lona tau mai ogalaau.
Suiga tulaga. O lenei laasaga e lua filifiliga: suiga, lea tatou te setiina ai tulafono o suiga, ma le puna - o le puna o faʻamatalaga mo le suiga mai le faʻafanua na maua mai. Afai e leai se fanua fa'apea i totonu o le fa'afanua na maua mai, ona faia lea. O le mea lea, e mafai ona faia ni faʻailoga e le faʻavae i luga o le faʻafanua na maua mai. I lenei laʻasaga, e mafai ona tatou faʻaogaina faʻamatalaga i totonu o le faʻafanua na maua mai e faʻaaoga ai se malosi malosi mamanu golang. E le gata i lea, e tatau ona tatou manatua o le faʻafanua na maua mai o loʻo faʻatumu atoa i le taimi o le faʻavasegaina, lea e mafai ai, mo se faʻataʻitaʻiga, ona siaki le tau o loʻo i ai: "{{if .tag}tag value exists{end}}". Fa'ata'ita'iga e lagolagoina tulaga, fa'agasele, ma nisi o galuega o manoa e pei o le Suia ma le Tipi.
Laasaga faatino. I lenei laasaga, e mafai ona e faia se mea i le mea na maua mai:
- Fausia se faʻailoga mai faʻamaumauga na maua mai, lea o le a faʻasinoina e Loki.
- Suia pe seti le taimi o mea na tupu mai le ogalaau.
- Suia le faʻamatalaga (log text) o le a alu i Loki.
- Fausia fua.
Laasaga o le filiga. Le tulaga fetaui, lea e mafai ai ona matou lafoina faʻamaumauga matou te le manaʻomia / dev / null, pe lafo i latou mo nisi gaioiga.

I le faʻaaogaina o le faʻataʻitaʻiga o le faʻaogaina o ogalaau masani nginx, o le a ou faʻaalia pe faʻafefea ona e faʻaogaina ogalaau e faʻaaoga ai le Promtail.

Mo le suʻega, seʻi o tatou suʻeina se nginx jwilder/nginx-proxy: ata alpine ma se tamaʻi daemon e mafai ona fesiligia ia lava e ala i le HTTP o le nginx-proxy. O le daemon e tele ona fa'ai'uga, lea e mafai ona tu'uina atu ai tali o lapo'a eseese, fa'atasi ai ma tulaga HTTP eseese ma fa'atuai eseese.

O le a matou aoina mai ogalaau mai koneteina docker, lea e mafai ona maua i luga o le ala /var/lib/docker/containers/ / -json.log

I le docker-compose.yml matou te setiina le Promtail ma faʻamaonia le ala i le config:

promtail:
  image: grafana/promtail:1.4.1
 // ...
 volumes:
   - /var/lib/docker/containers:/var/lib/docker/containers:ro
   - promtail-data:/var/lib/promtail/positions
   - ${PWD}/promtail/docker.yml:/etc/promtail/promtail.yml
 command:
   - '-config.file=/etc/promtail/promtail.yml'
 // ...

Faʻaopoopo le ala i ogalaau i le promtail.yml (o loʻo i ai se "docker" filifiliga i le config e faia tutusa i le laina e tasi, ae o le a le manino tele):

scrape_configs:
 - job_name: containers

   static_configs:
       labels:
         job: containerlogs
         __path__: /var/lib/docker/containers/*/*log  # for linux only

A fa'agaoioia lenei fa'atulagaga, o le a maua e Loki ni ogalaau mai koneteina uma. Ina ia aloese mai lenei mea, matou te suia tulaga o le suʻega nginx i le docker-compose.yml - faʻaopoopo le faʻapipiʻi i le faʻailoga:

proxy:
 image: nginx.test.v3
//…
 logging:
   driver: "json-file"
   options:
     tag: "{{.ImageName}}|{{.Name}}"

Fa'asa'o le promtail.yml ma fa'atutu le Pipeline. O ogalaau e faapea:

{"log":"u001b[0;33;1mnginx.1    | u001b[0mnginx.test 172.28.0.3 - - [13/Jun/2020:23:25:50 +0000] "GET /api/index HTTP/1.1" 200 0 "-" "Stub_Bot/0.1" "0.096"n","stream":"stdout","attrs":{"tag":"nginx.promtail.test|proxy.prober"},"time":"2020-06-13T23:25:50.66740443Z"}
{"log":"u001b[0;33;1mnginx.1    | u001b[0mnginx.test 172.28.0.3 - - [13/Jun/2020:23:25:50 +0000] "GET /200 HTTP/1.1" 200 0 "-" "Stub_Bot/0.1" "0.000"n","stream":"stdout","attrs":{"tag":"nginx.promtail.test|proxy.prober"},"time":"2020-06-13T23:25:50.702925272Z"}

laasaga o paipa:

 - json:
     expressions:
       stream: stream
       attrs: attrs
       tag: attrs.tag

Matou te aveese le vaitafe, attrs, attrs.tag fanua (pe a iai) mai le JSON o loʻo oʻo mai ma tuʻu i totonu o le faʻafanua o loʻo maua mai.

 - regex:
     expression: ^(?P<image_name>([^|]+))|(?P<container_name>([^|]+))$
     source: "tag"

Afai e mafai ona tuʻu le faʻailoga i totonu o le faʻafanua faʻapipiʻi, ona faʻaaogaina lea o le regexp matou te aveese igoa o le ata ma le atigipusa.

 - labels:
     image_name:
     container_name:

Matou te tofia igoa. Afai o ki image_name ma container_name o loʻo maua i faʻamaumauga na maua mai, ona tuʻuina atu lea o latou tau i igoa talafeagai.

 - match:
     selector: '{job="docker",container_name="",image_name=""}'
     action: drop

Matou te lafoai uma ogalaau e leai ni fa'ailoga image_name ma container_name seti.

  - match:
     selector: '{image_name="nginx.promtail.test"}'
     stages:
       - json:
           expressions:
             row: log

Mo ogalaau uma o latou image_name e tutusa ma nginx.promtail.test, matou te aveese le fanua ogalaau mai le ogalaau puna ma tuʻu i totonu o le faʻafanua faʻapipiʻi ma le ki laina.

  - regex:
         # suppress forego colors
         expression: .+nginx.+|.+[0m(?P<virtual_host>[a-z_.-]+) +(?P<nginxlog>.+)
         source: logrow

Matou te faʻamama le manoa faʻaoga i faʻamatalaga masani ma toso ese le nginx virtual host ma le nginx log line.

     - regex:
         source: nginxlog
         expression: ^(?P<ip>[w.]+) - (?P<user>[^ ]*) [(?P<timestamp>[^ ]+).*] "(?P<method>[^ ]*) (?P<request_url>[^ ]*) (?P<request_http_protocol>[^ ]*)" (?P<status>[d]+) (?P<bytes_out>[d]+) "(?P<http_referer>[^"]*)" "(?P<user_agent>[^"]*)"( "(?P<response_time>[d.]+)")?

Parse log nginx ma fa'amatalaga masani.

    - regex:
           source: request_url
           expression: ^.+.(?P<static_type>jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|pdf|txt|tar|wav|bmp|rtf|js|flv|swf|html|htm)$
     - regex:
           source: request_url
           expression: ^/photo/(?P<photo>[^/?.]+).*$
       - regex:
           source: request_url
           expression: ^/api/(?P<api_request>[^/?.]+).*$

Parse request_url. Faatasi ai ma le fesoasoani a regexp, matou te fuafuaina le mafuaʻaga o le talosaga: i statics, i ata, i le API ma seti le ki tutusa i le faʻafanua na maua.

       - template:
           source: request_type
           template: "{{if .photo}}photo{{else if .static_type}}static{{else if .api_request}}api{{else}}other{{end}}"

I le faʻaaogaina o faʻalapotopotoga faʻapitoa i le Faʻataʻitaʻiga, matou te siakiina fanua faʻapipiʻi i le faʻafanua faʻapipiʻi ma seti tulaga manaʻomia mo le talosaga_type fanua: ata, static, API. Tofi isi pe a le manuia. O le request_type o lo'o iai le ituaiga talosaga.

       - labels:
           api_request:
           virtual_host:
           request_type:
           status:

Matou te setiina igoa api_request, virtual_host, request_type ma le tulaga (HTTP tulaga) faʻavae i luga o mea na mafai ona matou tuʻuina i totonu o le faʻafanua na maua mai.

       - output:
           source: nginx_log_row

Suia galuega faatino. O lea la o le ogalaau o le nginx faʻamamaina mai le faʻafanua na maua mai e alu i Loki.

A maeʻa ona faʻaogaina le faʻatonuga o loʻo i luga, e mafai ona e vaʻaia o faʻamaumauga taʻitasi e faʻailogaina e faʻatatau i faʻamaumauga mai le ogalaau.

Ia manatua o le suʻeina o igoa ma se numera tele o tau (cardinality) e mafai ona faʻagesegese tele Loki. O lona uiga, e le tatau ona e tuʻuina i totonu o le faasino igoa, mo se faʻataʻitaʻiga, user_id. Faitau atili e uiga i lenei mea i le tusigaE fa'afefea ona fa'afaigofie ma fa'afaigofie e fa'ailoga i Loki". Ae e le o lona uiga e le mafai ona e suʻeina e le user_id e aunoa ma ni faʻasino igoa. E manaʻomia le faʻaogaina o filiga pe a suʻe ("grab" e tusa ai ma faʻamaumauga), ma o le faasino igoa iinei e galue o se faʻamatalaga o le vaitafe.

Va'aiga o fa'amaumauga

Loki e mafai ona galue o se faʻamatalaga faʻamatalaga mo siata Grafana e faʻaaoga ai LogQL. O vaega nei e lagolagoina:

fua - numera o fa'amaumauga i le sekone;
faitau i le taimi - o le numera o faʻamaumauga i totonu o le laina tuʻuina atu.

E iai fo'i galuega fa'aopoopo Sum, Avg ma isi. E mafai ona e fausia ni kalafi lavelave, mo se faʻataʻitaʻiga, se kalafi o le numera o mea sese HTTP:

Ole puna'oa ole fa'amaumauga a Loki e fai lava si le aoga nai lo le Prometheus fa'amaumauga (mo se fa'ata'ita'iga, e le mafai ona e suia le tala), ae e mafai ona fa'afeso'ota'i Loki e fai ma fa'apogai ituaiga Prometheus. Ou te le o mautinoa pe faʻamaonia lenei amio, ae faʻamasino i le tali mai le au atiaʻe "E faʻafefea ona faʻapipiʻi Loki e fai ma Prometheus datasource? · Fa'amatalaga #1222 · fa'ailoga/loki”, mo se faʻataʻitaʻiga, e matua faʻatulafonoina ma Loki e fetaui lelei ma PromQL.

Faʻaopoopo Loki e fai ma faʻamatalaga faʻamatalaga ma le ituaiga Prometheus ma faʻapipiʻi URL /loki:

Ma e mafai ona e faia kalafi, e pei o loʻo matou galulue ma metrics mai Prometheus:

Ou te manatu o le eseesega i galuega e le tumau ma o le a faʻaleleia e le au atinaʻe i le lumanaʻi.

Fuafuaga

Loki e maua ai le malosi e aveese mai ai numera numera mai ogalaau ma auina atu i Prometheus. Mo se faʻataʻitaʻiga, o le log nginx o loʻo i ai le numera o bytes i le tali, ma faʻapea foʻi, faʻatasi ai ma se suiga patino o le faʻasologa o faʻamaumauga masani, le taimi i sekone na manaʻomia e tali atu ai. O nei faʻamatalaga e mafai ona maua ma auina atu i Prometheus.

Fa'aopoopo se isi vaega ile promtail.yml:

- match:
   selector: '{request_type="api"}'
   stages:
     - metrics:
         http_nginx_response_time:
           type: Histogram
           description: "response time ms"
           source: response_time
           config:
             buckets: [0.010,0.050,0.100,0.200,0.500,1.0]
- match:
   selector: '{request_type=~"static|photo"}'
   stages:
     - metrics:
         http_nginx_response_bytes_sum:
           type: Counter
           description: "response bytes sum"
           source: bytes_out
           config:
             action: add
         http_nginx_response_bytes_count:
           type: Counter
           description: "response bytes count"
           source: bytes_out
           config:
             action: inc

O le filifiliga e mafai ai e oe ona faʻamalamalamaina ma faʻafou metrics e faʻavae i luga o faʻamaumauga mai le faʻafanua na aumai. O fua nei e le lafo ia Loki - e aliali mai i le Promtail/metrics endpoint. Prometheus e tatau ona faʻatulagaina e maua ai faʻamatalaga mai lenei laasaga. I le faʻataʻitaʻiga i luga, mo request_type = "api" matou te aoina se metric histogram. Faatasi ai ma lenei ituaiga o metrics e faigofie ona maua pasene. Mo statics ma ata, matou te aoina le aofaʻi o bytes ma le numera o laina na matou maua ai bytes e fuafua ai le averesi.

Faitau atili e uiga i metrics iinei.

Tatala se uafu i Promtail:

promtail:
     image: grafana/promtail:1.4.1
     container_name: monitoring.promtail
     expose:
       - 9080
     ports:
       - "9080:9080"

Matou te fa'amautinoa ua aliali mai fua fa'atatau i le promtail_custom prefix:

Faʻatulagaina o Prometheus. Fa'aopoopo galuega folafola:

- job_name: 'promtail'
 scrape_interval: 10s
 static_configs:
   - targets: ['promtail:9080']

Ma tusi se kalafi:

O le auala lea e mafai ai ona e su'eina, mo se fa'ata'ita'iga, fesili e fa lemu. E mafai fo'i ona e fa'atulaga le mata'ituina mo nei fua fa'atatau.

Fuaina

Loki e mafai ona i ai i tulaga uma e lua binary ma sharded (tulaga fa'asaga i luga ole laiga). I le tulaga lona lua, e mafai ona faʻasaoina faʻamaumauga i le ao, ma o pusi ma faʻamaufaʻailoga e teuina eseese. I le version 1.5, o loʻo faʻatinoina le mafai ona teuina i se tasi nofoaga, ae e leʻi fautuaina e faʻaaoga i le gaosiga.

E mafai ona teuina fasi mea i totonu o le S3-compatible storage, mo le teuina o faʻamatalaga, faʻaoga faʻamatalaga faʻasalalau faʻasalalau: Cassandra, BigTable poʻo DynamoDB. O isi vaega o Loki - Fa'asoa (mo le tusitusi) ma le Querier (mo fa'amatalaga) - e leai ni fa'ailoga ma fua fa'ata'atia.

I le DevOpsDays Vancouver 2019 conference, o se tasi o tagata auai Callum Styan na faasilasila mai e faapea, faatasi ai ma Loki, o lana poloketi o loʻo i ai ni petabytes o ogalaau ma se faʻailoga e itiiti ifo i le 1% o le aofaʻi atoa: "Fa'afefea ona Fa'asa'o e Loki Fua ma Fa'amaumauga - Ma Fa'asaoina Oe Tupe".

Faatusatusaga o Loki ma ELK

Fa'asinomaga tele

Ina ia faʻataʻitaʻiina le fua faʻatatau o le tele, na ou ave ogalaau mai le pusa nginx lea na faʻatulagaina ai le Pipeline i luga. O le faila faila o loʻo i ai laina 406 ma le aofaʻi atoa o le 624 MB. O ogalaau na gaosia i totonu o le itula, e tusa ma le 109 faamaumauga i le sekone.

O se faʻataʻitaʻiga o laina e lua mai le ogalaau:

Ina ua fa'asinoina e le ELK, na maua ai se fa'asinomaga o le 30,3 MB:

I le tulaga o Loki, na tuʻuina atu e uiga i le 128 KB o faʻamaumauga ma e uiga i le 3,8 MB o faʻamaumauga i pusi. E taua le matauina o le ogalaau na gaosia faʻapitoa ma e leʻi faʻaalia ai le tele o faʻamatalaga eseese. O se gzip faigofie i luga o le uluai Docker JSON log ma faʻamaumauga na tuʻuina atu ai le faʻamalosi o le 95,4%, ma tuʻuina atu e naʻo le faʻamamaina o le nginx log na auina atu ia Loki lava ia, o le faʻamalosi i le 4 MB e malamalama. O le aofaʻi o tulaga faʻapitoa mo Loki igoa e 35, lea e faʻamatalaina ai le laʻititi laʻititi o le faʻailoga. Mo ELK, na faʻamamaina foi le ogalaau. O le mea lea, na faʻapipiʻi e Loki faʻamaumauga muamua e 96%, ma ELK e 70%.

Fa'aaogāga manatua

Afai tatou te faʻatusatusa le faaputuga atoa o Prometheus ma ELK, ona "ai" lea o Loki i ni nai taimi. E manino lava o le Go service e faʻaaogaina le itiiti ifo nai lo le Java service, ma le faʻatusatusaina o le tele o le Heap Elasticsearch JVM ma le tuʻufaʻatasia mo Loki e le saʻo, ae ui i lea, e taua le matauina o Loki e faʻaaogaina le itiiti ifo o le manatua. O lona faʻaogaina o le CPU e le o manino, ae o loʻo i ai foi.

Televave

Loki "'ai" ogalaau vave. O le saoasaoa e faʻalagolago i le tele o mea - o le a le ituaiga o ogalaau, pe faʻafefea ona tatou faʻavasegaina, fesoʻotaʻiga, disk, ma isi - ae e mautinoa lava e maualuga atu nai lo le ELK (i laʻu suʻega - pe a ma le lua taimi). O loʻo faʻamatalaina lenei mea i le mea moni e faʻaitiitia e Loki faʻamatalaga i totonu o le faasino igoa ma, e tusa ai, faʻaalu le taimi itiiti i le faasino igoa. I lenei tulaga, o le tulaga e fesuiaʻi ma le saoasaoa o sailiga: Loki e faʻagesegese lemu i luga o faʻamatalaga e sili atu nai lo nai gigabytes, ae mo ELK, o le saoasaoa o le sailiga e le faʻalagolago i le tele o faʻamatalaga.

Su'e ogalaau

Loki e matua maualalo lava i le ELK i tulaga o le mafai ona suʻeina ogalaau. Grep faʻatasi ai ma faʻamatalaga masani o se mea malosi, ae e maualalo ifo i se tagata matutua faʻamaumauga. O le leai o ni su'esu'ega lautele, fa'apotopotoina na'o igoa, le mafai ona su'e e aunoa ma ni fa'ailoga - o nei mea uma e fa'atapula'aina ai i tatou i le su'eina o fa'amatalaga e fiafia i ai i Loki. E le faʻapea e leai se mea e mafai ona maua e faʻaaoga ai Loki, ae faʻamatalaina le tafe o le galue ma ogalaau, pe a e mauaina muamua se faʻafitauli i luga o siata Prometheus, ona vaʻai lea mo le mea na tupu i totonu o ogalaau e faʻaaoga ai nei igoa.

Ofisa

Muamua, e matagofie (fa'amalie, e le mafai ona tete'e). O Grafana e manaia foliga vaaia, ae o Kibana e sili atu ona aoga.

Loki lelei ma leaga

O faʻaopoopoga, e mafai ona maitauina o Loki e faʻatasi ma Prometheus, i le faasologa, matou te maua metrics ma mataala mai le pusa. E faigofie mo le aoina o ogalaau ma teuina i latou i Kubernetes Pods, aua o loʻo i ai se suʻesuʻega tautua na maua mai le Prometheus ma faʻapipiʻi otometi igoa.

O fa'aletonu - le lelei o fa'amaumauga. O nisi mea, e pei o foliga ma gafatia o Promtail, na ou mauaina naʻo le faagasologa o le suʻesuʻeina o le tulafono, le manuia o punaoa tatala. O le isi fa'aletonu o le vaivai o le mafai ona fa'asoa. Mo se faʻataʻitaʻiga, e le mafai e Loki ona faʻasalalau ogalaau e tele laina. E le gata i lea, o mea le lelei e aofia ai le mea moni e faapea o Loki o se tekinolosi laʻititi (faʻasaʻo 1.0 ia Novema 2019).

iʻuga

Loki o se 100% manaia tekinolosi e talafeagai mo galuega laiti ma feololo, e mafai ai ona e foia le tele o faafitauli o le tuufaatasia o ogalaau, suʻesuʻega ogalaau, mataʻituina ma auiliiliga o ogalaau.

Matou te le faʻaaogaina Loki i Badoo, aua o loʻo i ai le matou ELK stack e fetaui ma i matou ma ua faʻateleina i fofo masani i le tele o tausaga. Mo i matou, o le mea e tautevateva ai o le sailiga i totonu o ogalaau. Faatasi ai ma le toeitiiti 100 GB o ogalaau i le aso, e taua mo i tatou le mafai ona maua mea uma ma sina mea itiiti ma fai vave. Mo le siataina ma le mataʻituina, matou te faʻaogaina isi fofo e faʻatatau i o matou manaʻoga ma tuʻufaʻatasia le tasi ma le isi. O loʻo iai faʻamanuiaga faʻapitoa a le Loki stack, ae o le a le sili atu nai lo mea tatou te maua, ma o ona faʻamanuiaga o le a le sili atu nai lo le tau o le faimalaga.

Ma e ui ina uma suʻesuʻega na manino mai e le mafai ona matou faʻaogaina Loki, matou te faʻamoemoe o le a fesoasoani lenei pou ia te oe i le filifilia.

O loʻo iai le fale teu oloa ma le code faʻaaogaina i totonu o le tusiga iinei.

puna: www.habr.com

Aoina o ogalaau mai Loki