Ana tattara gundumomi daga Loki

A Badoo, muna sa ido kan sabbin fasahohi koyaushe kuma muna kimanta ko sun cancanci amfani da su a cikin tsarinmu. Muna so mu raba ɗaya daga cikin waɗannan karatun tare da al'umma. An sadaukar da shi ga Loki, tsarin tara log.

Loki shine mafita don adanawa da duba rajistan ayyukan, kuma wannan tarin kuma yana ba da tsari mai sassauƙa don tantance su da aika bayanai zuwa Prometheus. A watan Mayu, an sake fitar da wani sabuntawa, wanda masu ƙirƙira ke haɓakawa sosai. Mun kasance da sha'awar abin da Loki zai iya yi, irin ƙarfin da yake bayarwa, kuma har zuwa wane irin aiki zai iya zama madadin ELK, tarin da muke amfani da shi a yanzu.

Menene Loki

Grafana Loki saitin abubuwa ne don cikakken tsarin aiki tare da rajistan ayyukan. Ba kamar sauran tsare-tsare masu kama da juna ba, Loki ya dogara ne akan ra'ayin yin lissafin metadata kawai - alamomin (daidai da a cikin Prometheus), da matsawa kansu rajistan ayyukan cikin chunks daban-daban.

Shafin gida, GitHub

Kafin mu shiga abin da za ku iya yi da Loki, Ina so in fayyace abin da muke nufi da "tunanin ƙididdige metadata kawai." Bari mu kwatanta tsarin Loki da kuma hanyar yin ƙididdiga a cikin hanyoyin gargajiya kamar Elasticsearch, ta amfani da misalin layi daga log ɗin nginx:

172.19.0.4 - - [01/Jun/2020:12:05:03 +0000] "GET /purchase?user_id=75146478&item_id=34234 HTTP/1.1" 500 8102 "-" "Stub_Bot/3.0" "0.001"

Tsarin al'ada yana rarraba gabaɗayan jeri, gami da filayen da ke da adadi mai yawa na ƙimar mai amfani_id da item_id, da adana komai cikin manyan fihirisa. Amfanin wannan hanyar ita ce za ku iya gudanar da tambayoyin hadaddun da sauri, tun da kusan dukkanin bayanai suna cikin ma'auni. Amma wannan ya zo a farashi a cikin cewa index ya zama babba, wanda ke fassara zuwa bukatun ƙwaƙwalwar ajiya. Sakamakon haka, jigon rubutun cikakken rubutu yana kwatankwacin girmansa da rajistan ayyukan da kansu. Don bincika cikin sauri ta cikinsa, dole ne a loda fihirisar cikin ƙwaƙwalwar ajiya. Kuma yawancin gungumen azaba, da sauri da index girma da kuma yawan ƙwaƙwalwar da yake cinyewa.

Hanyar Loki tana buƙatar kawai za a fitar da bayanan da suka wajaba daga kirtani, adadin ƙimar waɗanda ƙananansu ne. Ta wannan hanyar za mu sami ƙaramin maƙasudi kuma za mu iya bincika bayanan ta hanyar tace su ta lokaci da filayen da aka ƙididdige su, sannan mu bincika sauran tare da maganganu na yau da kullun ko binciken substring. Tsarin ba ze zama mafi sauri ba, amma Loki ya raba buƙatun zuwa sassa da yawa kuma yana aiwatar da su a layi daya, yana sarrafa adadi mai yawa na bayanai cikin ɗan gajeren lokaci. Adadin shards da buƙatun layi ɗaya a cikinsu ana iya daidaita su; don haka, adadin bayanan da za'a iya sarrafa su a kowane raka'a na lokaci ya dogara ne akan adadin albarkatun da aka bayar.

Wannan cinikin tsakanin babban fihirisa mai sauri da ƙarami, mai daidaita ma'aunin ƙarfi-ƙarfi yana ba Loki damar sarrafa farashin tsarin. Ana iya daidaita shi da sassauƙa kuma a faɗaɗa shi gwargwadon buƙatu.

Tarin Loki ya ƙunshi abubuwa uku: Promtail, Loki, Grafana. Promtail yana tattara rajistan ayyukan, sarrafa su kuma yana aika su zuwa Loki. Loki yana kiyaye su. Kuma Grafana na iya neman bayanai daga Loki da nuna shi. Gabaɗaya, Loki za a iya amfani da shi ba kawai don adana rajistan ayyukan da bincika ta hanyar su ba. Dukkanin tari yana ba da dama mai girma don sarrafawa da nazarin bayanan masu shigowa ta amfani da hanyar Prometheus.
Ana iya samun bayanin tsarin shigarwa a nan.

Shiga Bincike

Kuna iya bincika rajistan ayyukan a cikin ƙa'idar Grafana ta musamman - Explorer. Tambayoyin suna amfani da yaren LogQL, wanda yayi kama da PromQL da aka yi amfani da shi a cikin Prometheus. A ka'ida, ana iya tunanin shi azaman grep da aka rarraba.

Binciken bincike yayi kama da haka:

Buƙatar kanta ta ƙunshi sassa biyu: mai zaɓa da tacewa. Selector bincike ne ta amfani da bayanan metadata (tambayoyin) waɗanda aka sanya wa rajistan ayyukan, kuma tacewa kirtani ne na bincike ko regexp wanda ke tace bayanan da mai zaɓe ya ayyana. A cikin misalin da aka bayar: A cikin takalmin gyaran kafa akwai mai zaɓe, duk abin da ya biyo baya shine tacewa.

{image_name="nginx.promtail.test"} |= "index"

Saboda yadda Loki ke aiki, ba za ku iya yin tambayoyi ba tare da mai zaɓe ba, amma ana iya yin tambura gaba ɗaya kamar yadda kuke so.

Zaɓaɓɓen ƙimar maɓalli ne a cikin takalmin gyaran kafa. Kuna iya haɗa masu zaɓaɓɓu kuma saka yanayin bincike daban-daban ta amfani da masu aiki =, != ko maganganu na yau da kullun:

{instance=~"kafka-[23]",name!="kafka-dev"} 
// Найдёт логи с лейблом instance, имеющие значение kafka-2, kafka-3, и исключит dev 

Tace rubutu ne ko regexp wanda zai tace duk bayanan da aka karɓa daga mai zaɓin.

Yana yiwuwa a sami jadawali ad-hoc dangane da bayanan da aka karɓa a cikin yanayin awo. Misali, zaku iya gano sau nawa shigarwa mai ɗauke da kirtani ya bayyana a cikin rajistan ayyukan nginx:

Ana iya samun cikakken bayanin iyawar a cikin takaddun LogQL.

Yin nazarin rajista

Akwai hanyoyi da yawa don tattara katako:

• Yin amfani da Promtail, madaidaicin abin da ke tattare da tarin rajistan ayyukan.
• Kai tsaye daga kwandon docker ta amfani da Direban Loki Docker.
• Yi amfani da Fluentd ko Fluent Bit, wanda zai iya aika bayanai zuwa Loki. Ba kamar Promtail ba, suna da shirye-shiryen parsers don kusan kowane nau'in log kuma suna iya sarrafa rajistan ayyukan layi.

Yawancin lokaci ana amfani da Promtail don tantancewa. Yana yin abubuwa uku:

• Nemo tushen bayanai.
• Haɗa musu lakabi.
• Aika bayanai zuwa Loki.

A halin yanzu Promtail na iya karanta rajistan ayyukan daga fayilolin gida da kuma daga tsarin jarida. Dole ne a sanya shi a kan kowace na'ura wanda aka tattara rajistan ayyukan daga ciki.

Akwai haɗin kai tare da Kubernetes: Promtail ta atomatik, ta hanyar Kubernetes REST API, yana gane yanayin gungu kuma yana tattara rajistan ayyukan daga kumburi, sabis ko kwafsa, nan da nan sanya alamun da ke kan metadata daga Kubernetes (sunan pod, sunan fayil, da sauransu) .

Hakanan zaka iya rataya lakabi bisa bayanai daga log ɗin ta amfani da Pipeline. Pipeline Promtail zai iya ƙunsar nau'ikan matakai guda huɗu. Karin bayani a ciki takardun shaida, Nan da nan zan lura da wasu nuances.

1. Matakan tantancewa. Wannan shine matakin RegEx da JSON. A wannan mataki, muna fitar da bayanai daga rajistan ayyukan cikin abin da ake kira taswirar da aka fitar. Za mu iya cirewa daga JSON ta hanyar kwafin filayen da muke buƙata a cikin taswirar da aka fitar kawai, ko kuma ta hanyar maganganu na yau da kullun (RegEx), inda ƙungiyoyi masu suna suna “taswira” a cikin taswirar da aka fitar. Taswirar da aka ciro shine mabuɗin-darajar maɓalli, inda maɓalli shine sunan filin, kuma ƙimarsa shine ƙimarsa daga rajistan ayyukan.
2. Sauya matakai. Wannan mataki yana da zaɓuɓɓuka guda biyu: canzawa, inda muke saita ƙa'idodin canji, da tushe - tushen bayanai don canzawa daga taswirar da aka fitar. Idan babu irin wannan filin a cikin taswirar da aka fitar, za a ƙirƙira shi. Ta wannan hanyar yana yiwuwa a ƙirƙira takalmi waɗanda ba su dogara da taswirar da aka ciro ba. A wannan mataki za mu iya sarrafa bayanan da ke cikin taswirar da aka fitar ta amfani da ingantaccen iko Samfurin Golang. Bugu da ƙari, dole ne mu tuna cewa an ɗora taswirar da aka cire gaba ɗaya a lokacin nazarin, wanda ya sa ya yiwu, alal misali, don duba ƙimar da ke ciki: "{{idan .tag} darajar tag ya wanzu{ƙarshen}}". Samfura yana goyan bayan yanayi, madaukai, da wasu ayyukan kirtani kamar Sauya da Gyara.
3. Matakan aiki. A wannan lokaci za ku iya yin wani abu tare da abin da aka ciro:
   • Ƙirƙiri lakabi daga bayanan da aka fitar, wanda Loki za ta yi maƙasudi.
   • Canja ko saita lokacin taron daga log ɗin.
   • Canja bayanan (rubutun log) wanda zai je Loki.
   • Ƙirƙiri ma'auni.
4. Matakan tacewa. Matakin wasa, inda zamu iya ko dai aika shigarwar da ba mu buƙatar /dev/null ko tura su don ƙarin aiki.

Yin amfani da misalin sarrafa rajistan ayyukan nginx na yau da kullun, zan nuna yadda zaku iya tantance rajistan ayyukan ta amfani da Promtail.

Don gwajin, bari mu ɗauki matsayin nginx-proxy wani ingantaccen hoton nginx jwilder/nginx-proxy:alpine da ƙaramin daemon wanda zai iya tambayar kansa ta hanyar HTTP. Daemon yana da wuraren ƙarewa da yawa, waɗanda zai iya ba da martani na masu girma dabam, tare da matsayi daban-daban na HTTP kuma tare da jinkiri daban-daban.

Za mu tattara rajistan ayyukan daga kwantena docker, waɗanda za a iya samu tare da hanyar /var/lib/docker/containers/ / -json.log

A cikin docker-compose.yml mun saita Promtail kuma mu ƙayyade hanyar zuwa saitin:

promtail:
  image: grafana/promtail:1.4.1
 // ...
 volumes:
   - /var/lib/docker/containers:/var/lib/docker/containers:ro
   - promtail-data:/var/lib/promtail/positions
   - ${PWD}/promtail/docker.yml:/etc/promtail/promtail.yml
 command:
   - '-config.file=/etc/promtail/promtail.yml'
 // ...

Ƙara hanyar zuwa rajistan ayyukan zuwa promtail.yml (akwai zaɓi na "docker" a cikin saitin, wanda ke yin abu iri ɗaya a cikin layi ɗaya, amma ba zai bayyana ba):

scrape_configs:
 - job_name: containers

   static_configs:
       labels:
         job: containerlogs
         __path__: /var/lib/docker/containers/*/*log  # for linux only

Lokacin da aka kunna wannan saitin, za a aika rajistan ayyukan daga duk kwantena zuwa Loki. Don guje wa wannan, muna canza saitunan gwajin nginx a cikin docker-compose.yml - ƙara filin alamar shiga:

proxy:
 image: nginx.test.v3
//…
 logging:
   driver: "json-file"
   options:
     tag: "{{.ImageName}}|{{.Name}}"

Gyara promtail.yml da kafa Pipeline. Shigar ya haɗa da rajistan ayyukan nau'in mai zuwa:

{"log":"u001b[0;33;1mnginx.1    | u001b[0mnginx.test 172.28.0.3 - - [13/Jun/2020:23:25:50 +0000] "GET /api/index HTTP/1.1" 200 0 "-" "Stub_Bot/0.1" "0.096"n","stream":"stdout","attrs":{"tag":"nginx.promtail.test|proxy.prober"},"time":"2020-06-13T23:25:50.66740443Z"}
{"log":"u001b[0;33;1mnginx.1    | u001b[0mnginx.test 172.28.0.3 - - [13/Jun/2020:23:25:50 +0000] "GET /200 HTTP/1.1" 200 0 "-" "Stub_Bot/0.1" "0.000"n","stream":"stdout","attrs":{"tag":"nginx.promtail.test|proxy.prober"},"time":"2020-06-13T23:25:50.702925272Z"}

Matakin bututun mai:

 - json:
     expressions:
       stream: stream
       attrs: attrs
       tag: attrs.tag

Muna fitar da filayen filayen, attrs, attrs.tag (idan sun kasance) daga JSON mai shigowa kuma mu sanya su cikin taswirar da aka ciro.

 - regex:
     expression: ^(?P<image_name>([^|]+))|(?P<container_name>([^|]+))$
     source: "tag"

Idan muka sami nasarar sanya filin tag a cikin taswirar da aka fitar, to ta amfani da regexp muna fitar da sunayen hoton da akwati.

 - labels:
     image_name:
     container_name:

Muna ba da lakabi. Idan an sami sunan sunan hoton da maɓallan sunan kwantena a cikin bayanan da aka fitar, to za a sanya kimarsu zuwa alamomin da suka dace.

 - match:
     selector: '{job="docker",container_name="",image_name=""}'
     action: drop

Muna watsar da duk rajistan ayyukan da ba su da alamun alamar image_name da container_name.

  - match:
     selector: '{image_name="nginx.promtail.test"}'
     stages:
       - json:
           expressions:
             row: log

Ga duk rajistan ayyukan da sunan hoton su nginx.promtail.test, cire filin log daga majigin tushe kuma saka shi cikin taswirar da aka ciro tare da maɓallin layi.

  - regex:
         # suppress forego colors
         expression: .+nginx.+|.+[0m(?P<virtual_host>[a-z_.-]+) +(?P<nginxlog>.+)
         source: logrow

Muna share layin shigarwa tare da maganganun yau da kullun kuma muna fitar da mai watsa shiri na nginx da layin log na nginx.

     - regex:
         source: nginxlog
         expression: ^(?P<ip>[w.]+) - (?P<user>[^ ]*) [(?P<timestamp>[^ ]+).*] "(?P<method>[^ ]*) (?P<request_url>[^ ]*) (?P<request_http_protocol>[^ ]*)" (?P<status>[d]+) (?P<bytes_out>[d]+) "(?P<http_referer>[^"]*)" "(?P<user_agent>[^"]*)"( "(?P<response_time>[d.]+)")?

Fassara log nginx ta amfani da maganganu na yau da kullun.

    - regex:
           source: request_url
           expression: ^.+.(?P<static_type>jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|pdf|txt|tar|wav|bmp|rtf|js|flv|swf|html|htm)$
     - regex:
           source: request_url
           expression: ^/photo/(?P<photo>[^/?.]+).*$
       - regex:
           source: request_url
           expression: ^/api/(?P<api_request>[^/?.]+).*$

Bari mu bincika request_url. Yin amfani da regexp muna ƙayyade manufar buƙatun: zuwa tsayayyen bayanai, zuwa hotuna, zuwa API kuma muna saita maɓallin da ya dace a cikin taswirar da aka fitar.

       - template:
           source: request_type
           template: "{{if .photo}}photo{{else if .static_type}}static{{else if .api_request}}api{{else}}other{{end}}"

Yin amfani da ma'aikata masu aiki a cikin Samfura, muna bincika filayen da aka shigar a cikin taswirar da aka fitar kuma muna saita ƙimar da ake buƙata don filin buƙatar_type: hoto, a tsaye, API. Sanya wani idan ya gaza. request_type yanzu ya ƙunshi nau'in buƙatar.

       - labels:
           api_request:
           virtual_host:
           request_type:
           status:

Mun saita lakabin api_request, Virtual_host, request_type da matsayi (HTTP status) dangane da abin da muka yi nasarar saka a cikin taswirar da aka fitar.

       - output:
           source: nginx_log_row

Canja fitarwa. Yanzu log ɗin nginx mai tsabta daga taswirar da aka cire yana zuwa Loki.

Bayan gudanar da saitin da ke sama, za ku ga cewa kowace shigarwa an ba da lakabi bisa bayanai daga log ɗin.

Abu daya da ya kamata a tuna shi ne cewa maido da lakabi tare da adadi mai yawa na dabi'u (kardinanci) na iya rage jinkirin Loki sosai. Wato, kada ku sanya, misali, user_id a cikin fihirisar. Kara karantawa game da wannan a cikin labarin "Yadda lakabi a Loki zai iya sa tambayoyin log cikin sauri da sauƙi" Amma wannan ba yana nufin cewa ba za ku iya bincika ta user_id ba tare da fihirisa ba. Kuna buƙatar amfani da masu tacewa yayin bincike ("kama" bayanan), kuma fihirisar nan tana aiki azaman mai gano rafi.

Kallon gungumen azaba

Loki na iya aiki azaman tushen bayanai don jadawali na Grafana ta amfani da LogQL. Ana goyan bayan fasalulluka masu zuwa:

• adadin - adadin rikodin da biyu;
• ƙidaya akan lokaci - adadin rikodin a cikin kewayon da aka ƙayyade.

Hakanan akwai ayyukan tarawa Sum, Avg da sauransu. Kuna iya gina hotuna masu rikitarwa, misali jadawali na adadin kurakuran HTTP:

Madaidaicin tushen bayanan Loki ya ɗan rage a cikin aiki idan aka kwatanta da tushen bayanan Prometheus (misali, ba za ku iya canza almara ba), amma ana iya haɗa Loki azaman tushen tushen Prometheus. Ban tabbata ba idan an rubuta halayen wannan, amma yin la'akari da martanin masu haɓakawa "Yadda ake saita Loki azaman tushen bayanan Prometheus? · Fitowa ta #1222 · grafana/loki”, alal misali, cikakken doka ne, kuma Loki ya dace da PromQL.

Ƙara Loki azaman tushen bayanai tare da nau'in Prometheus kuma ƙara URL / loki:

Kuma za mu iya yin jadawali, kamar muna aiki tare da awo daga Prometheus:

Ina tsammanin cewa rashin daidaituwa a cikin ayyuka na ɗan lokaci ne kuma masu haɓakawa za su gyara wannan a nan gaba.

Ma'auni

Loki yana ba da ikon cire ma'aunin ƙididdiga daga rajistan ayyukan kuma aika su zuwa Prometheus. Misali, log ɗin nginx yana ƙunshe da adadin bytes a kowane amsa, haka kuma, tare da wani gyare-gyare na daidaitaccen tsarin log ɗin, lokacin cikin daƙiƙan da ya ɗauki amsa. Ana iya fitar da wannan bayanan kuma a aika zuwa Prometheus.

Ƙara wani sashe zuwa promtail.yml:

- match:
   selector: '{request_type="api"}'
   stages:
     - metrics:
         http_nginx_response_time:
           type: Histogram
           description: "response time ms"
           source: response_time
           config:
             buckets: [0.010,0.050,0.100,0.200,0.500,1.0]
- match:
   selector: '{request_type=~"static|photo"}'
   stages:
     - metrics:
         http_nginx_response_bytes_sum:
           type: Counter
           description: "response bytes sum"
           source: bytes_out
           config:
             action: add
         http_nginx_response_bytes_count:
           type: Counter
           description: "response bytes count"
           source: bytes_out
           config:
             action: inc

Zaɓin yana ba ku damar ayyana da sabunta awo bisa bayanai daga taswirar da aka ciro. Ba a aika waɗannan ma'auni zuwa Loki ba - sun bayyana a ƙarshen Promtail/metrics. Dole ne a saita Prometheus don karɓar bayanan da aka karɓa a wannan matakin. A cikin misalin da ke sama, don request_type=“api” muna tattara ma'auni na histogram. Tare da irin wannan nau'in awo yana dacewa don samun kashi. Don a tsaye da hoto, muna tattara jimlar bytes da adadin layuka da muka karɓi bytes don ƙididdige matsakaicin.

Kara karantawa game da awo a nan.

Bude tashar jiragen ruwa akan Promtail:

promtail:
     image: grafana/promtail:1.4.1
     container_name: monitoring.promtail
     expose:
       - 9080
     ports:
       - "9080:9080"

Tabbatar cewa ma'auni tare da prefix na promtail_custom sun bayyana:

Saita Prometheus. Ƙara tallan aiki:

- job_name: 'promtail'
 scrape_interval: 10s
 static_configs:
   - targets: ['promtail:9080']

Kuma muna zana jadawali:

Ta wannan hanyar za ku iya gano, alal misali, tambayoyi huɗu mafi hankali. Hakanan zaka iya saita sa ido don waɗannan ma'auni.

Sikeli

Loki na iya zama ko dai a cikin yanayin binary guda ɗaya ko kuma cikin yanayin sharared (yanayin a kwance-mai iya daidaitawa). A cikin akwati na biyu, yana iya adana bayanai zuwa gajimare, kuma ana adana ƙugiya da fihirisa daban. Shafin 1.5 yana gabatar da ikon adanawa a wuri ɗaya, amma har yanzu ba a ba da shawarar yin amfani da shi a cikin samarwa ba.

Ana iya adana chunks a cikin ma'ajiyar da ta dace ta S3, kuma ana iya amfani da ma'aunin bayanai a kwance don adana fihirisa: Cassandra, BigTable ko DynamoDB. Sauran sassan Loki - Masu Rarraba (don rubuce-rubuce) da Querier (don tambayoyi) - ba su da ƙasa kuma suna da ma'auni a kwance.

A taron DevOpsDays Vancouver 2019, ɗaya daga cikin mahalarta Callum Styan ya sanar da cewa tare da Loki aikin nasa yana da petabytes na rajistan ayyukan tare da index na ƙasa da 1% na jimlar girman: "Yadda Loki Ke Daidaita Ma'auni da Logs - Kuma Yana Ajiye Ku Kuɗi".

Kwatanta Loki da ELK

Girman fihirisa

Don gwada girman fihirisar da aka samu, Na ɗauki rajistan ayyukan daga kwandon nginx wanda aka saita Pipeline na sama. Fayil ɗin log ɗin ya ƙunshi layuka 406 tare da jimillar ƙarar 624 MB. An samar da rajistan ayyukan cikin sa'a guda, kusan shigarwar 109 a sakan daya.

Misali na layi biyu daga log ɗin:

Lokacin da ELK ya nuna shi, wannan ya ba da girman index na 30,3 MB:

A cikin yanayin Loki, wannan ya haifar da kusan 128 KB na maƙasudi da kusan 3,8 MB na bayanai a cikin gungu. Yana da kyau a lura cewa log ɗin an ƙirƙira shi ta hanyar wucin gadi kuma ba shi da babban adadin bayanai. Gzip mai sauƙi akan ainihin Docker JSON log tare da bayanai ya ba da matsa lamba na 95,4%, kuma la'akari da cewa kawai tsabtace log nginx aka aika zuwa Loki kanta, matsawa har zuwa 4 MB abu ne mai fahimta. Jimlar adadin ƙima na musamman na alamun Loki shine 35, wanda ke bayanin ƙaramin girman ma'aunin. Don ELK an kuma share log ɗin. Don haka, Loki ya matsa ainihin bayanan da kashi 96%, kuma ELK da kashi 70%.

Amfanin ƙwaƙwalwar ajiya

Idan muka kwatanta dukan Prometheus da ELK tari, to Loki "ci" sau da yawa ƙasa. A bayyane yake cewa sabis na Go yana cinye ƙasa da sabis na Java, kuma kwatanta girman JVM Heap Elasticsearch da ƙwaƙwalwar da aka keɓe don Loki ba daidai ba ne, amma duk da haka yana da kyau a lura cewa Loki yana amfani da ƙarancin ƙwaƙwalwar ajiya. Amfaninsa na CPU ba a bayyane yake ba, amma kuma yana nan.

Speed

Loki “ya cinye” yana yin rajista da sauri. Gudun ya dogara da dalilai da yawa - irin nau'in rajistan ayyukan, yadda muke daɗaɗaɗɗen da muke yi a cikin rarraba su, hanyar sadarwa, faifai, da sauransu - amma tabbas ya fi ELK (a cikin gwaji na - kusan sau biyu). An bayyana wannan ta gaskiyar cewa Loki yana sanya ƙananan bayanai a cikin index kuma, bisa ga haka, yana ciyar da ɗan lokaci akan ƙididdiga. Tare da saurin bincike, yanayin ya kasance akasin haka: Loki a hankali yana raguwa akan bayanai da suka fi gigabytes da yawa, yayin da saurin binciken ELK bai dogara da girman bayanan ba.

Shiga Bincike

Loki ya yi ƙasa da ELK sosai dangane da iyawar binciken log. Grep tare da maganganu na yau da kullun yana da ƙarfi, amma yana ƙasa da babban bayanan bayanai. Rashin kewayon tambayoyin, tarawa kawai ta alamu, rashin iya bincika ba tare da lakabi ba - duk wannan yana iyakance mu wajen neman bayanan sha'awar Loki. Wannan ba yana nufin cewa ba za a iya samun wani abu ta amfani da Loki ba, amma yana bayyana ma'anar aiki tare da rajistan ayyukan lokacin da kuka fara samun matsala a cikin ginshiƙi na Prometheus, sannan ku yi amfani da waɗannan alamun don neman abin da ya faru a cikin rajistan ayyukan.

dubawa

Da farko, yana da kyau (yi hakuri, ba za a iya tsayayya ba). Grafana yana da kyakkyawar mu'amala mai kyau, amma Kibana ya fi fasali da wadata.

Ribobi da fursunoni na Loki

Ɗaya daga cikin fa'idodin shine Loki yana haɗawa tare da Prometheus, don haka muna samun awo da faɗakarwa daga cikin akwatin. Ya dace don tattara rajistan ayyukan da adana su daga Kubernetes Pods, saboda yana da binciken sabis da aka gada daga Prometheus kuma yana haɗe ta atomatik.

Ƙarƙashin ƙasa shine takaddun rauni. Wasu abubuwa, alal misali, fasali da iyawar Promtail, na gano kawai a cikin tsarin nazarin lambar, sa'a shine tushen budewa. Wani rashin lahani shine rashin iya tantancewa. Misali, Loki ba zai iya tantance rajistan ayyukan layi daya ba. Wani rashin lahani shine Loki fasaha ce ta ɗan ƙaramin matashi (sakin 1.0 ya kasance a cikin Nuwamba 2019).

ƙarshe

Loki shine fasaha mai ban sha'awa 100% wanda ya dace da ƙananan ayyuka masu girma da matsakaici, yana ba ku damar magance matsalolin da yawa na haɗin log, binciken log, saka idanu da bincike na log.

Ba ma amfani da Loki a cikin Badoo saboda muna da tarin ELK wanda ya dace da mu kuma wanda ya cika da matsaloli daban-daban na al'ada tsawon shekaru. A gare mu, toshewar tuntuɓe yana nema ta gungumen azaba. Tare da kusan 100 GB na rajistan ayyukan a kowace rana, yana da mahimmanci a gare mu mu sami damar samun komai da ɗan ƙara kaɗan kuma muyi sauri. Don tsarawa da saka idanu, muna amfani da wasu mafita waɗanda aka keɓance da bukatunmu kuma an haɗa su da juna. Tarin Loki yana da fa'idodi na zahiri, amma ba zai ba mu fiye da yadda muke da shi ba, kuma fa'idodinsa tabbas ba zai wuce kuɗin ƙaura ba.

Kuma ko da yake bayan bincike ya bayyana a fili cewa ba za mu iya amfani da Loki ba, muna fatan wannan post ɗin zai taimaka muku a cikin zaɓinku.

Wurin ajiya tare da lambar da aka yi amfani da shi a cikin labarin yana nan a nan.

source: www.habr.com