Kupanga deta yosasinthika ndi GROK
Ngati mukugwiritsa ntchito stack ya Elastic (ELK) ndipo mukufuna kupanga mapu a Logstash ku Elasticsearch, ndiye kuti izi ndi zanu.
ELK stack ndi chidule cha mapulojekiti atatu otseguka: Elasticsearch, Logstash ndi Kibana. Onse pamodzi amapanga nsanja yoyendetsera chipika.
- Elasticsearch ndi kufufuza ndi kusanthula dongosolo.
- Logstash ndi mapaipi opangira data pa seva omwe amalowetsa data kuchokera kuzinthu zingapo nthawi imodzi, kuisintha, kenako kuitumiza ku "stash" monga Elasticsearch.
- Kibana imalola ogwiritsa ntchito kuwona deta pogwiritsa ntchito ma chart ndi ma graph mu Elasticsearch.
Nkhwangwa anabwera pambuyo pake ndipo ndi wopepuka wotumiza deta. Kuyambika kwa Beats kunasintha Elk Stack kukhala Elastic Stack, koma sindiyo mfundo.
Nkhaniyi ikunena za Grok, yomwe ili mu Logstash yomwe ingasinthe zipika zanu zisanatumizidwe ku stash. Pazolinga zathu, ndingolankhula za kukonza deta kuchokera ku Logstash kupita ku Elasticsearch.
Grok ndi fyuluta mkati mwa Logstash yomwe imagwiritsidwa ntchito kusanthula deta yosasinthika kukhala chinthu chokhazikika komanso chofunsidwa. Imakhala pamwamba pa mawu okhazikika (regex) ndipo imagwiritsa ntchito zolemba kuti zigwirizane ndi zingwe mumafayilo alogi.
Monga momwe tidzawonera m'zigawo zotsatirazi, kugwiritsa ntchito Grok kumapanga kusiyana kwakukulu pankhani yoyendetsera bwino log.
Popanda Grok deta yanu yolembera sinapangidwe
Popanda Grok, zipika zikatumizidwa kuchokera ku Logstash kupita ku Elasticsearch ndikuperekedwa ku Kibana, zimangowoneka pamtengo wa uthenga.
Kufunsa zambiri zatanthauzo pankhaniyi ndikovuta chifukwa zonse za chipika zimasungidwa mu kiyi imodzi. Zingakhale bwino ngati mauthenga a chipika anali okonzedwa bwino.
Deta yosakonzedwa kuchokera ku zipika
localhost GET /v2/applink/5c2f4bb3e9fda1234edc64d 400 46ms 5bc6e716b5d6cb35fc9687c0Ngati muyang'anitsitsa deta yaiwisi, mudzawona kuti ili ndi zigawo zosiyana, zomwe zimalekanitsidwa ndi danga.
Kwa opanga odziwa zambiri, mutha kulingalira zomwe gawo lililonse limatanthauza komanso zomwe uthenga wa chipikawu ukuchokera pa foni ya API. Chiwonetsero cha chinthu chilichonse chafotokozedwa pansipa.
Mawonekedwe opangidwa ndi data yathu
- localhost == chilengedwe
- GET == njira
- β /v2/applink/5c2f4bb3e9fda1234edc64d == url
- 400 == response_status
- 46ms == response_time
- β 5bc6e716b5d6cb35fc9687c0 == user_id
Monga tikuwonera mu data yokonzedwa, pali dongosolo la zipika zosasinthika. Chotsatira ndikukonza mapulogalamu a data yaiwisi. Apa ndi pamene Grok amawala.
Zithunzi za Grok
Zomangidwa mu Grok templates
Logstash imabwera ndi ma tempulo opitilira 100 opangidwa kuti azitha kupanga zomwe sizinapangike. Muyenera kugwiritsa ntchito mwayi uwu ngati kuli kotheka kwa ma syslogs ngati apache, linux, haproxy, aws ndi zina zotero.
Komabe, chimachitika ndi chiyani mukakhala ndi zipika zachizolowezi monga momwe zilili pamwambapa? Muyenera kupanga template yanu ya Grok.
Zithunzi za Custom Grok
Muyenera kuyesa kupanga template yanu ya Grok. Ndinagwiritsa ntchito ΠΈ .
Dziwani kuti template ya Grok syntax ili motere: %{SYNTAX:SEMANTIC}
Chinthu choyamba chimene ndinayesera kuchita chinali kupita ku tabu Discover mu Grok debugger. Ndinkaganiza kuti zingakhale zabwino ngati chida ichi chitha kupanga chojambula cha Grok, koma sichinali chothandiza chifukwa chinangopeza machesi awiri.
Pogwiritsa ntchito izi, ndinayamba kupanga template yanga mu Grok debugger pogwiritsa ntchito syntax yomwe imapezeka patsamba la Elastic Github.
Nditasewera mozungulira ndi ma syntaxes osiyanasiyana, pomaliza pake ndidatha kupanga zidziwitso momwe ndimafunira.
Grok Debugger Link
Mawu oyamba:
localhost GET /v2/applink/5c2f4bb3e9fda1234edc64d 400 46ms 5bc6e716b5d6cb35fc9687c0Chitsanzo:
%{WORD:environment} %{WORD:method} %{URIPATH:url} %{NUMBER:response_status} %{WORD:response_time} %{USERNAME:user_id}Zomwe zidachitika pomaliza
{
"environment": [
[
"localhost"
]
],
"method": [
[
"GET"
]
],
"url": [
[
"/v2/applink/5c2f4bb3e9fda1234edc64d"
]
],
"response_status": [
[
"400"
]
],
"BASE10NUM": [
[
"400"
]
],
"response_time": [
[
"46ms"
]
],
"user_id": [
[
"5bc6e716b5d6cb35fc9687c0"
]
]
}Ndi template ya Grok ndi ma data omwe ali m'manja, chomaliza ndikuwonjezera ku Logstash.
Kusintha fayilo ya Logstash.conf
Pa seva pomwe mudayika stack ya ELK, pitani ku kasinthidwe ka Logstash:
sudo vi /etc/logstash/conf.d/logstash.confMatani zosintha.
input {
file {
path => "/your_logs/*.log"
}
}
filter{
grok {
match => { "message" => "%{WORD:environment} %{WORD:method} %{URIPATH:url} %{NUMBER:response_status} %{WORD:response_time} %{USERNAME:user_id}"}
}
}
output {
elasticsearch {
hosts => [ "localhost:9200" ]
}
}Mukasunga zosintha zanu, yambitsaninso Logstash ndikuwona momwe ilili kuti muwonetsetse kuti ikugwirabe ntchito.
sudo service logstash restart
sudo service logstash statusPomaliza, kuwonetsetsa kuti kusintha kwachitika, Onetsetsani kuti mwasintha index yanu ya Elasticsearch ya Logstash ku Kibana!
Ndi Grok, deta yanu ya chipika idapangidwa!
Monga tikuonera pachithunzi pamwambapa, Grok amatha kufananitsa deta ya chipika ndi Elasticsearch. Izi zimapangitsa kuti zikhale zosavuta kusamalira zipika ndikufunsa mwachangu zambiri. M'malo mokumba mafayilo a log kuti musinthe, mutha kungosefa ndi zomwe mukuyang'ana, monga chilengedwe kapena url.
Yesani mawu a Grok! Ngati muli ndi njira ina yochitira izi kapena muli ndi vuto lililonse ndi zitsanzo pamwambapa, ingolembani ndemanga pansipa kuti mundidziwitse.
Zikomo powerenga-ndipo chonde nditsatireni pano pa Medium kuti mupeze zolemba zosangalatsa zamapulogalamu!
Zida
PS
Njira ya Telegraph ndi
Source: www.habr.com