Ukwakha idatha engahlelekile nge-GROK
Uma usebenzisa isitaki se-Elastic (ELK) futhi unentshisekelo yokumepha amalogi angokwezifiso e-Logstash ku-Elasticsearch, lokhu okuthunyelwe kungokwakho.
Isitaki se-ELK siyisifinyezo samaphrojekthi amathathu omthombo ovulekile: i-Elasticsearch, i-Logstash ne-Kibana. Bendawonye bakha inkundla yokuphatha amalogi.
- Islastiki wuhlelo lokusesha nokuhlaziya.
- Logstash iyipayipi yokucubungula idatha eseceleni kweseva engenisa idatha evela emithonjeni eminingi ngesikhathi esisodwa, iyiguqule, bese iyithumela "ku-stash" njenge-Elasticsearch.
- Kibana ivumela abasebenzisi ukuthi babone idatha besebenzisa amashadi namagrafu ku-Elasticsearch.
Beats yeza kamuva futhi ingumthumeli wedatha engasindi. Ukwethulwa kwe-Beats kuguqule i-Elk Stack yaba yi-Elastic Stack, kodwa akulona iphuzu.
Lesi sihloko sikhuluma nge-Grok, okuyisici ku-Logstash esingaguqula izingodo zakho ngaphambi kokuthi zithunyelwe ku-stash. Ngezinjongo zethu, ngizokhuluma kuphela ngokucubungula idatha kusuka ku-Logstash kuya ku-Elasticsearch.
I-Grok isihlungi esingaphakathi kwe-Logstash esisetshenziselwa ukucozulula idatha engahlelekile ibe into ehlelekile futhi ebuzwayo. Ihlala phezu kwenkulumo evamile (regex) futhi isebenzisa amaphethini ombhalo ukuze ifane neyunithi yezinhlamvu kumafayela okungena.
Njengoba sizobona ezigabeni ezilandelayo, ukusebenzisa i-Grok kwenza umehluko omkhulu uma kuziwa ekuphathweni kwelogi okuphumelelayo.
Ngaphandle kwe-Grok idatha yakho yelogi ayihlelekile
Ngaphandle kwe-Grok, lapho amalogi ethunyelwa esuka ku-Logstash eya ku-Elasticsearch futhi enikezwa ngesi-Kibana, avela kuphela enanini lomlayezo.
Ukubuza ulwazi olubalulekile kulesi simo kunzima ngoba yonke idatha yelogi igcinwa kukhiye owodwa. Kungaba ngcono uma imilayezo yelogi ihlelwe kangcono.
Idatha engahlelekile evela kulogi
localhost GET /v2/applink/5c2f4bb3e9fda1234edc64d 400 46ms 5bc6e716b5d6cb35fc9687c0Uma ubhekisisa idatha eluhlaza, uzobona ukuthi empeleni iqukethe izingxenye ezihlukene, ngayinye ihlukaniswe yisikhala.
Konjiniyela abanolwazi olunzulu, ungaqagela ukuthi ingxenye ngayinye isho ukuthini nokuthi lowo mlayezo welogi uvela ocingweni lwe-API. Ukwethulwa kwento ngayinye kuvezwe ngezansi.
Ukubuka okuhleliwe kwedatha yethu
- localhost == imvelo
- THOLA == indlela
- β /v2/applink/5c2f4bb3e9fda1234edc64d == url
- 400 ==isimo_sempendulo
- 46ms == impendulo_isikhathi
- β 5bc6e716b5d6cb35fc9687c0 == user_id
Njengoba sibona kudatha ehleliwe, kukhona i-oda lamalogi angahlelekile. Isinyathelo esilandelayo ukucutshungulwa kwesofthiwe yedatha eluhlaza. Yilapho i-Grok ikhanya khona.
Grok Izifanekiso
Izifanekiso ezakhelwe ngaphakathi ze-Grok
I-Logstash iza nezifanekiso ezakhelwe ngaphakathi ezingaphezu kwe-100 zokuhlela idatha engahlelekile. Kufanele nakanjani usebenzise lokhu noma nini lapho kungenzeka kuma-syslogs ajwayelekile afana ne-apache, i-linux, i-haproxy, i-aws njalonjalo.
Nokho, kwenzekani uma unamalogi angokwezifiso njengasesibonelweni esingenhla? Kufanele wakhe isifanekiso sakho se-Grok.
Izifanekiso ze-Grok zangokwezifiso
Kufanele uzame ukwakha isifanekiso sakho se-Grok. ngisebenzise ΠΈ .
Qaphela ukuthi i-syntax yesifanekiso se-Grok imi kanje: %{SYNTAX:SEMANTIC}
Into yokuqala engizame ukuyenza kwaba ukuya kuthebhu Discover ku-debugger ye-Grok. Bengicabanga ukuthi kuzoba kuhle uma leli thuluzi likwazi ukukhiqiza ngokuzenzakalelayo iphethini ye-Grok, kodwa alizange libe wusizo kakhulu njengoba lithole ukufana okubili kuphela.
Ngisebenzisa lokhu kutholwa, ngaqala ukudala isifanekiso sami ku-Grok debugger ngisebenzisa i-syntax etholakala ekhasini le-Elastic Github.
Ngemva kokudlala ngama-syntaxes ahlukene, ekugcineni ngikwazile ukuhlela idatha yelogi ngendlela engangifuna ngayo.
Isixhumanisi se-Grok Debugger
Umbhalo wangempela:
localhost GET /v2/applink/5c2f4bb3e9fda1234edc64d 400 46ms 5bc6e716b5d6cb35fc9687c0Iphethini:
%{WORD:environment} %{WORD:method} %{URIPATH:url} %{NUMBER:response_status} %{WORD:response_time} %{USERNAME:user_id}Kwenzekani ekugcineni
{
"environment": [
[
"localhost"
]
],
"method": [
[
"GET"
]
],
"url": [
[
"/v2/applink/5c2f4bb3e9fda1234edc64d"
]
],
"response_status": [
[
"400"
]
],
"BASE10NUM": [
[
"400"
]
],
"response_time": [
[
"46ms"
]
],
"user_id": [
[
"5bc6e716b5d6cb35fc9687c0"
]
]
}Ngesifanekiso se-Grok kanye nedatha emephu esandleni, isinyathelo sokugcina ukuyengeza ku-Logstash.
Ibuyekeza ifayela lokumisa le-Logstash.conf
Kuseva lapho ofake khona isitaki se-ELK, iya ekucushweni kwe-Logstash:
sudo vi /etc/logstash/conf.d/logstash.confNamathisela izinguquko.
input {
file {
path => "/your_logs/*.log"
}
}
filter{
grok {
match => { "message" => "%{WORD:environment} %{WORD:method} %{URIPATH:url} %{NUMBER:response_status} %{WORD:response_time} %{USERNAME:user_id}"}
}
}
output {
elasticsearch {
hosts => [ "localhost:9200" ]
}
}Ngemva kokulondoloza izinguquko zakho, qala kabusha i-Logstash futhi uhlole isimo sayo ukuze uqiniseke ukuthi isasebenza.
sudo service logstash restart
sudo service logstash statusOkokugcina, ukwenza isiqiniseko sokuthi izinguquko seziqalile ukusebenza, Qiniseka ukuthi ubuyekeza inkomba yakho ye-Elasticsearch ye-Logstash e-Kibana!
Nge-Grok, idatha yakho yelogi yakhiwe!
Njengoba sibona esithombeni esingenhla, i-Grok iyakwazi ukufanisa ngokuzenzakalelayo idatha yelogi ne-Elasticsearch. Lokhu kwenza kube lula ukuphatha amalogi kanye nokubuza ngokushesha ulwazi. Esikhundleni sokumba amafayela elogi ukuze ulungise iphutha, ungavele uhlunge ngalokho okufunayo, njengendawo ezungezile noma i-url.
Zama izinkulumo ze-Grok! Uma unenye indlela yokwenza lokhu noma unezinkinga ngezibonelo ezingenhla, mane ubhale amazwana ngezansi ukuze ungazise.
Ngiyabonga ngokufundaβfuthi ngicela ungilandele lapha ku-Medium ukuze uthole izindatshana ezithakazelisayo zobunjiniyela besofthiwe!
Izinsiza
PS
Isiteshi socingo ngo
Source: www.habr.com