Ke kūkulu ʻana i ka ʻikepili i hoʻonohonoho ʻole ʻia me GROK
Inā ʻoe e hoʻohana nei i ka waihona Elastic (ELK) a makemake ʻoe i ka palapala ʻana i nā logstash maʻamau i Elasticsearch, a laila nou kēia pou.
ʻO ka ELK stack kahi acronym no ʻekolu papahana open source: Elasticsearch, Logstash a me Kibana. Hoʻohui pū lākou i kahi kahua hoʻokele log.
- Elasticsearch he ʻōnaehana ʻimi a me ka nānā ʻana.
- ʻO Logstash he pipeline hoʻoponopono ʻikepili ʻaoʻao server e hoʻokomo i ka ʻikepili mai nā kumu he nui i ka manawa like, hoʻololi iā ia, a laila hoʻouna iā ia i kahi "stash" e like me Elasticsearch.
- kibana hiki i nā mea hoʻohana ke nānā i ka ʻikepili me ka hoʻohana ʻana i nā pakuhi a me nā kiʻi ma Elasticsearch.
Kau hele mai ma hope a he mea lawe ʻikepili māmā. ʻO ka hoʻokomo ʻana o Beats i hoʻololi i ka Elk Stack i ka Elastic Stack, akā ʻaʻole ia ke kumu.
ʻO kēia ʻatikala e pili ana iā Grok, kahi hiʻohiʻona i Logstash hiki ke hoʻololi i kāu mau lāʻau ma mua o ka hoʻouna ʻia ʻana i ka stash. No kā mākou kumu, e kamaʻilio wale wau e pili ana i ka hoʻoili ʻana i ka ʻikepili mai Logstash i Elasticsearch.
He kānana ʻo Grok i loko o Logstash i hoʻohana ʻia e hoʻokaʻawale i ka ʻikepili i kūkulu ʻole ʻia i kahi mea i kūkulu ʻia a hiki ke nīnau ʻia. Noho ʻo ia ma luna o kahi ʻōlelo maʻamau (regex) a hoʻohana i nā ʻano kikokikona e hoʻohālikelike i nā kaula i nā faila log.
E like me kā mākou e ʻike ai ma nā ʻāpana aʻe, ʻo ka hoʻohana ʻana iā Grok e hoʻololi nui i ka wā e pili ana i ka hoʻokele log kūpono.
Me ka ʻole o Grok ʻaʻole i hoʻonohonoho ʻia kāu ʻikepili log
Me ka ʻole o Grok, ke hoʻouna ʻia nā lāʻau mai Logstash i Elasticsearch a hāʻawi ʻia ma Kibana, ʻike wale ʻia lākou i ka waiwai memo.
Paʻakikī ka nīnau ʻana i ka ʻike koʻikoʻi ma kēia kūlana no ka mea mālama ʻia nā ʻikepili log a pau i hoʻokahi kī. ʻOi aku ka maikaʻi inā ʻoi aku ka maikaʻi o ka hoʻonohonoho ʻana i nā memo log.
ʻIke i hoʻonohonoho ʻole ʻia mai nā lāʻau
localhost GET /v2/applink/5c2f4bb3e9fda1234edc64d 400 46ms 5bc6e716b5d6cb35fc9687c0Inā ʻoe e nānā pono i ka ʻikepili maka, e ʻike ʻoe aia i loko o nā ʻāpana like ʻole, i hoʻokaʻawale ʻia kēlā me kēia me kahi ākea.
No nā mea hoʻomohala ʻoi aku ka ʻike, hiki paha iā ʻoe ke koho i ke ʻano o kēlā me kēia ʻāpana a me ke ʻano o kēlā memo log mai ke kelepona API. Hōʻike ʻia ka hōʻike o kēlā me kēia mea ma lalo nei.
Nānā i kūkulu ʻia o kā mākou ʻikepili
- localhost == kaiapuni
- GET == ala
- /v2/applink/5c2f4bb3e9fda1234edc64d == url
- 400 == pane_kūlana
- 46ms == pane_manawa
- 5bc6e716b5d6cb35fc9687c0 == user_id
E like me kā mākou e ʻike ai i ka ʻikepili i kūkulu ʻia, aia kahi kauoha no nā lāʻau i kūkulu ʻole ʻia. ʻO ka hana aʻe, ʻo ia ka hoʻoponopono polokalamu o ka ʻikepili maka. ʻO kēia kahi e ʻālohilohi ai ʻo Grok.
Nā Kūlana Grok
ʻO nā hiʻohiʻona Grok i kūkulu ʻia
Hele mai ʻo Logstash me 100 mau hiʻohiʻona i kūkulu ʻia no ka hoʻonohonoho ʻana i ka ʻikepili i kūkulu ʻole ʻia. Pono ʻoe e hoʻohana pono i kēia inā hiki i nā syslogs maʻamau e like me apache, linux, haproxy, aws a pēlā aku.
Eia naʻe, he aha ka hopena inā loaʻa iā ʻoe nā lāʻau maʻamau e like me ka laʻana ma luna? Pono ʻoe e kūkulu i kāu iho Grok template.
Nā hoʻohālike Grok maʻamau
Pono ʻoe e hoʻāʻo e kūkulu i kāu iho Grok template. Ua hoʻohana au и .
E hoʻomanaʻo ʻo ka syntax template Grok penei: %{SYNTAX:SEMANTIC}
ʻO ka mea mua aʻu i hoʻāʻo ai e hele i ka pā Mob i ka Grok debugger. Manaʻo wau he mea ʻoluʻolu inā hiki i kēia hāmeʻa ke hoʻohua i kahi kumu Grok, akā ʻaʻole maikaʻi loa ia no ka mea ua loaʻa ʻelua mau pāʻani.
Me ka hoʻohana ʻana i kēia ʻike, hoʻomaka wau e hana i kaʻu iho ma ka Grok debugger me ka hoʻohana ʻana i ka syntax i loaʻa ma ka ʻaoʻao Elastic Github.
Ma hope o ka pāʻani ʻana me nā syntax ʻokoʻa, ua hiki iaʻu ke kūkulu i ka ʻikepili log e like me kaʻu i makemake ai.
ʻO Grok Debugger Link
kikokikona kumu:
localhost GET /v2/applink/5c2f4bb3e9fda1234edc64d 400 46ms 5bc6e716b5d6cb35fc9687c0kumu:
%{WORD:environment} %{WORD:method} %{URIPATH:url} %{NUMBER:response_status} %{WORD:response_time} %{USERNAME:user_id}He aha ka hopena
{
"environment": [
[
"localhost"
]
],
"method": [
[
"GET"
]
],
"url": [
[
"/v2/applink/5c2f4bb3e9fda1234edc64d"
]
],
"response_status": [
[
"400"
]
],
"BASE10NUM": [
[
"400"
]
],
"response_time": [
[
"46ms"
]
],
"user_id": [
[
"5bc6e716b5d6cb35fc9687c0"
]
]
}Me ka Grok template a me ka palapala ʻikepili i ka lima, ʻo ka hana hope e hoʻohui iā ia i Logstash.
Hoʻohou i ka faila hoʻonohonoho Logstash.conf
Ma ke kikowaena kahi āu i hoʻokomo ai i ka waihona ELK, e hele i ka hoʻonohonoho Logstash:
sudo vi /etc/logstash/conf.d/logstash.confHoʻopili i nā hoʻololi.
input {
file {
path => "/your_logs/*.log"
}
}
filter{
grok {
match => { "message" => "%{WORD:environment} %{WORD:method} %{URIPATH:url} %{NUMBER:response_status} %{WORD:response_time} %{USERNAME:user_id}"}
}
}
output {
elasticsearch {
hosts => [ "localhost:9200" ]
}
}Ma hope o ka mālama ʻana i kāu mau hoʻololi, e hoʻomaka hou iā Logstash a nānā i kona kūlana e ʻike pono e hana mau ana.
sudo service logstash restart
sudo service logstash statusʻO ka hope, e hōʻoia i ka hopena o nā loli, E ʻoluʻolu e hōʻano hou i kāu helu Elasticsearch no Logstash ma Kibana!
Me Grok, ua hoʻonohonoho ʻia kāu ʻikepili log!
E like me kā mākou e ʻike ai ma ke kiʻi ma luna, hiki iā Grok ke hoʻohālikelike i ka ʻikepili log me Elasticsearch. He mea maʻalahi kēia i ka mālama ʻana i nā lāʻau a me ka huli ʻana i ka ʻike. Ma kahi o ka ʻeli ʻana i nā faila log e debug, hiki iā ʻoe ke kānana ma o ka mea āu e ʻimi nei, e like me kahi kaiapuni a i ʻole url.
E ho'āʻo i nā ʻōlelo Grok! Inā loaʻa iā ʻoe kahi ala ʻē aʻe e hana ai i kēia a i ʻole nā pilikia me nā hiʻohiʻona ma luna, e kākau wale i kahi manaʻo ma lalo e haʻi mai iaʻu.
Mahalo no ka heluhelu ʻana—a e ʻoluʻolu e hahai mai iaʻu ma ʻaneʻi ma Medium no nā ʻatikala ʻenehana lako polokalamu hoihoi!
Nā Punawai
PS
Kaila Telegram na
Source: www.habr.com