I ka hope loa halawai makou ELK ahu, he aha nā huahana lako polokalamu. A ʻo ka hana mua a kahi ʻenekinia e hana nei me ka ELK stack e hoʻouna ana i nā lāʻau no ka mālama ʻana ma ka elasticsearch no ka nānā ʻana ma hope. Eia nō naʻe, he lehelehe wale nō kēia, nā hale kūʻai elasticsearch i nā lāʻau ma ke ʻano o nā palapala me kekahi mau māla a me nā waiwai, ʻo ia hoʻi, pono ka ʻenekinia e hoʻohana i nā mea hana like ʻole e hoʻopau i ka leka i hoʻouna ʻia mai nā ʻōnaehana hope. Hiki ke hana ʻia kēia ma nā ʻano he nui - e kākau i kahi papahana iā ʻoe iho e hoʻohui i nā palapala i ka waihona me ka hoʻohana ʻana i ka API, a i ʻole e hoʻohana i nā hoʻonā i hana ʻia. Ma kēia papa e noʻonoʻo mākou i ka hopena ʻO Logstash, he ʻāpana ia o ka waihona ELK. E nānā mākou pehea e hiki ai iā mākou ke hoʻouna i nā lāʻau mai nā ʻōnaehana endpoint i Logstash, a laila e hoʻonohonoho mākou i kahi faila hoʻonohonoho e hoʻopau a hoʻihoʻi i ka waihona Elasticsearch. No ka hana ʻana i kēia, lawe mākou i nā lāʻau mai ka pā ahi Check Point e like me ka ʻōnaehana komo.
ʻAʻole uhi ka papa i ka hoʻokomo ʻana o ELK stack, no ka mea he nui nā ʻatikala ma kēia kumuhana; e noʻonoʻo mākou i ka ʻāpana hoʻonohonoho.
E hoʻolālā i kahi hoʻolālā hana no ka hoʻonohonoho Logstash:
- ʻO ka nānā ʻana i ka elasticsearch e ʻae i nā lāʻau (e nānā i ka hana a me ka wehe ʻana o ke awa).
- Noʻonoʻo mākou pehea e hiki ai iā mākou ke hoʻouna i nā hanana iā Logstash, koho i kahi ala, a hoʻokō.
- Hoʻonohonoho mākou i ka Input i ka faila hoʻonohonoho Logstash.
- Hoʻonohonoho mākou i ka Output i ka faila hoʻonohonoho Logstash ma ke ʻano debug i mea e maopopo ai ke ʻano o ka memo log.
- Hoʻonohonoho i kāna kānana.
- Hoʻonohonoho i ka Output kūpono ma ElasticSearch.
- Hoʻomaka ka Logstash.
- Ke nānā nei i nā lāʻau ma Kibana.
E nānā pono kākou i kēlā me kēia wahi:
ʻO ka nānā ʻana i kēlā elasticsearch e ʻae i nā lāʻau
No ka hana ʻana i kēia, hiki iā ʻoe ke hoʻohana i ke kauoha curl e nānā i ke komo ʻana iā Elasticsearch mai ka ʻōnaehana kahi i kau ʻia ai ʻo Logstash. Inā ua hoʻonohonoho ʻia kāu hōʻoia, a laila hoʻololi mākou i ka mea hoʻohana / ʻōlelo huna ma o ka curl, e kuhikuhi ana i ke awa 9200 inā ʻaʻole ʻoe i hoʻololi. Inā loaʻa iā ʻoe kahi pane e like me ka mea ma lalo nei, a laila ua hoʻonohonoho pono nā mea a pau.
[elastic@elasticsearch ~]$ curl -u <<user_name>> : <<password>> -sS -XGET "<<ip_address_elasticsearch>>:9200"
{
"name" : "elastic-1",
"cluster_name" : "project",
"cluster_uuid" : "sQzjTTuCR8q4ZO6DrEis0A",
"version" : {
"number" : "7.4.1",
"build_flavor" : "default",
"build_type" : "rpm",
"build_hash" : "fc0eeb6e2c25915d63d871d344e3d0b45ea0ea1e",
"build_date" : "2019-10-22T17:16:35.176724Z",
"build_snapshot" : false,
"lucene_version" : "8.2.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
[elastic@elasticsearch ~]$
Inā ʻaʻole i loaʻa ka pane, a laila aia kekahi mau ʻano hewa: ʻaʻole holo ke kaʻina hana elasticsearch, hōʻike ʻia ke awa hewa ʻole, a i ʻole ua ālai ʻia ke awa e kahi pā ahi ma ke kikowaena kahi i hoʻokomo ʻia ai elasticsearch.
E nānā kākou pehea e hiki ai iā ʻoe ke hoʻouna i nā lāʻau iā Logstash mai kahi pā ahi kiko
Mai ka kikowaena hoʻokele Check Point hiki iā ʻoe ke hoʻouna i nā lāʻau iā Logstash ma o syslog me ka hoʻohana ʻana i ka pono log_exporter, hiki iā ʻoe ke heluhelu hou e pili ana iā ia maanei , eia mākou e waiho wale i ke kauoha e hana i ke kahawai:
cp_log_export hoʻohui i ka inoa check_point_syslog target-server > target-port 5555 protocol tcp format generic read-mode semi-unified
> - ka helu o ke kikowaena kahi e holo ai ʻo Logstash, target-port 5555 - ke awa kahi e hoʻouna aku ai mākou i nā lāʻau, hoʻouna i nā lāʻau ma o tcp hiki ke hoʻouka i ka server, no laila i kekahi mau manawa ʻoi aku ka pololei o ka hoʻohana ʻana i ka udp .
Hoʻonohonoho i ka INPUT ma ka faila hoʻonohonoho Logstash
Ma ka paʻamau, aia ka faila hoʻonohonoho i ka papa kuhikuhi /etc/logstash/conf.d/. Aia ka waihona hoʻonohonoho he 3 mau ʻāpana koʻikoʻi: INPUT, FILTER, OUTPUT. IN hoʻokomo o hōʻike mākou i kahi e lawe ai ka ʻōnaehana i nā lāʻau mai, in FILT parse i ka log - hoʻonohonoho pehea e puʻunaue ai i ka memo i loko o nā māla a me nā waiwai, ma ia auoiaea hoʻonohonoho mākou i ke kahawai puka - kahi e hoʻouna ʻia ai nā log parsed.
ʻO ka mea mua, e hoʻonohonoho i ka INPUT, e noʻonoʻo i kekahi o nā ʻano i hiki ke - faila, tcp a me exe.
Tcp:
input {
tcp {
port => 5555
host => “10.10.1.205”
type => "checkpoint"
mode => "server"
}
}
mode => "server"
E hōʻike ana e ʻae ana ʻo Logstash i nā pilina.
awa => 5555
host => "10.10.1.205"
ʻAe mākou i nā pilina ma o ka IP address 10.10.1.205 (Logstash), port 5555 - pono e ʻae ʻia ke awa e ke kulekele pā ahi.
ʻano => "wahi nānā"
Hōʻailona mākou i ka palapala, kūpono loa inā loaʻa iā ʻoe kekahi mau pilina e hiki mai ana. A laila, no kēlā me kēia pilina hiki iā ʻoe ke kākau i kāu kānana ponoʻī me ka hoʻohana ʻana i ka logical if construct.
Kōnae:
input {
file {
path => "/var/log/openvas_report/*"
type => "openvas"
start_position => "beginning"
}
}
ʻO ka wehewehe ʻana o nā hoʻonohonoho:
ala => "/var/log/openvas_report/*"
Hōʻike mākou i ka papa kuhikuhi kahi e heluhelu ʻia ai nā faila.
ʻano => "openvas"
ʻAno hanana.
start_position => "hoʻomaka"
Ke hoʻololi nei i kahi faila, heluhelu ʻo ia i ka faila holoʻokoʻa; inā ʻoe e hoʻonoho i ka "hope", kali ka ʻōnaehana no nā moʻolelo hou e ʻike ʻia ma ka hope o ka faila.
Lunahana:
input {
exec {
command => "ls -alh"
interval => 30
}
}
Ke hoʻohana nei i kēia hoʻokomo, hoʻomaka ʻia kahi kauoha shell (wale nō!)
kauoha => "ls -alh"
ʻO ke kauoha nona ka huahana a mākou e makemake ai.
waena => 30
Kauoha i ka manawa hoʻopaʻa inoa i kekona.
No ka loaʻa ʻana o nā lāʻau mai ka pā ahi, hoʻopaʻa inoa mākou i kahi kānana tcp ai ole ia, pud, e pili ana i ka hoʻouna ʻia ʻana o nā lāʻau iā Logstash.
Hoʻonohonoho mākou i ka Output i ka faila hoʻonohonoho Logstash ma ke ʻano debug i mea e maopopo ai ke ʻano o ka memo log
Ma hope o kā mākou hoʻonohonoho ʻana i ka INPUT, pono mākou e hoʻomaopopo i ke ʻano o ka memo log a me nā ʻano e pono ai ke hoʻohana ʻia e hoʻonohonoho i ka kānana log (parser).
No ka hana ʻana i kēia, e hoʻohana mākou i kahi kānana e hoʻopuka i ka hopena i stdout i mea e ʻike ai i ka memo mua; ʻo ka faila hoʻonohonoho piha i kēia manawa e like me kēia:
input
{
tcp
{
port => 5555
type => "checkpoint"
mode => "server"
host => “10.10.1.205”
}
}
output
{
if [type] == "checkpoint"
{
stdout { codec=> json }
}
}
E holo i ke kauoha e nānā:
sudo /usr/share/logstash/bin//logstash -f /etc/logstash/conf.d/checkpoint.conf
ʻIke mākou i ka hopena, hiki ke kaomi i ke kiʻi:
Inā ʻoe e kope iā ia e like me kēia:
action="Accept" conn_direction="Internal" contextnum="1" ifdir="outbound" ifname="bond1.101" logid="0" loguid="{0x5dfb8c13,0x5,0xfe0a0a0a,0xc0000000}" origin="10.10.10.254" originsicname="CN=ts-spb-cpgw-01,O=cp-spb-mgmt-01.tssolution.local.kncafb" sequencenum="8" time="1576766483" version="5" context_num="1" dst="10.10.10.10" dst_machine_name="[email protected]" layer_name="TSS-Standard Security" layer_name="TSS-Standard Application" layer_uuid="dae7f01c-4c98-4c3a-a643-bfbb8fcf40f0" layer_uuid="dbee3718-cf2f-4de0-8681-529cb75be9a6" match_id="8" match_id="33554431" parent_rule="0" parent_rule="0" rule_action="Accept" rule_action="Accept" rule_name="Implicit Cleanup" rule_uid="6dc2396f-9644-4546-8f32-95d98a3344e6" product="VPN-1 & FireWall-1" proto="17" s_port="37317" service="53" service_id="domain-udp" src="10.10.1.180" ","type":"qqqqq","host":"10.10.10.250","@version":"1","port":50620}{"@timestamp":"2019-12-19T14:50:12.153Z","message":"time="1576766483" action="Accept" conn_direction="Internal" contextnum="1" ifdir="outbound" ifname="bond1.101" logid="0" loguid="{0x5dfb8c13,
Ke nānā nei mākou i kēia mau memo, hoʻomaopopo mākou ua like ke ʻano o nā lāʻau: kahua = waiwai a kī = waiwai, ʻo ia hoʻi, kūpono ke kānana i kapa ʻia ʻo kv. No ke koho ʻana i ka kānana kūpono no kēlā me kēia hihia kikoʻī, he mea maikaʻi e hoʻomaʻamaʻa iā ʻoe iho me lākou i ka palapala ʻenehana, a i ʻole e nīnau i kahi hoaaloha.
Hoʻonohonoho i kāna kānana
I ka pae hope a mākou i koho ai i kv, ua hōʻike ʻia ka hoʻonohonoho o kēia kānana ma lalo nei.
filter {
if [type] == "checkpoint"{
kv {
value_split => "="
allow_duplicate_values => false
}
}
}
Koho mākou i ka hōʻailona e puʻunaue ai mākou i ke kahua a me ka waiwai - "=". Inā loaʻa iā mākou nā helu like i loko o ka log, mālama mākou i hoʻokahi wale nō manawa i ka waihona, i ʻole e hoʻopau ʻoe i kahi ʻano o nā waiwai like, ʻo ia hoʻi, inā loaʻa iā mākou ka memo "foo = kekahi foo = kekahi" kākau mākou i ka foo wale nō. = kekahi.
Hoʻonohonoho i ka Output kūpono ma ElasticSearch
Ke hoʻonohonoho ʻia ke kānana, hiki iā ʻoe ke hoʻouka i nā lāʻau i ka waihona ʻasticlaau:
output
{
if [type] == "checkpoint"
{
elasticsearch
{
hosts => ["10.10.1.200:9200"]
index => "checkpoint-%{+YYYY.MM.dd}"
user => "tssolution"
password => "cool"
}
}
}
Inā hoʻopaʻa inoa ʻia ka palapala me ke ʻano kikoʻī, mālama mākou i ka hanana i ka waihona elasticsearch, ka mea e ʻae i nā pilina ma 10.10.1.200 ma ke awa 9200 ma ke ʻano maʻamau. Mālama ʻia kēlā me kēia palapala i kahi kikoʻī kikoʻī, i kēia hihia mākou e mālama i ka index "checkpoint-" + ka lā manawa o kēia manawa. Hiki ke loaʻa i kēlā me kēia papa kuhikuhi kahi hoʻonohonoho kikoʻī o nā māla, a i ʻole hana ʻia i ka wā e ʻike ʻia ai kahi kahua hou i kahi leka; hiki ke ʻike ʻia nā hoʻonohonoho kahua a me ko lākou ʻano ma nā palapala palapala.
Inā ua hoʻonohonoho ʻia ka hōʻoia (e nānā mākou ma hope), pono e kuhikuhi ʻia nā hōʻoia no ke kākau ʻana i kahi kikoʻī kikoʻī, ma kēia hiʻohiʻona ʻo ia ka "tssolution" me ka ʻōlelo huna "cool". Hiki iā ʻoe ke hoʻokaʻawale i nā kuleana o ka mea hoʻohana e kākau i nā moʻolelo wale nō i kahi kuhikuhi kikoʻī a ʻaʻohe mea hou aʻe.
E hoʻokuʻu iā Logstash.
faila hoʻonohonoho Logstash:
input
{
tcp
{
port => 5555
type => "checkpoint"
mode => "server"
host => “10.10.1.205”
}
}
filter {
if [type] == "checkpoint"{
kv {
value_split => "="
allow_duplicate_values => false
}
}
}
output
{
if [type] == "checkpoint"
{
elasticsearch
{
hosts => ["10.10.1.200:9200"]
index => "checkpoint-%{+YYYY.MM.dd}"
user => "tssolution"
password => "cool"
}
}
}
Nānā mākou i ka faila hoʻonohonoho no ka pololei:
/usr/share/logstash/bin//logstash -f checkpoint.conf
E hoʻomaka i ka hana Logstash:
sudo systemctl hoʻomaka logstash
Nānā mākou ua hoʻomaka ke kaʻina hana:
sudo systemctl status logstash
E nānā inā ua piʻi ke kumu:
netstat -nat |grep 5555
Ke nānā nei i nā lāʻau ma Kibana.
Ma hope o ka holo ʻana o nā mea a pau, e hele i Kibana - Discover, e hōʻoia i ka hoʻonohonoho pono ʻana o nā mea āpau, hiki ke kaomi ʻia ke kiʻi!
Aia nā lāʻau a pau a hiki iā mākou ke ʻike i nā kahua āpau a me ko lākou mau waiwai!
hopena
Ua nānā mākou pehea e kākau ai i kahi faila hoʻonohonoho Logstash, a ma muli o ka loaʻa ʻana o kahi parser o nā māla a me nā waiwai āpau. I kēia manawa hiki iā mākou ke hana me ka ʻimi a me ka hoʻolālā ʻana i nā kahua kikoʻī. Ma hope o ka papa e nānā mākou i ka ʻike maka ma Kibana a hana i kahi dashboard maʻalahi. Pono e ʻōlelo ʻia e hoʻonui mau ʻia ka faila hoʻonohonoho Logstash i kekahi mau kūlana, no ka laʻana, ke makemake mākou e hoʻololi i ka waiwai o kahi kahua mai kahi helu i kahi huaʻōlelo. Ma nā ʻatikala aʻe e hana mau mākou i kēia.
No laila e hoʻomau (, , , ), .
Source: www.habr.com