I ka pili ʻana i ka pau ʻana o ke kūʻai aku ma Rūsia o ka Splunk logging and analytics system, ua kū mai ka nīnau: he aha ka mea hiki ke pani i kēia hoʻonā? Ma hope o ka hoʻolilo ʻana i ka manawa e hoʻomaʻamaʻa iaʻu iho me nā hoʻonā ʻokoʻa, ua hoʻoholo wau i kahi hopena no ke kanaka maoli - "ELK ahu". Pono kēia ʻōnaehana i ka manawa e hoʻonohonoho ai, akā ʻo ka hopena hiki iā ʻoe ke loaʻa i kahi ʻōnaehana ikaika loa no ka nānā ʻana i ke kūlana a pane koke i nā hanana palekana ʻike i ka hui. Ma kēia pūʻulu ʻatikala, e nānā mākou i nā mana kumu (a ʻaʻole paha) o ka waihona ELK, e noʻonoʻo pehea e hiki ai iā ʻoe ke parse i nā lāʻau, pehea e kūkulu ai i nā kiʻi a me nā dashboards, a he aha nā hana hoihoi e hiki ke hana me ka hoʻohana ʻana i ka laʻana o nā lāʻau. ka pā ahi ʻo Check Point a i ʻole ka mea nānā palekana OpenVas. No ka hoʻomaka ʻana, e nānā kākou i ke ʻano o ia - ka waihona ELK, a me nā ʻāpana o ia mea.
"ELK ahu" he acronym no ʻekolu papahana open source: Elasticsearch, ʻO Logstash и kibana. Hoʻomohala ʻia e Elastic me nā papahana pili āpau. ʻO Elasticsearch ke kumu o ka ʻōnaehana holoʻokoʻa, kahi e hui pū ai i nā hana o kahi waihona, ʻimi a me ka ʻōnaehana analytical. ʻO Logstash kahi pipeline hoʻoili ʻikepili ʻaoʻao server e loaʻa ana ka ʻikepili mai nā kumu he nui i ka manawa like, e hoʻopau i ka log, a laila hoʻouna iā ia i kahi waihona Elasticsearch. Hāʻawi ʻo Kibana i nā mea hoʻohana e nānā i ka ʻikepili me ka hoʻohana ʻana i nā pakuhi a me nā kiʻi ma Elasticsearch. Hiki iā ʻoe ke lawelawe i ka waihona ma o Kibana. Ma hope aʻe, e noʻonoʻo mākou i kēlā me kēia ʻōnaehana i nā kikoʻī hou aku.
ʻO Logstash
He mea pono ʻo Logstash no ka hoʻoili ʻana i nā hanana log mai nā kumu like ʻole, kahi e hiki ai iā ʻoe ke koho i nā māla a me kā lākou mau waiwai i kahi leka, a hiki iā ʻoe ke hoʻonohonoho i ka kānana ʻikepili a me ka hoʻoponopono. Ma hope o nā manipulations āpau, hoʻihoʻi hou ʻo Logstash i nā hanana i ka hale kūʻai ʻikepili hope loa. Hoʻonohonoho ʻia ka pono ma o nā faila hoʻonohonoho.
ʻO kahi hoʻonohonoho logstash maʻamau he faila (nā) i loaʻa i kekahi mau kahawai e hiki mai ana o ka ʻike (hoʻokomo), nā kānana he nui no kēia ʻike (filter) a me nā kahawai puka (output). Me he mea lā hoʻokahi a ʻoi aʻe paha nā faila hoʻonohonoho, aia ma ka mana maʻalahi (ʻaʻohe mea ʻole) e like me kēia:
input {
}
filter {
}
output {
}
Ma INPUT, hoʻonohonoho mākou i ka awa e hoʻouna ʻia ai nā lāʻau i ka protocol, a i ʻole kahi waihona e heluhelu ai i nā faila hou a i ʻole e hoʻonui mau ʻia. Ma FILTER mākou e hoʻonohonoho i ka log parser: parsing field, hoʻoponopono waiwai, hoʻohui i nā ʻāpana hou a holoi paha iā lākou. ʻO FILTER kahi kahua no ka mālama ʻana i ka memo e hiki mai ana iā Logstash me nā koho hoʻoponopono he nui. Ma ka hoʻopuka hoʻonohonoho mākou i kahi a mākou e hoʻouna ai i ka log parsed i hala, inā he elasticsearch kahi noi JSON i hoʻouna ʻia i nā māla me nā waiwai i hoʻouna ʻia, a i ʻole he ʻāpana o ka debug hiki ke hoʻopuka ʻia i stdout a i kākau ʻia i kahi faila.
Elasticsearch
I ka hoʻomaka ʻana, ʻo Elasticsearch kahi hopena no ka ʻimi kikokikona piha, akā me nā pono ʻē aʻe e like me ka maʻalahi scaling, replication a me nā mea ʻē aʻe, ka mea i maʻalahi ka huahana a me kahi hopena maikaʻi no nā papahana kiʻekiʻe me ka nui o ka ʻikepili. ʻO Elasticsearch kahi hale kūʻai palapala JSON pili ʻole (NoSQL) a me ka ʻenekini huli e pili ana i ka ʻimi kikokikona piha ʻo Lucene. ʻO Java Virtual Machine ka papa hana lako, no laila pono ka ʻōnaehana i ka nui o nā mea hana a me nā kumuwaiwai RAM e hana.
ʻO kēlā me kēia memo e hiki mai ana, me Logstash a i ʻole ka hoʻohana ʻana i ka API noiʻi, ua kuhikuhi ʻia ma ke ʻano he "palapala" - e like me kahi papa ma SQL relational. Hoʻopaʻa ʻia nā palapala āpau i kahi index - kahi analogue o kahi waihona ma SQL.
Ka laʻana o kahi palapala ma ka waihona:
{
"_index": "checkpoint-2019.10.10",
"_type": "_doc",
"_id": "yvNZcWwBygXz5W1aycBy",
"_version": 1,
"_score": null,
"_source": {
"layer_uuid": [
"dae7f01c-4c98-4c3a-a643-bfbb8fcf40f0",
"dbee3718-cf2f-4de0-8681-529cb75be9a6"
],
"outzone": "External",
"layer_name": [
"TSS-Standard Security",
"TSS-Standard Application"
],
"time": "1565269565",
"dst": "103.5.198.210",
"parent_rule": "0",
"host": "10.10.10.250",
"ifname": "eth6",
]
}
Hoʻokumu ʻia nā hana āpau me ka waihona ma nā noi JSON me ka hoʻohana ʻana i ka REST API, kahi e hana ai i nā palapala ma ka index a i ʻole kekahi mau helu helu ma ke ʻano: nīnau - pane. I mea e ʻike ai i nā pane āpau i nā noi, ua kākau ʻia ʻo Kibana, ʻo ia ka lawelawe pūnaewele.
kibana
ʻAe ʻo Kibana iā ʻoe e ʻimi, hoʻihoʻi i ka ʻikepili a me ka huli ʻana i nā ʻikepili mai ka elasticsearch database, akā nui nā kiʻi nani a me nā dashboard i kūkulu ʻia ma muli o nā pane. Loaʻa i ka ʻōnaehana ka hana hoʻokele database elasticsearch; ma nā ʻatikala aʻe e nānā mākou i kēia lawelawe i nā kikoʻī hou aku. I kēia manawa e hōʻike mākou i kahi laʻana o nā dashboards no ka pā ahi Check Point a me ka OpenVas vulnerability scanner hiki ke kūkulu ʻia.
He laʻana o ka dashboard no Check Point, hiki ke kaomi i ke kiʻi:
He laʻana o ka dashboard no OpenVas, hiki ke kaomi i ke kiʻi:
hopena
Ua nānā mākou i ke ʻano o ia mea ELK ahu, ua kamaʻāina mākou i nā huahana nui, ma hope o ka papa e noʻonoʻo kaʻawale mākou i ke kākau ʻana i kahi faila hoʻonohonoho Logstash, hoʻonohonoho i nā dashboards ma Kibana, e kamaʻāina me nā noi API, automation a me nā mea hou aku!
No laila e hoʻomau (, , , ), .
Source: www.habr.com