Nyob rau hauv kev twb kev txuas nrog rau qhov kawg ntawm kev muag khoom nyob rau hauv Russia ntawm lub Splunk logging thiab analytics system, cov lus nug tshwm sim: dab tsi yuav hloov qhov kev daws teeb meem no? Tom qab siv sijhawm los paub txog kuv tus kheej nrog cov kev daws teeb meem sib txawv, kuv tau daws qhov kev daws teeb meem rau tus txiv neej tiag tiag - "ELK pawg". Lub kaw lus no yuav siv sijhawm los teeb tsa, tab sis vim li ntawd koj tuaj yeem tau txais lub zog muaj zog heev rau kev tshuaj xyuas cov xwm txheej thiab teb sai sai rau cov ntaub ntawv kev nyab xeeb hauv lub koom haum. Nyob rau hauv cov kab lus no, peb yuav saib qhov yooj yim (lossis tej zaum tsis yog) lub peev xwm ntawm ELK pawg, xav txog seb koj tuaj yeem txheeb xyuas cov cav, yuav ua li cas tsim cov duab thiab cov dashboards, thiab cov haujlwm nthuav dav twg tuaj yeem ua tiav siv cov piv txwv ntawm cov cav los ntawm lub Check Point firewall lossis OpenVas kev ruaj ntseg scanner. Yuav pib nrog, cia saib seb nws yog dab tsi - ELK pawg, thiab cov khoom nws muaj dab tsi.
"ELK pawg" yog ib qho acronym rau peb qhov qhib qhov project: Elasticsearch, Logstash ΠΈ Kibana. Tsim los ntawm Elastic nrog rau tag nrho cov haujlwm cuam tshuam. Elasticsearch yog lub hauv paus tseem ceeb ntawm tag nrho cov kab ke, uas sib txuas cov haujlwm ntawm cov ntaub ntawv, kev tshawb nrhiav thiab kev tshuaj xyuas. Logstash yog lub server-sab cov ntaub ntawv ua cov kav dej uas tau txais cov ntaub ntawv los ntawm ntau qhov chaw ib txhij, txheeb xyuas lub cav, thiab tom qab ntawd xa mus rau Elasticsearch database. Kibana tso cai rau cov neeg siv pom cov ntaub ntawv siv cov kab kos thiab cov duab hauv Elasticsearch. Koj tuaj yeem tswj hwm cov ntaub ntawv los ntawm Kibana. Tom ntej no, peb yuav txiav txim siab txhua qhov kev sib cais hauv kev nthuav dav ntxiv.
Logstash
Logstash yog ib qho khoom siv rau kev ua cov txheej xwm los ntawm ntau qhov chaw, uas koj tuaj yeem xaiv cov teb thiab lawv cov txiaj ntsig hauv cov lus, thiab koj tuaj yeem teeb tsa cov ntaub ntawv lim thiab kho. Tom qab tag nrho cov manipulations, Logstash redirects txheej xwm mus rau lub kawg cov ntaub ntawv khw. Cov nqi hluav taws xob tau teeb tsa tsuas yog los ntawm cov ntaub ntawv teeb tsa.
Ib lub logstash configuration yog ib cov ntaub ntawv uas muaj ob peb kwj ntawm cov ntaub ntawv (input), ob peb lim rau cov ntaub ntawv no (lim) thiab ob peb kwj tawm (tso zis). Nws zoo li ib lossis ntau cov ntaub ntawv teeb tsa, uas nyob rau hauv qhov yooj yim version (uas tsis muaj dab tsi hlo li) zoo li qhov no:
input {
}
filter {
}
output {
}
Hauv INPUT peb teeb tsa qhov chaw nres nkoj lub cav yuav raug xa mus rau thiab dhau los ntawm cov txheej txheem twg, lossis los ntawm cov ntawv twg los nyeem cov ntaub ntawv tshiab lossis hloov kho tas li. Hauv FILTER peb teeb tsa lub cav parser: parsing teb, kho qhov tseem ceeb, ntxiv cov tsis muaj tshiab lossis tshem tawm. FILTER yog daim teb rau kev tswj cov lus uas tuaj rau Logstash nrog ntau yam kev xaiv kho. Hauv cov zis peb teeb tsa qhov chaw uas peb xa cov ntaub ntawv parsed lawm, nyob rau hauv rooj plaub nws yog elasticsearch ib daim ntawv thov JSON raug xa mus rau qhov chaw uas muaj nuj nqis raug xa mus, lossis ua ib feem ntawm qhov kev debug nws tuaj yeem tso tawm rau stdout lossis sau rau hauv cov ntaub ntawv.
Elasticsearch
Thaum xub thawj, Elasticsearch yog ib qho kev daws teeb meem rau kev tshawb nrhiav tag nrho, tab sis nrog cov khoom siv ntxiv xws li kev ntsuas tau yooj yim, rov ua dua thiab lwm yam, uas ua rau cov khoom lag luam yooj yim heev thiab cov kev daws teeb meem zoo rau cov haujlwm siab nrog cov ntaub ntawv loj. Elasticsearch yog qhov tsis sib xws (NoSQL) JSON cov ntaub ntawv khaws cia thiab kev tshawb fawb cav raws li Lucene cov ntawv nyeem puv. Lub platform kho vajtse yog Java Virtual Machine, yog li lub kaw lus xav tau ntau tus processor thiab RAM cov peev txheej los ua haujlwm.
Txhua cov lus xa tuaj, nrog rau Logstash lossis siv cov lus nug API, raug ntsuas raws li "cov ntaub ntawv" - piv rau lub rooj hauv kev sib raug zoo SQL. Tag nrho cov ntaub ntawv khaws cia hauv qhov ntsuas - qhov sib piv ntawm cov ntaub ntawv hauv SQL.
Piv txwv ntawm cov ntaub ntawv hauv database:
{
"_index": "checkpoint-2019.10.10",
"_type": "_doc",
"_id": "yvNZcWwBygXz5W1aycBy",
"_version": 1,
"_score": null,
"_source": {
"layer_uuid": [
"dae7f01c-4c98-4c3a-a643-bfbb8fcf40f0",
"dbee3718-cf2f-4de0-8681-529cb75be9a6"
],
"outzone": "External",
"layer_name": [
"TSS-Standard Security",
"TSS-Standard Application"
],
"time": "1565269565",
"dst": "103.5.198.210",
"parent_rule": "0",
"host": "10.10.10.250",
"ifname": "eth6",
]
}
Txhua qhov kev ua haujlwm nrog cov ntaub ntawv yog raws li JSON thov siv REST API, uas yog tsim cov ntaub ntawv los ntawm kev ntsuas lossis qee qhov kev txheeb cais hauv hom: lo lus nug - teb. Txhawm rau pom tag nrho cov lus teb rau kev thov, Kibana tau sau, uas yog lub vev xaib kev pabcuam.
Kibana
Kibana tso cai rau koj los tshawb nrhiav, khaws cov ntaub ntawv thiab nug cov txheeb cais los ntawm cov ntaub ntawv elasticsearch, tab sis ntau cov duab zoo nkauj thiab cov dashboards yog tsim los ntawm cov lus teb. Lub kaw lus kuj muaj elasticsearch database tswj kev ua haujlwm; nyob rau hauv cov ntawv tom ntej peb yuav saib cov kev pabcuam no kom ntxaws ntxiv. Tam sim no cia peb qhia ib qho piv txwv ntawm dashboards rau Check Point firewall thiab OpenVas qhov tsis zoo scanner uas tuaj yeem tsim.
Ib qho piv txwv ntawm dashboard rau Check Point, daim duab yog clickable:
Ib qho piv txwv ntawm dashboard rau OpenVas, daim duab yog clickable:
xaus
Peb saib seb nws muaj dab tsi ELK pawg, peb tau paub me ntsis nrog cov khoom tseem ceeb, tom qab hauv chav kawm peb yuav cais txiav txim siab sau cov ntaub ntawv Logstash teeb tsa, teeb tsa dashboards ntawm Kibana, tau paub nrog API thov, automation thiab ntau ntxiv!
Yog li nyob twj ywm, , , ), .
Tau qhov twg los: www.hab.com