🥇 2. Elastic pawg: tsom xam ntawm kev ruaj ntseg cav. Logstash

Thaum kawg Tshooj peb tau ntsib ELK pawg, dab tsi software khoom nws muaj. Thiab thawj txoj haujlwm uas tus kws ua haujlwm ntsib thaum ua haujlwm nrog ELK pawg yog xa cov cav rau kev khaws cia hauv elasticsearch rau kev tshuaj xyuas tom ntej. Txawm li cas los xij, qhov no tsuas yog kev pabcuam di ncauj xwb, elasticsearch khw muag khoom hauv daim ntawv ntawm cov ntaub ntawv nrog qee qhov chaw thiab qhov tseem ceeb, uas txhais tau hais tias tus engineer yuav tsum siv ntau yam cuab yeej los txheeb xyuas cov lus uas tau xa los ntawm cov tshuab kawg. Qhov no tuaj yeem ua tau ntau txoj hauv kev - sau ib qho program koj tus kheej uas yuav ntxiv cov ntaub ntawv rau hauv cov ntaub ntawv siv API, lossis siv cov kev daws teeb meem npaj txhij. Hauv chav kawm no peb yuav xav txog qhov kev daws teeb meem Logstash, uas yog ib feem ntawm ELK pawg. Peb yuav saib yuav ua li cas peb tuaj yeem xa cov ntawv teev lus los ntawm cov txheej txheem kawg mus rau Logstash, thiab tom qab ntawd peb yuav teeb tsa cov ntaub ntawv teeb tsa los txheeb xyuas thiab hloov mus rau Elasticsearch database. Txhawm rau ua qhov no, peb nqa cov cav los ntawm Check Point firewall raws li cov khoom nkag.

Cov chav kawm tsis suav nrog kev teeb tsa ntawm ELK pawg, vim tias muaj ntau cov khoom ntawm cov ncauj lus no; peb yuav xav txog cov kev teeb tsa.

Wb kos ib qho kev npaj ua rau Logstash configuration:

Tshawb xyuas tias elasticsearch yuav lees txais cov ntawv teev npe (xyuas cov haujlwm ua haujlwm thiab qhib qhov chaw nres nkoj).
Peb xav txog yuav ua li cas peb tuaj yeem xa cov xwm txheej rau Logstash, xaiv ib txoj hauv kev, thiab siv nws.
Peb configure Input hauv Logstash configuration file.
Peb configure Output nyob rau hauv lub Logstash configuration ntaub ntawv nyob rau hauv debug hom nyob rau hauv thiaj li yuav to taub dab tsi log lus zoo li.
Kev teeb tsa Filter.
Teeb tsa qhov tso zis kom raug hauv ElasticSearch.
Logstash launches.
Tshawb xyuas cov cav hauv Kibana.

Cia peb saib ntawm txhua lub ntsiab lus hauv kev nthuav dav ntxiv:

Tshawb xyuas tias elasticsearch yuav txais cov cav

Txhawm rau ua qhov no, koj tuaj yeem siv cov lus txib curl los tshawb xyuas Elasticsearch los ntawm lub kaw lus uas Logstash tau xa mus. Yog tias koj muaj authentication teeb tsa, ces peb kuj hloov tus neeg siv / lo lus zais ntawm curl, qhia qhov chaw nres nkoj 9200 yog tias koj tsis tau hloov nws. Yog tias koj tau txais cov lus teb zoo ib yam li cov hauv qab no, ces txhua yam nyob rau hauv kev txiav txim.

[elastic@elasticsearch ~]$ curl -u <<user_name>> : <<password>> -sS -XGET "<<ip_address_elasticsearch>>:9200"
{
  "name" : "elastic-1",
  "cluster_name" : "project",
  "cluster_uuid" : "sQzjTTuCR8q4ZO6DrEis0A",
  "version" : {
    "number" : "7.4.1",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "fc0eeb6e2c25915d63d871d344e3d0b45ea0ea1e",
    "build_date" : "2019-10-22T17:16:35.176724Z",
    "build_snapshot" : false,
    "lucene_version" : "8.2.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}
[elastic@elasticsearch ~]$

Yog tias tsis tau txais cov lus teb, ces tej zaum yuav muaj ntau hom kev ua yuam kev: cov txheej txheem elasticsearch tsis khiav, qhov chaw nres nkoj tsis raug, lossis qhov chaw nres nkoj raug thaiv los ntawm firewall ntawm lub server qhov twg elasticsearch raug teeb tsa.

Cia peb saib yuav ua li cas koj tuaj yeem xa cov cav mus rau Logstash los ntawm qhov chaw kuaj xyuas qhov hluav taws xob

Los ntawm Check Point tswj server koj tuaj yeem xa cov cav mus rau Logstash ntawm syslog siv lub log_exporter utility, koj tuaj yeem nyeem ntxiv txog nws ntawm no Tshooj, ntawm no peb yuav tawm tsuas yog cov lus txib uas tsim cov kwj:

cp_log_export ntxiv lub npe check_point_syslog target-server < > phiaj-chaw nres nkoj 5555 raws tu qauv tcp hom ntawv nyeem hom semi-unified

< > - chaw nyob ntawm tus neeg rau zaub mov uas Logstash khiav, lub hom phiaj-chaw nres nkoj 5555 - chaw nres nkoj uas peb yuav xa cov cav, xa cov cav ntawm tcp tuaj yeem thauj cov neeg rau zaub mov, yog li qee zaum nws yog qhov tseeb dua los siv udp.

Teeb tsa INPUT hauv Logstash configuration file

Los ntawm lub neej ntawd, cov ntaub ntawv teeb tsa nyob hauv /etc/logstash/conf.d/ directory. Cov ntaub ntawv teeb tsa muaj 3 qhov tseem ceeb: INPUT, FILTER, OUTPUT. IN Tswv yim peb qhia qhov twg qhov system yuav coj cov cav los ntawm, hauv LIM parse lub cav - teeb tsa yuav ua li cas faib cov lus rau hauv cov teb thiab qhov tseem ceeb, hauv ZIS peb configure cov zis kwj - qhov twg cov parsed cav yuav xa.

Ua ntej, cia peb teeb tsa INPUT, xav txog qee yam uas tuaj yeem ua tau - file, tcp thiab exe.

Tcp:

input {
tcp {
    port => 5555
    host => “10.10.1.205”
    type => "checkpoint"
    mode => "server"
}
}

hom => "server"
Qhia tias Logstash tab tom txais kev sib txuas.

chaw = 5555
host => "10.10.1.205"
Peb lees txais kev sib txuas ntawm IP chaw nyob 10.10.1.205 (Logstash), chaw nres nkoj 5555 - qhov chaw nres nkoj yuav tsum tau tso cai los ntawm txoj cai firewall.

hom => "checkpoint"
Peb kos cov ntaub ntawv, yooj yim heev yog tias koj muaj ob peb qhov kev sib txuas. Tom qab ntawd, rau txhua qhov kev sib txuas koj tuaj yeem sau koj tus kheej lim siv qhov laj thawj yog tsim.

Cov ntaub ntawv:

input {
  file {
    path => "/var/log/openvas_report/*"
    type => "openvas"
    start_position => "beginning"
    }
}

Cov lus piav qhia ntawm kev teeb tsa:
path => "/var/log/openvas_report/*"
Peb qhia cov npe uas cov ntaub ntawv yuav tsum tau nyeem.

yam => "openvas"
Hom xwm txheej.

start_position => "pib"
Thaum hloov cov ntaub ntawv, nws nyeem tag nrho cov ntaub ntawv; yog tias koj teem "kawg", lub kaw lus tos kom cov ntaub ntawv tshiab tshwm sim thaum kawg ntawm cov ntaub ntawv.

Exec:

input {
  exec {
    command => "ls -alh"
    interval => 30
  }
}

Siv cov tswv yim no, ib qho (tsuas yog!) plhaub hais kom ua thiab nws cov zis tau hloov mus rau hauv cov lus teev tseg.

command => "ls -alh"
Cov lus txib uas nws cov zis peb txaus siab rau.

ncua = 30
Hais kom invocation interval hauv vib nas this.

Txhawm rau kom tau txais cov cav los ntawm firewall, peb sau npe rau lub lim tcp los yog udp, nyob ntawm seb cov cav raug xa mus rau Logstash li cas.

Peb configure Output nyob rau hauv lub Logstash configuration ntaub ntawv nyob rau hauv debug hom nyob rau hauv thiaj li yuav to taub dab tsi log lus zoo li

Tom qab peb tau teeb tsa INPUT, peb yuav tsum nkag siab tias cov ntawv teev lus yuav zoo li cas thiab cov txheej txheem twg yuav tsum tau siv los teeb tsa lub cav lim (parser).

Ua li no, peb yuav siv lub lim uas tso tawm qhov tshwm sim rau stdout txhawm rau saib cov lus qub; cov ntaub ntawv teeb tsa tiav tam sim no yuav zoo li no:

input 
{
         tcp 
         {
                port => 5555
  	  	type => "checkpoint"
    		mode => "server"
                host => “10.10.1.205”
   	 }
}

output 
{
	if [type] == "checkpoint" 
       {
		stdout { codec=> json }
	}
}

Khiav cov lus txib kom kuaj xyuas:
sudo /usr/share/logstash/bin//logstash -f /etc/logstash/conf.d/checkpoint.conf
Peb pom qhov tshwm sim, daim duab yog clickable:

Yog tias koj luam nws yuav zoo li no:

action="Accept" conn_direction="Internal" contextnum="1" ifdir="outbound" ifname="bond1.101" logid="0" loguid="{0x5dfb8c13,0x5,0xfe0a0a0a,0xc0000000}" origin="10.10.10.254" originsicname="CN=ts-spb-cpgw-01,O=cp-spb-mgmt-01.tssolution.local.kncafb" sequencenum="8" time="1576766483" version="5" context_num="1" dst="10.10.10.10" dst_machine_name="[email protected]" layer_name="TSS-Standard Security" layer_name="TSS-Standard Application" layer_uuid="dae7f01c-4c98-4c3a-a643-bfbb8fcf40f0" layer_uuid="dbee3718-cf2f-4de0-8681-529cb75be9a6" match_id="8" match_id="33554431" parent_rule="0" parent_rule="0" rule_action="Accept" rule_action="Accept" rule_name="Implicit Cleanup" rule_uid="6dc2396f-9644-4546-8f32-95d98a3344e6" product="VPN-1 & FireWall-1" proto="17" s_port="37317" service="53" service_id="domain-udp" src="10.10.1.180" ","type":"qqqqq","host":"10.10.10.250","@version":"1","port":50620}{"@timestamp":"2019-12-19T14:50:12.153Z","message":"time="1576766483" action="Accept" conn_direction="Internal" contextnum="1" ifdir="outbound" ifname="bond1.101" logid="0" loguid="{0x5dfb8c13,

Saib ntawm cov lus no, peb nkag siab tias cov cav zoo li: teb = tus nqi lossis qhov tseem ceeb = tus nqi, uas txhais tau tias lub lim hu ua kv yog qhov tsim nyog. Txhawm rau xaiv cov lim zoo rau txhua qhov xwm txheej tshwj xeeb, nws yuav yog lub tswv yim zoo kom paub koj tus kheej nrog lawv hauv cov ntaub ntawv kev tshaj lij, lossis nug ib tus phooj ywg.

Kev teeb tsa Filter

Nyob rau theem kawg peb xaiv kv, kev teeb tsa ntawm cov lim dej no tau nthuav tawm hauv qab no:

filter {
if [type] == "checkpoint"{
	kv {
		value_split => "="
		allow_duplicate_values => false
	}
}
}

Peb xaiv lub cim uas peb yuav faib lub teb thiab tus nqi - "=". Yog tias peb muaj cov ntawv sau zoo ib yam hauv lub cav, peb tsuas yog khaws ib qho piv txwv hauv cov ntaub ntawv, txwv tsis pub koj yuav xaus nrog cov khoom muaj txiaj ntsig zoo ib yam, uas yog, yog tias peb muaj cov lus "foo = qee qhov foo = qee qhov" peb sau nkaus xwb foo = ib co.

Teeb tsa qhov tso zis kom raug hauv ElasticSearch

Thaum Filter tau teeb tsa, koj tuaj yeem upload cov cav rau hauv cov ntaub ntawv ywj siab nrhiav:

output 
{
if [type] == "checkpoint"
{
 	elasticsearch 
        {
		hosts => ["10.10.1.200:9200"]
		index => "checkpoint-%{+YYYY.MM.dd}"
    		user => "tssolution"
    		password => "cool"
  	}
}
}

Yog tias cov ntaub ntawv tau kos npe nrog hom chaw kuaj xyuas, peb khaws qhov xwm txheej rau hauv elasticsearch database, uas lees txais kev sib txuas ntawm 10.10.1.200 ntawm chaw nres nkoj 9200 los ntawm lub neej ntawd. Txhua daim ntawv tau txais kev cawmdim rau ib qho kev ntsuas tshwj xeeb, qhov no peb khaws cia rau qhov ntsuas "checkpoint-" + hnub tim tam sim no. Txhua qhov Performance index tuaj yeem muaj cov txheej txheem tshwj xeeb, lossis yog tsim tau thaum lub teb tshiab tshwm hauv cov lus; cov chaw teeb tsa thiab lawv hom tuaj yeem pom hauv daim ntawv qhia.

Yog hais tias koj muaj authentication configured (peb mam li saib tom qab), cov ntaub ntawv pov thawj rau kev sau ntawv mus rau ib qho kev ntsuas tshwj xeeb yuav tsum tau teev tseg, hauv qhov piv txwv no nws yog "tssolution" nrog lo lus zais "txias". Koj tuaj yeem sib txawv cov neeg siv txoj cai los sau cov cav tsuas yog rau ib qho kev ntsuas tshwj xeeb thiab tsis muaj ntxiv lawm.

Tua tawm Logstash.

Logstash configuration file:

input 
{
         tcp 
         {
                port => 5555
  	  	type => "checkpoint"
    		mode => "server"
                host => “10.10.1.205”
   	 }
}

filter {
        if [type] == "checkpoint"{
	kv {
		value_split => "="
		allow_duplicate_values => false
	}
        }
}

output 
{
if [type] == "checkpoint"
{
 	elasticsearch 
        {
		hosts => ["10.10.1.200:9200"]
		index => "checkpoint-%{+YYYY.MM.dd}"
    		user => "tssolution"
    		password => "cool"
  	}
}
}

Peb xyuas cov ntaub ntawv configuration kom raug:
/usr/share/logstash/bin//logstash -f checkpoint.conf

Pib qhov txheej txheem Logstash:
sudo systemctl pib logstash

Peb xyuas tias cov txheej txheem tau pib:
sudo systemctl xwm txheej logstash

Cia peb kuaj seb lub qhov (socket) puas:
netstat -nat | grep 5555

Tshawb xyuas cov cav hauv Kibana.

Tom qab txhua yam khiav, mus rau Kibana - Tshawb nrhiav, nco ntsoov tias txhua yam teeb tsa raug, daim duab yog clickable!

Tag nrho cov cav nyob rau hauv qhov chaw thiab peb tuaj yeem pom tag nrho cov teb thiab lawv cov nqi!

xaus

Peb tau saib yuav ua li cas sau cov ntaub ntawv Logstash configuration, thiab yog li ntawd peb tau txais ib qho parser ntawm tag nrho cov teb thiab qhov tseem ceeb. Tam sim no peb tuaj yeem ua haujlwm nrog kev tshawb nrhiav thiab npaj rau thaj chaw tshwj xeeb. Tom ntej no hauv chav kawm peb yuav saib pom kev pom hauv Kibana thiab tsim lub dashboard yooj yim. Nws yog ib qho tsim nyog hais tias Logstash teeb tsa cov ntaub ntawv yuav tsum tau hloov kho tas li hauv qee qhov xwm txheej, piv txwv li, thaum peb xav hloov tus nqi ntawm daim teb los ntawm tus lej mus rau ib lo lus. Hauv cov ntawv tom ntej peb yuav ua qhov no tas li.

Yog li nyob twj ywmTelegram, Facebook, VK, TS Solution Blog), Yandex.Zen.

Tau qhov twg los: www.hab.com

2. Elastic pawg: tsom xam ntawm kev ruaj ntseg cav. Logstash