A karshe mun hadu Farashin ELK, wadanne kayan software ya kunsa. Kuma aikin farko da injiniya ke fuskanta lokacin aiki tare da tarin ELK yana aika rajistan ayyukan ajiya a cikin bincike na elastick don bincike na gaba. Koyaya, wannan sabis ɗin leɓe ne kawai, elasticsearch yana adana rajistan ayyukan a cikin nau'ikan takardu tare da wasu fage da ƙima, wanda ke nufin injiniyan dole ne ya yi amfani da kayan aiki daban-daban don rarraba saƙon da aka aiko daga tsarin ƙarshe. Ana iya yin wannan ta hanyoyi da yawa - rubuta shirin da kanku wanda zai ƙara takardu zuwa ma'ajin bayanai ta amfani da API, ko amfani da shirye-shiryen mafita. A cikin wannan kwas za mu yi la'akari da mafita Logstash, wanda ke cikin tarin ELK. Za mu duba yadda za mu iya aika rajistan ayyukan daga tsarin ƙarshen ƙarshen zuwa Logstash, sannan za mu saita fayil ɗin sanyi don karkata da turawa zuwa bayanan Elasticsearch. Don yin wannan, muna ɗaukar rajistan ayyukan daga Tacewar zaɓi na Check Point azaman tsarin mai shigowa.
Wannan hanya ba ta rufe shigar da tari na ELK, tun da akwai adadi mai yawa na labarai akan wannan batu; za mu yi la'akari da sashin daidaitawa.
Bari mu tsara tsarin aiki don daidaitawar Logstash:
- Duba wannan binciken na roba zai karɓi rajistan ayyukan (duba ayyuka da buɗewar tashar jiragen ruwa).
- Muna la'akari da yadda za mu iya aika abubuwan da suka faru zuwa Logstash, zaɓi hanya, da aiwatar da shi.
- Muna saita Input a cikin fayil ɗin daidaitawar Logstash.
- Muna saita fitarwa a cikin fayil ɗin daidaitawar Logstash a cikin yanayin lalata don fahimtar yadda saƙon log ɗin yayi kama.
- Saita Tace.
- Saita ingantaccen fitarwa a cikin ElasticSearch.
- Logstash yana buɗewa.
- Duba rajistan ayyukan a Kibana.
Bari mu kalli kowane batu dalla-dalla:
Duba wannan binciken na roba zai karɓi rajistan ayyukan
Don yin wannan, zaku iya amfani da umarnin curl don bincika damar zuwa Elasticsearch daga tsarin da aka tura Logstash. Idan an daidaita ku, to muna kuma canja wurin mai amfani / kalmar wucewa ta hanyar curl, ƙayyade tashar jiragen ruwa 9200 idan ba ku canza shi ba. Idan kun sami amsa irin wannan na ƙasa, to komai yana cikin tsari.
[elastic@elasticsearch ~]$ curl -u <<user_name>> : <<password>> -sS -XGET "<<ip_address_elasticsearch>>:9200"
{
"name" : "elastic-1",
"cluster_name" : "project",
"cluster_uuid" : "sQzjTTuCR8q4ZO6DrEis0A",
"version" : {
"number" : "7.4.1",
"build_flavor" : "default",
"build_type" : "rpm",
"build_hash" : "fc0eeb6e2c25915d63d871d344e3d0b45ea0ea1e",
"build_date" : "2019-10-22T17:16:35.176724Z",
"build_snapshot" : false,
"lucene_version" : "8.2.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
[elastic@elasticsearch ~]$
Idan ba a karɓi amsa ba, to ana iya samun nau'ikan kurakurai da yawa: tsarin bincike na elastick ba ya gudana, an ƙayyade tashar tashar da ba daidai ba, ko kuma an toshe tashar ta hanyar wuta akan sabar inda aka shigar da elasticsearch.
Bari mu kalli yadda zaku iya aika rajistan ayyukan zuwa Logstash daga tacewar zaɓi
Daga uwar garken sarrafawa na Check Point zaku iya aika rajistan ayyukan zuwa Logstash ta hanyar syslog ta amfani da log_exporter utility, zaku iya karanta ƙarin game da shi anan. , a nan za mu bar kawai umarnin da ke haifar da rafi:
cp_log_export ƙara suna check_point_syslog target-server < > manufa-tashar jiragen ruwa 5555 yarjejeniya tcp tsarin juzu'i na yanayin karanta-haɗin kai
< > - adireshin uwar garken da Logstash ke aiki a kai, tashar tashar tashar 5555 - tashar da za mu aika da rajistan ayyukan, aika rajistan ayyukan ta hanyar tcp na iya ɗaukar uwar garken, don haka a wasu lokuta yana da kyau a yi amfani da udp.
Saita INPUT a cikin fayil ɗin daidaitawar Logstash
Ta hanyar tsoho, fayil ɗin daidaitawa yana cikin /etc/logstash/conf.d/ directory. Fayil ɗin daidaitawa ya ƙunshi sassa 3 masu ma'ana: INPUT, FILTER, OUTPUT. IN Input mun nuna inda tsarin zai dauki logs daga, a ciki tace parse log - saita yadda ake raba saƙon zuwa filaye da ƙima, a cikin fitarwa muna saita rafi na fitarwa - inda za a aika da rajistan ayyukan.
Da farko, bari mu saita INPUT, la'akari da wasu nau'ikan da zasu iya zama - fayil, tcp da exe.
Tcp:
input {
tcp {
port => 5555
host => “10.10.1.205”
type => "checkpoint"
mode => "server"
}
}
yanayin => "Server"
Yana nuna cewa Logstash yana karɓar haɗi.
tashar jiragen ruwa => 5555
Mai watsa shiri => "10.10.1.205"
Muna karɓar haɗi ta hanyar adireshin IP 10.10.1.205 (Logstash), tashar jiragen ruwa 5555 - dole ne a ba da izinin tashar tashar ta hanyar tsarin Tacewar zaɓi.
type => "Maganar bincike"
Muna yiwa takardar alama, dacewa sosai idan kuna da haɗin haɗin gwiwa da yawa. Daga baya, ga kowane haɗin gwiwa za ka iya rubuta naka tace ta amfani da ma'ana idan an gina shi.
Fayil:
input {
file {
path => "/var/log/openvas_report/*"
type => "openvas"
start_position => "beginning"
}
}
Bayanin saitunan:
hanya => "/var/log/openvas_report/*"
Muna nuna jagorar da ake buƙatar karanta fayilolin.
type => "budewa"
Nau'in taron.
start_position => "farawa"
Lokacin canza fayil, yana karanta fayil ɗin gabaɗaya; idan kun saita “ƙarshen”, tsarin yana jiran sabbin bayanan su bayyana a ƙarshen fayil ɗin.
Exec:
input {
exec {
command => "ls -alh"
interval => 30
}
}
Yin amfani da wannan shigarwar, an ƙaddamar da umarnin harsashi (kawai!) kuma ana mayar da abin da ke fitowa zuwa saƙon log.
umarni => "ls-alh"
Umurnin wanda muke sha'awar fitowar sa.
tazarar => 30
Tazarar kiran umarni a cikin daƙiƙa.
Domin karɓar rajistan ayyukan daga Tacewar zaɓi, muna yin rijistar tacewa tcp ko udp, dangane da yadda ake aika rajistan ayyukan zuwa Logstash.
Muna saita fitarwa a cikin fayil ɗin daidaitawar Logstash a cikin yanayin lalata don fahimtar yadda saƙon log ɗin yayi kama.
Bayan mun saita INPUT, muna buƙatar fahimtar yadda saƙon log ɗin zai kasance da kuma wadanne hanyoyin da ake buƙatar amfani da su don saita tace log (parser).
Don yin wannan, za mu yi amfani da tacewa wanda ke fitar da sakamakon zuwa stdout don duba saƙon asali; cikakken fayil ɗin daidaitawa a yanzu zai yi kama da haka:
input
{
tcp
{
port => 5555
type => "checkpoint"
mode => "server"
host => “10.10.1.205”
}
}
output
{
if [type] == "checkpoint"
{
stdout { codec=> json }
}
}
Gudanar da umarnin don duba:
sudo /usr/share/logstash/bin//logstash -f /etc/logstash/conf.d/checkpoint.conf
Muna ganin sakamakon, hoton yana dannawa:
Idan ka kwafa shi zai kasance kamar haka:
action="Accept" conn_direction="Internal" contextnum="1" ifdir="outbound" ifname="bond1.101" logid="0" loguid="{0x5dfb8c13,0x5,0xfe0a0a0a,0xc0000000}" origin="10.10.10.254" originsicname="CN=ts-spb-cpgw-01,O=cp-spb-mgmt-01.tssolution.local.kncafb" sequencenum="8" time="1576766483" version="5" context_num="1" dst="10.10.10.10" dst_machine_name="[email protected]" layer_name="TSS-Standard Security" layer_name="TSS-Standard Application" layer_uuid="dae7f01c-4c98-4c3a-a643-bfbb8fcf40f0" layer_uuid="dbee3718-cf2f-4de0-8681-529cb75be9a6" match_id="8" match_id="33554431" parent_rule="0" parent_rule="0" rule_action="Accept" rule_action="Accept" rule_name="Implicit Cleanup" rule_uid="6dc2396f-9644-4546-8f32-95d98a3344e6" product="VPN-1 & FireWall-1" proto="17" s_port="37317" service="53" service_id="domain-udp" src="10.10.1.180" ","type":"qqqqq","host":"10.10.10.250","@version":"1","port":50620}{"@timestamp":"2019-12-19T14:50:12.153Z","message":"time="1576766483" action="Accept" conn_direction="Internal" contextnum="1" ifdir="outbound" ifname="bond1.101" logid="0" loguid="{0x5dfb8c13,
Idan muka kalli waɗannan saƙonnin, za mu fahimci cewa rajistan ayyukan suna kama da: filin = darajar ko maɓalli = ƙima, wanda ke nufin tacewa mai suna kv ya dace. Domin zaɓar madaidaicin tacewa ga kowane takamaiman lamari, zai zama kyakkyawan ra'ayi don sanin kanku da su a cikin takaddun fasaha, ko tambayi aboki.
Saita Tace
A mataki na ƙarshe da muka zaɓi kv, an gabatar da tsarin wannan tace a ƙasa:
filter {
if [type] == "checkpoint"{
kv {
value_split => "="
allow_duplicate_values => false
}
}
}
Mun zaɓi alamar da za mu raba filin da darajar - "=". Idan muna da shigarwa iri ɗaya a cikin log ɗin, za mu adana misali ɗaya kawai a cikin ma'ajin bayanai, in ba haka ba za ku ƙare da ƙima iri ɗaya, wato, idan muna da saƙon "foo = wasu foo= wasu" muna rubuta foo kawai. = wasu.
Saita ingantaccen fitarwa a cikin ElasticSearch
Da zarar an saita Tace, zaku iya loda rajistan ayyukan zuwa ma'adanar bayanai maganin roba:
output
{
if [type] == "checkpoint"
{
elasticsearch
{
hosts => ["10.10.1.200:9200"]
index => "checkpoint-%{+YYYY.MM.dd}"
user => "tssolution"
password => "cool"
}
}
}
Idan takardar ta sanya hannu tare da nau'in wurin bincike, muna adana taron zuwa bayanan bincike na elasticsearch, wanda ke karɓar haɗin kai akan 10.10.1.200 akan tashar jiragen ruwa 9200 ta tsohuwa. Ana ajiye kowace takarda zuwa takamaiman maƙasudi, a wannan yanayin muna adanawa zuwa ma'aunin "Checkpoint-" + kwanan lokaci na yanzu. Kowace fihirisa na iya samun takamaiman saitin filayen, ko kuma ana ƙirƙira ta atomatik lokacin da sabon filin ya bayyana a cikin saƙo; ana iya duba saitunan filin da nau'in su a cikin taswira.
Idan an saita tantancewa (zamu duba shi daga baya), dole ne a ƙayyade takaddun shaidar rubutawa zuwa takamaiman maƙasudi, a cikin wannan misalin shine “tssolution” tare da kalmar sirri “cool”. Kuna iya bambanta haƙƙoƙin mai amfani don rubuta rajistan ayyukan kawai zuwa takamaiman fihirisa kuma babu ƙari.
Kaddamar da Logstash.
Fayil ɗin daidaitawar Logstash:
input
{
tcp
{
port => 5555
type => "checkpoint"
mode => "server"
host => “10.10.1.205”
}
}
filter {
if [type] == "checkpoint"{
kv {
value_split => "="
allow_duplicate_values => false
}
}
}
output
{
if [type] == "checkpoint"
{
elasticsearch
{
hosts => ["10.10.1.200:9200"]
index => "checkpoint-%{+YYYY.MM.dd}"
user => "tssolution"
password => "cool"
}
}
}
Muna duba fayil ɗin daidaitawa don daidaito:
/usr/share/logstash/bin//logstash -f checkpoint.conf
Fara tsarin Logstash:
sudo systemctl fara logstash
Mun duba cewa tsari ya fara:
sudo systemctl matsayi logstash
Mu duba idan soket ɗin ya tashi:
netstat -nat | grep 5555
Duba rajistan ayyukan a Kibana.
Bayan komai yana gudana, je zuwa Kibana - Discover, tabbatar cewa an daidaita komai daidai, ana iya danna hoton!
Duk gundumomi suna cikin wurin kuma muna iya ganin duk filayen da ƙimar su!
ƙarshe
Mun kalli yadda ake rubuta fayil ɗin daidaitawar Logstash, kuma a sakamakon haka mun sami fassarori na duk filayen da ƙimar. Yanzu za mu iya aiki tare da bincike da ƙirƙira don takamaiman filayen. Na gaba a cikin kwas ɗin za mu kalli gani a cikin Kibana kuma mu ƙirƙiri dashboard mai sauƙi. Ya kamata a ambata cewa fayil ɗin sanyi na Logstash yana buƙatar sabuntawa akai-akai a wasu yanayi, alal misali, lokacin da muke son maye gurbin darajar filin daga lamba zuwa kalma. A cikin kasidu masu zuwa za mu yi haka kullum.
Don haka ku kasance da mu, , , ), .
source: www.habr.com