I-ProHoster > Π‘Π»ΠΎΠ³ > Ukuphatha > I-Splunk Universal Forwarder kudokhi njengomqoqi welogi yesistimu

I-Splunk Universal Forwarder kudokhi njengomqoqi welogi yesistimu

I-Splunk Universal Forwarder kudokhi njengomqoqi welogi yesistimu

I-Splunk ingenye yeqoqo lemibhalo yezentengiselwano eqashelwa kakhulu kanye nemikhiqizo yokuhlaziya. Ngisho namanje, lapho ukuthengisa kungasakwenziwa eRussia, lesi akusona isizathu sokungabhali imiyalelo/ukwenza kanjani lo mkhiqizo.

Inhloso: qoqa amalogi wesistimu kuma-docker node ku-Splunk ngaphandle kokushintsha ukucushwa komshini wokusingatha

Ngingathanda ukuqala ngendlela esemthethweni, ebukeka iyinqaba uma usebenzisa i-Docker.
Xhuma kuhabhu le-Docker
Yini esinayo:

1. Isithombe se-Pullim

$ docker pull splunk/universalforwarder:latest

2. Qala isitsha ngemingcele edingekayo

$ docker run -d  -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=<password>' splunk/universalforwarder:latest

3. Singena esitsheni

docker exec -it <container-id> /bin/bash

Okulandelayo, siyacelwa ukuthi siye ekhelini elaziwayo embhalweni.

Futhi ulungiselele isiqukathi ngemva kokuthi siqalile:


./splunk add forward-server <host name or ip address>:<listening port>
./splunk add monitor /var/log
./splunk restart

Linda. Ini?

Kodwa izimanga azigcini lapho. Uma usebenzisa isiqukathi esivela esithombeni esisemthethweni ngemodi yokusebenzisana, uzobona okulandelayo:

Ukudumala kancane


$ docker run -it -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=password' splunk/universalforwarder:latest

PLAY [Run default Splunk provisioning] *******************************************************************************************************************************************************************************************************
Tuesday 09 April 2019  13:40:38 +0000 (0:00:00.096)       0:00:00.096 *********

TASK [Gathering Facts] ***********************************************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:39 +0000 (0:00:01.520)       0:00:01.616 *********

TASK [Get actual hostname] *******************************************************************************************************************************************************************************************************************
changed: [localhost]
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.599)       0:00:02.215 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.054)       0:00:02.270 *********

TASK [set_fact] ******************************************************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.075)       0:00:02.346 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.067)       0:00:02.413 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.060)       0:00:02.473 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.051)       0:00:02.525 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.056)       0:00:02.582 *********
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.216)       0:00:02.798 *********
included: /opt/ansible/roles/splunk_common/tasks/change_splunk_directory_owner.yml for localhost
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.087)       0:00:02.886 *********

TASK [splunk_common : Update Splunk directory owner] *****************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.324)       0:00:03.210 *********
included: /opt/ansible/roles/splunk_common/tasks/get_facts.yml for localhost
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.094)       0:00:03.305 *********

Π½Ρƒ ΠΈ Ρ‚Π°ΠΊ Π΄Π°Π»Π΅Π΅...

Kuhle. Isithombe asiqukethe ngisho ne-artifact. Okusho ukuthi, ngaso sonke isikhathi uma uqala kuzothatha isikhathi ukulanda ingobo yomlando ngamabhanari, uyikhiphe futhi uyilungiselele.
Kuthiwani nge-docker-way nakho konke lokho?

Cha ngiyabonga. Sizothatha umzila ohlukile. Kuthiwani uma senza yonke le misebenzi esigabeni somhlangano? Bese sihamba!

Ukuze ngingalibali isikhathi eside, ngizokukhombisa isithombe sokugcina ngaso leso sikhathi:

I-Dockerfile

# Π’ΡƒΡ‚ Ρƒ ΠΊΠΎΠ³ΠΎ ΠΊΠ°ΠΊΠΈΠ΅ прСдпочтСния
FROM centos:7

# Π—Π°Π΄Π°Ρ‘ΠΌ ΠΏΠ΅Ρ€Π΅ΠΌΠ΅Π½Π½Ρ‹Π΅, Ρ‡Ρ‚ΠΎΠ±Ρ‹ ΠΊΠ°ΠΆΠ΄Ρ‹ΠΉ Ρ€Π°Π· ΠΏΡ€ΠΈ стартС Π½Π΅ ΡƒΠΊΠ°Π·Ρ‹Π²Π°Ρ‚ΡŒ ΠΈΡ…
ENV SPLUNK_HOME /splunkforwarder
ENV SPLUNK_ROLE splunk_heavy_forwarder
ENV SPLUNK_PASSWORD changeme
ENV SPLUNK_START_ARGS --accept-license

# Π‘Ρ‚Π°Π²ΠΈΠΌ ΠΏΠ°ΠΊΠ΅Ρ‚Ρ‹
# wget - Ρ‡Ρ‚ΠΎΠ±Ρ‹ ΡΠΊΠ°Ρ‡Π°Ρ‚ΡŒ Π°Ρ€Ρ‚Π΅Ρ„Π°ΠΊΡ‚Ρ‹
# expect - понадобится для ΠΏΠ΅Ρ€Π²ΠΎΠ½Π°Ρ‡Π°Π»ΡŒΠ½ΠΎΠ³ΠΎ запуска Splunk Π½Π° этапС сборки
# jq - ΠΈΡΠΏΠΎΠ»ΡŒΠ·ΡƒΠ΅Ρ‚ΡΡ Π² скриптах, ΠΊΠΎΡ‚ΠΎΡ€Ρ‹Π΅ ΡΠΎΠ±ΠΈΡ€Π°ΡŽΡ‚ статистику Π΄ΠΎΠΊΠ΅Ρ€Π°
RUN yum install -y epel-release 
    && yum install -y wget expect jq

# ΠšΠ°Ρ‡Π°Π΅ΠΌ, распаковываСм, удаляСм
RUN wget -O splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.2.4&product=universalforwarder&filename=splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz&wget=true' 
    && wget -O docker-18.09.3.tgz 'https://download.docker.com/linux/static/stable/x86_64/docker-18.09.3.tgz' 
    && tar -xvf splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 
    && tar -xvf docker-18.09.3.tgz  
    && rm -f splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 
    && rm -f docker-18.09.3.tgz

# Π‘ shell скриптами всё понятно, Π° Π²ΠΎΡ‚ inputs.conf, splunkclouduf.spl ΠΈ first_start.sh Π½ΡƒΠΆΠ΄Π°ΡŽΡ‚ΡΡ Π² пояснСнии. Об этом расскаТу послС source тэга.
COPY [ "inputs.conf", "docker-stats/props.conf", "/splunkforwarder/etc/system/local/" ]
COPY [ "docker-stats/docker_events.sh", "docker-stats/docker_inspect.sh", "docker-stats/docker_stats.sh", "docker-stats/docker_top.sh", "/splunkforwarder/bin/scripts/" ]
COPY splunkclouduf.spl /splunkclouduf.spl
COPY first_start.sh /splunkforwarder/bin/

#  Π”Π°Ρ‘ΠΌ ΠΏΡ€Π°Π²Π° Π½Π° исполнСниС, добавляСм ΠΏΠΎΠ»ΡŒΠ·ΠΎΠ²Π°Ρ‚Π΅Π»Ρ ΠΈ выполняСм ΠΏΠ΅Ρ€Π²ΠΎΠ½Π°Ρ‡Π°Π»ΡŒΠ½ΡƒΡŽ настройку
RUN chmod +x /splunkforwarder/bin/scripts/*.sh 
    && groupadd -r splunk 
    && useradd -r -m -g splunk splunk 
    && echo "%sudo ALL=NOPASSWD:ALL" >> /etc/sudoers 
    && chown -R splunk:splunk $SPLUNK_HOME 
    && /splunkforwarder/bin/first_start.sh 
    && /splunkforwarder/bin/splunk install app /splunkclouduf.spl -auth admin:changeme 
    && /splunkforwarder/bin/splunk restart

# ΠšΠΎΠΏΠΈΡ€ΡƒΠ΅ΠΌ ΠΈΠ½ΠΈΡ‚ скрипты
COPY [ "init/entrypoint.sh", "init/checkstate.sh", "/sbin/" ]

# По ТСланию. ΠšΠΎΠΌΡƒ Π½ΡƒΠΆΠ½ΠΎ локально ΠΈΠΌΠ΅Ρ‚ΡŒ ΠΊΠΎΠ½Ρ„ΠΈΠ³ΠΈ/Π»ΠΎΠ³ΠΈ, ΠΊΠΎΠΌΡƒ Π½Π΅Ρ‚.
VOLUME [ "/splunkforwarder/etc", "/splunkforwarder/var" ]

HEALTHCHECK --interval=30s --timeout=30s --start-period=3m --retries=5 CMD /sbin/checkstate.sh || exit 1

ENTRYPOINT [ "/sbin/entrypoint.sh" ]
CMD [ "start-service" ]

Ngakho yini equkethwe kuyo

qala_kuqala.sh

#!/usr/bin/expect -f
set timeout -1
spawn /splunkforwarder/bin/splunk start --accept-license
expect "Please enter an administrator username: "
send -- "adminr"
expect "Please enter a new password: "
send -- "changemer"
expect "Please confirm new password: "
send -- "changemer"
expect eof

Ekuqaleni kokuqala, u-Splunk ukucela ukuthi unikeze igama lokungena/iphasiwedi, KODWA le datha iyasetshenziswa kuphela ukwenza imiyalo yokulawula yalokho kufakwa okuthile, okungukuthi, ngaphakathi kwesiqukathi. Thina nje sifuna ukwethula isitsha ukuze yonke into isebenze nezingodo zigeleze njengomfula. Yiqiniso, lokhu kuyi-hardcode, kodwa angizange ngithole ezinye izindlela.

Ngokuqhubekayo ngokweskripthi senziwa

/splunkforwarder/bin/splunk install app /splunkclouduf.spl -auth admin:changeme

i-splunkclouduf.spl β€” Leli yifayela lokuqinisekisa le-Splunk Universal Forwarder, elingadawunilodeka kusixhumi esibonakalayo sewebhu.

Ungachofoza kuphi ukuze ulande (ezithombeni)I-Splunk Universal Forwarder kudokhi njengomqoqi welogi yesistimu

I-Splunk Universal Forwarder kudokhi njengomqoqi welogi yesistimu
Lena ingobo yomlando evamile engakhululwa. Ngaphakathi kukhona izitifiketi kanye nephasiwedi yokuxhuma ku-SplunkCloud yethu kanye okuphumayo.conf ngohlu lwezimo zethu zokufaka. Leli fayela lizosebenza kuze kube yilapho ufaka kabusha ukufaka kwakho kwe-Splunk noma wengeza inodi yokufaka uma ukufakwa kusendaweni. Ngakho-ke, akukho lutho olungalungile ngokuyingeza ngaphakathi kwesitsha.

Futhi into yokugcina ukuqala kabusha. Yebo, ukuze usebenzise izinguquko, udinga ukuyiqala kabusha.

Kwethu okokufaka.conf sengeza izingodo esifuna ukuzithumela ku-Splunk. Akudingekile ukwengeza leli fayela esithombeni uma, ngokwesibonelo, usakaza okulungiselelwe nge-puppet. Okuwukuphela kwento ukuthi uMdluliseli ubona izilungiselelo lapho i-daemon iqala, ngaphandle kwalokho izodinga ./splunk qala kabusha.

Hlobo luni lwemibhalo yezibalo ze-docker? Kunesixazululo esidala ku-Github kusuka outcoldman, imibhalo yathathwa lapho futhi yalungiswa ukuze isebenze nezinguqulo zamanje ze-Docker (ce-17.*) kanye ne-Splunk (7.*).

Ngedatha etholiwe, ungakha okulandelayo

amadeshibhodi: (izithombe ezimbalwa)I-Splunk Universal Forwarder kudokhi njengomqoqi welogi yesistimu

I-Splunk Universal Forwarder kudokhi njengomqoqi welogi yesistimu
Ikhodi yomthombo yamadeshi ikusixhumanisi esinikezwe ekugcineni kwe-athikili. Sicela uqaphele ukuthi kunezinkambu ezikhethiwe ezi-2: 1 - ukukhethwa kwenkomba (kuseshwe ngemaski), ukukhetha komsingathi/isitsha. Cishe uzodinga ukubuyekeza imaski yenkomba, kuye ngamagama owasebenzisayo.

Sengiphetha, ngithanda ukunakwa kulo msebenzi qala() Π²

indawo yokungena.sh

start() {
    trap teardown EXIT
	if [ -z $SPLUNK_INDEX ]; then
	echo "'SPLUNK_INDEX' env variable is empty or not defined. Should be 'dev' or 'prd'." >&2
	exit 1
	else
	sed -e "s/@index@/$SPLUNK_INDEX/" -i ${SPLUNK_HOME}/etc/system/local/inputs.conf
	fi
	sed -e "s/@hostname@/$(cat /etc/hostname)/" -i ${SPLUNK_HOME}/etc/system/local/inputs.conf
    sh -c "echo 'starting' > /tmp/splunk-container.state"
	${SPLUNK_HOME}/bin/splunk start
    watch_for_failure
}

Endabeni yami, endaweni ngayinye kanye nenhlangano ngayinye ngayinye, kungaba isicelo esitsheni noma umshini wokusingatha, sisebenzisa inkomba ehlukile. Ngale ndlela, isivinini sokusesha ngeke sihlupheke uma kukhona ukuqoqwa okubalulekile kwedatha. Umthetho olula usetshenziswa ukubiza izinkomba: _. Ngakho-ke, ukuze isiqukathi sibe yindawo yonke, ngaphambi kokwethula i-daemon ngokwayo, siyayishintsha sed-th wildcard egameni lendawo. Igama eliguquguqukayo lendawo lidluliswa ngokuguquguquka kwemvelo. Kuzwakala kuhlekisa.

Kuyafaneleka futhi ukuqaphela ukuthi ngesizathu esithile i-Splunk ayithinteki ngokuba khona kwepharamitha ye-docker igama lomkhosi. Usazothumela inkani izingodo ezinomazisi wesiqukathi sakhe enkundleni yokusingatha. Njengesixazululo, ungakwazi ukukhweza / njll / igama lomkhosi kusuka emshinini wokusingathwa futhi ekuqaleni yenza ukushintshwa okufana namagama ezinkomba.

Isibonelo se-docker-compose.yml

version: '2'
services:
  splunk-forwarder:
    image: "${IMAGE_REPO}/docker-stats-splunk-forwarder:${IMAGE_VERSION}"
    environment:
      SPLUNK_INDEX: ${ENVIRONMENT}
    volumes:
    - /etc/hostname:/etc/hostname:ro
    - /var/log:/var/log
    - /var/run/docker.sock:/var/run/docker.sock:ro

Umphumela

Yebo, mhlawumbe ikhambi alilungile futhi ngokuqinisekile alikho kuwo wonke umuntu, ngoba maningi "i-hardcode". Kodwa ngokusekelwe kulo, wonke umuntu angakha isithombe sakhe siqu futhi asibeke endaweni yakhe yangasese, uma, njengoba kwenzeka, udinga i-Splunk Forwarder ku-Docker.

Izinkomba:

Isixazululo esivela esihlokweni
Isixazululo esivela ku-outcoldman esisigqugquzele ukuthi siphinde sisebenzise eminye imisebenzi
Kwe. imibhalo yokusetha i-Universal Forwarder

Source: www.habr.com

Engeza amazwana