🥇I-Splunk Universal Forwarder ku-Docker Njengomqoqi Welogi Yesistimu

I-Splunk ingenye yeqoqo lemibhalo yezentengiselwano eqashelwa kakhulu kanye nemikhiqizo yokuhlaziya. Ngisho namanje, lapho ukuthengisa kungasakwenziwa eRussia, lesi akusona isizathu sokungabhali imiyalelo/ukwenza kanjani lo mkhiqizo.

Inhloso: qoqa amalogi wesistimu kuma-docker node ku-Splunk ngaphandle kokushintsha ukucushwa komshini wokusingatha

Ngingathanda ukuqala ngendlela esemthethweni, ebukeka iyinqaba uma usebenzisa i-Docker.
Xhuma kuhabhu le-Docker
Yini esinayo:

1. Isithombe se-Pullim

$ docker pull splunk/universalforwarder:latest

2. Qala isitsha ngemingcele edingekayo

$ docker run -d  -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=<password>' splunk/universalforwarder:latest

3. Singena esitsheni

docker exec -it <container-id> /bin/bash

Okulandelayo, siyacelwa ukuthi siye ekhelini elaziwayo embhalweni.

Futhi ulungiselele isiqukathi ngemva kokuthi siqalile:


./splunk add forward-server <host name or ip address>:<listening port>
./splunk add monitor /var/log
./splunk restart

Linda. Ini?

Kodwa izimanga azigcini lapho. Uma usebenzisa isiqukathi esivela esithombeni esisemthethweni ngemodi yokusebenzisana, uzobona okulandelayo:

Ukudumala kancane


$ docker run -it -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=password' splunk/universalforwarder:latest

PLAY [Run default Splunk provisioning] *******************************************************************************************************************************************************************************************************
Tuesday 09 April 2019  13:40:38 +0000 (0:00:00.096)       0:00:00.096 *********

TASK [Gathering Facts] ***********************************************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:39 +0000 (0:00:01.520)       0:00:01.616 *********

TASK [Get actual hostname] *******************************************************************************************************************************************************************************************************************
changed: [localhost]
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.599)       0:00:02.215 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.054)       0:00:02.270 *********

TASK [set_fact] ******************************************************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.075)       0:00:02.346 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.067)       0:00:02.413 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.060)       0:00:02.473 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.051)       0:00:02.525 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.056)       0:00:02.582 *********
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.216)       0:00:02.798 *********
included: /opt/ansible/roles/splunk_common/tasks/change_splunk_directory_owner.yml for localhost
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.087)       0:00:02.886 *********

TASK [splunk_common : Update Splunk directory owner] *****************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.324)       0:00:03.210 *********
included: /opt/ansible/roles/splunk_common/tasks/get_facts.yml for localhost
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.094)       0:00:03.305 *********

ну и так далее...

Kuhle. Isithombe asiqukethe ngisho ne-artifact. Okusho ukuthi, ngaso sonke isikhathi uma uqala kuzothatha isikhathi ukulanda ingobo yomlando ngamabhanari, uyikhiphe futhi uyilungiselele.
Kuthiwani nge-docker-way nakho konke lokho?

Cha ngiyabonga. Sizothatha umzila ohlukile. Kuthiwani uma senza yonke le misebenzi esigabeni somhlangano? Bese sihamba!

Ukuze ngingalibali isikhathi eside, ngizokukhombisa isithombe sokugcina ngaso leso sikhathi:

I-Dockerfile

# Тут у кого какие предпочтения
FROM centos:7

# Задаём переменные, чтобы каждый раз при старте не указывать их
ENV SPLUNK_HOME /splunkforwarder
ENV SPLUNK_ROLE splunk_heavy_forwarder
ENV SPLUNK_PASSWORD changeme
ENV SPLUNK_START_ARGS --accept-license

# Ставим пакеты
# wget - чтобы скачать артефакты
# expect - понадобится для первоначального запуска Splunk на этапе сборки
# jq - используется в скриптах, которые собирают статистику докера
RUN yum install -y epel-release 
    && yum install -y wget expect jq

# Качаем, распаковываем, удаляем
RUN wget -O splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.2.4&product=universalforwarder&filename=splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz&wget=true' 
    && wget -O docker-18.09.3.tgz 'https://download.docker.com/linux/static/stable/x86_64/docker-18.09.3.tgz' 
    && tar -xvf splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 
    && tar -xvf docker-18.09.3.tgz  
    && rm -f splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 
    && rm -f docker-18.09.3.tgz

# С shell скриптами всё понятно, а вот inputs.conf, splunkclouduf.spl и first_start.sh нуждаются в пояснении. Об этом расскажу после source тэга.
COPY [ "inputs.conf", "docker-stats/props.conf", "/splunkforwarder/etc/system/local/" ]
COPY [ "docker-stats/docker_events.sh", "docker-stats/docker_inspect.sh", "docker-stats/docker_stats.sh", "docker-stats/docker_top.sh", "/splunkforwarder/bin/scripts/" ]
COPY splunkclouduf.spl /splunkclouduf.spl
COPY first_start.sh /splunkforwarder/bin/

#  Даём права на исполнение, добавляем пользователя и выполняем первоначальную настройку
RUN chmod +x /splunkforwarder/bin/scripts/*.sh 
    && groupadd -r splunk 
    && useradd -r -m -g splunk splunk 
    && echo "%sudo ALL=NOPASSWD:ALL" >> /etc/sudoers 
    && chown -R splunk:splunk $SPLUNK_HOME 
    && /splunkforwarder/bin/first_start.sh 
    && /splunkforwarder/bin/splunk install app /splunkclouduf.spl -auth admin:changeme 
    && /splunkforwarder/bin/splunk restart

# Копируем инит скрипты
COPY [ "init/entrypoint.sh", "init/checkstate.sh", "/sbin/" ]

# По желанию. Кому нужно локально иметь конфиги/логи, кому нет.
VOLUME [ "/splunkforwarder/etc", "/splunkforwarder/var" ]

HEALTHCHECK --interval=30s --timeout=30s --start-period=3m --retries=5 CMD /sbin/checkstate.sh || exit 1

ENTRYPOINT [ "/sbin/entrypoint.sh" ]
CMD [ "start-service" ]

Ngakho yini equkethwe kuyo

qala_kuqala.sh

#!/usr/bin/expect -f
set timeout -1
spawn /splunkforwarder/bin/splunk start --accept-license
expect "Please enter an administrator username: "
send -- "adminr"
expect "Please enter a new password: "
send -- "changemer"
expect "Please confirm new password: "
send -- "changemer"
expect eof

Ekuqaleni kokuqala, u-Splunk ukucela ukuthi unikeze igama lokungena/iphasiwedi, KODWA le datha iyasetshenziswa kuphela ukwenza imiyalo yokulawula yalokho kufakwa okuthile, okungukuthi, ngaphakathi kwesiqukathi. Thina nje sifuna ukwethula isitsha ukuze yonke into isebenze nezingodo zigeleze njengomfula. Yiqiniso, lokhu kuyi-hardcode, kodwa angizange ngithole ezinye izindlela.

Ngokuqhubekayo ngokweskripthi senziwa

/splunkforwarder/bin/splunk install app /splunkclouduf.spl -auth admin:changeme

i-splunkclouduf.spl — Leli yifayela lokuqinisekisa le-Splunk Universal Forwarder, elingadawunilodeka kusixhumi esibonakalayo sewebhu.

Ungachofoza kuphi ukuze ulande (ezithombeni)

Lena ingobo yomlando evamile engakhululwa. Ngaphakathi kukhona izitifiketi kanye nephasiwedi yokuxhuma ku-SplunkCloud yethu kanye okuphumayo.conf ngohlu lwezimo zethu zokufaka. Leli fayela lizosebenza kuze kube yilapho ufaka kabusha ukufaka kwakho kwe-Splunk noma wengeza inodi yokufaka uma ukufakwa kusendaweni. Ngakho-ke, akukho lutho olungalungile ngokuyingeza ngaphakathi kwesitsha.

Futhi into yokugcina ukuqala kabusha. Yebo, ukuze usebenzise izinguquko, udinga ukuyiqala kabusha.

Kwethu okokufaka.conf sengeza izingodo esifuna ukuzithumela ku-Splunk. Akudingekile ukwengeza leli fayela esithombeni uma, ngokwesibonelo, usakaza okulungiselelwe nge-puppet. Okuwukuphela kwento ukuthi uMdluliseli ubona izilungiselelo lapho i-daemon iqala, ngaphandle kwalokho izodinga ./splunk qala kabusha.

Hlobo luni lwemibhalo yezibalo ze-docker? Kunesixazululo esidala ku-Github kusuka outcoldman, imibhalo yathathwa lapho futhi yalungiswa ukuze isebenze nezinguqulo zamanje ze-Docker (ce-17.*) kanye ne-Splunk (7.*).

Ngedatha etholiwe, ungakha okulandelayo

amadeshibhodi: (izithombe ezimbalwa)

Ikhodi yomthombo yamadeshi ikusixhumanisi esinikezwe ekugcineni kwe-athikili. Sicela uqaphele ukuthi kunezinkambu ezikhethiwe ezi-2: 1 - ukukhethwa kwenkomba (kuseshwe ngemaski), ukukhetha komsingathi/isitsha. Cishe uzodinga ukubuyekeza imaski yenkomba, kuye ngamagama owasebenzisayo.

Sengiphetha, ngithanda ukunakwa kulo msebenzi qala() в

indawo yokungena.sh

start() {
    trap teardown EXIT
	if [ -z $SPLUNK_INDEX ]; then
	echo "'SPLUNK_INDEX' env variable is empty or not defined. Should be 'dev' or 'prd'." >&2
	exit 1
	else
	sed -e "s/@index@/$SPLUNK_INDEX/" -i ${SPLUNK_HOME}/etc/system/local/inputs.conf
	fi
	sed -e "s/@hostname@/$(cat /etc/hostname)/" -i ${SPLUNK_HOME}/etc/system/local/inputs.conf
    sh -c "echo 'starting' > /tmp/splunk-container.state"
	${SPLUNK_HOME}/bin/splunk start
    watch_for_failure
}

Endabeni yami, endaweni ngayinye kanye nenhlangano ngayinye ngayinye, kungaba isicelo esitsheni noma umshini wokusingatha, sisebenzisa inkomba ehlukile. Ngale ndlela, isivinini sokusesha ngeke sihlupheke uma kukhona ukuqoqwa okubalulekile kwedatha. Umthetho olula usetshenziswa ukubiza izinkomba: _. Ngakho-ke, ukuze isiqukathi sibe yindawo yonke, ngaphambi kokwethula i-daemon ngokwayo, siyayishintsha sed-th wildcard egameni lendawo. Igama eliguquguqukayo lendawo lidluliswa ngokuguquguquka kwemvelo. Kuzwakala kuhlekisa.

Kuyafaneleka futhi ukuqaphela ukuthi ngesizathu esithile i-Splunk ayithinteki ngokuba khona kwepharamitha ye-docker igama lomkhosi. Usazothumela inkani izingodo ezinomazisi wesiqukathi sakhe enkundleni yokusingatha. Njengesixazululo, ungakwazi ukukhweza / njll / igama lomkhosi kusuka emshinini wokusingathwa futhi ekuqaleni yenza ukushintshwa okufana namagama ezinkomba.

Isibonelo se-docker-compose.yml

version: '2'
services:
  splunk-forwarder:
    image: "${IMAGE_REPO}/docker-stats-splunk-forwarder:${IMAGE_VERSION}"
    environment:
      SPLUNK_INDEX: ${ENVIRONMENT}
    volumes:
    - /etc/hostname:/etc/hostname:ro
    - /var/log:/var/log
    - /var/run/docker.sock:/var/run/docker.sock:ro

Umphumela

Yebo, mhlawumbe ikhambi alilungile futhi ngokuqinisekile alikho kuwo wonke umuntu, ngoba maningi "i-hardcode". Kodwa ngokusekelwe kulo, wonke umuntu angakha isithombe sakhe siqu futhi asibeke endaweni yakhe yangasese, uma, njengoba kwenzeka, udinga i-Splunk Forwarder ku-Docker.

Izinkomba:

Isixazululo esivela esihlokweni
Isixazululo esivela ku-outcoldman esisigqugquzele ukuthi siphinde sisebenzise eminye imisebenzi
Kwe. imibhalo yokusetha i-Universal Forwarder

Source: www.habr.com

I-Splunk Universal Forwarder kudokhi njengomqoqi welogi yesistimu