🥇Splunk Oludari gbogbo agbaye ni Docker bi Akojọpọ Wọle Eto

Splunk jẹ ọkan ninu ọpọlọpọ awọn gbigba akọọlẹ iṣowo ti o ṣe idanimọ julọ ati awọn ọja itupalẹ. Paapaa ni bayi, nigbati awọn tita ko ba ṣe ni Russia, eyi kii ṣe idi kan lati kọ awọn ilana / bii-si fun ọja yii.

Nkan: gba awọn igbasilẹ eto lati awọn apa docker ni Splunk laisi iyipada iṣeto ẹrọ ogun

Emi yoo fẹ lati bẹrẹ pẹlu ọna osise, eyiti o dabi ajeji diẹ nigba lilo Docker.
Ọna asopọ si ibudo Docker
Kini a ni:

1. Pullim aworan

$ docker pull splunk/universalforwarder:latest

2. Bẹrẹ eiyan pẹlu awọn paramita pataki

$ docker run -d  -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=<password>' splunk/universalforwarder:latest

3. A lọ sinu apo eiyan

docker exec -it <container-id> /bin/bash

Nigbamii ti, a beere lọwọ wa lati lọ si adirẹsi ti a mọ ninu iwe-ipamọ naa.

Ati tunto apoti lẹhin ti o bẹrẹ:


./splunk add forward-server <host name or ip address>:<listening port>
./splunk add monitor /var/log
./splunk restart

Duro. Kini?

Ṣugbọn awọn iyanilẹnu ko pari nibẹ. Ti o ba ṣiṣẹ eiyan lati aworan osise ni ipo ibaraenisepo, iwọ yoo rii atẹle naa:

A bit ti oriyin


$ docker run -it -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=password' splunk/universalforwarder:latest

PLAY [Run default Splunk provisioning] *******************************************************************************************************************************************************************************************************
Tuesday 09 April 2019  13:40:38 +0000 (0:00:00.096)       0:00:00.096 *********

TASK [Gathering Facts] ***********************************************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:39 +0000 (0:00:01.520)       0:00:01.616 *********

TASK [Get actual hostname] *******************************************************************************************************************************************************************************************************************
changed: [localhost]
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.599)       0:00:02.215 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.054)       0:00:02.270 *********

TASK [set_fact] ******************************************************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.075)       0:00:02.346 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.067)       0:00:02.413 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.060)       0:00:02.473 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.051)       0:00:02.525 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.056)       0:00:02.582 *********
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.216)       0:00:02.798 *********
included: /opt/ansible/roles/splunk_common/tasks/change_splunk_directory_owner.yml for localhost
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.087)       0:00:02.886 *********

TASK [splunk_common : Update Splunk directory owner] *****************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.324)       0:00:03.210 *********
included: /opt/ansible/roles/splunk_common/tasks/get_facts.yml for localhost
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.094)       0:00:03.305 *********

ну и так далее...

Nla. Àwòrán náà kò tilẹ̀ ní ohun ìṣẹ̀ǹbáyé nínú. Iyẹn ni, ni gbogbo igba ti o ba bẹrẹ yoo gba akoko lati ṣe igbasilẹ iwe-ipamọ pẹlu awọn alakomeji, ṣiṣi silẹ ati tunto.
Kini nipa ọna docker ati gbogbo iyẹn?

Rara o se. A yoo gba ọna ti o yatọ. Kini ti a ba ṣe gbogbo awọn iṣẹ wọnyi ni ipele apejọ? Lẹhinna jẹ ki a lọ!

Ni ibere ki o ma ṣe pẹ ju, Emi yoo fi aworan ikẹhin han ọ lẹsẹkẹsẹ:

dockerfile

# Тут у кого какие предпочтения
FROM centos:7

# Задаём переменные, чтобы каждый раз при старте не указывать их
ENV SPLUNK_HOME /splunkforwarder
ENV SPLUNK_ROLE splunk_heavy_forwarder
ENV SPLUNK_PASSWORD changeme
ENV SPLUNK_START_ARGS --accept-license

# Ставим пакеты
# wget - чтобы скачать артефакты
# expect - понадобится для первоначального запуска Splunk на этапе сборки
# jq - используется в скриптах, которые собирают статистику докера
RUN yum install -y epel-release 
    && yum install -y wget expect jq

# Качаем, распаковываем, удаляем
RUN wget -O splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.2.4&product=universalforwarder&filename=splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz&wget=true' 
    && wget -O docker-18.09.3.tgz 'https://download.docker.com/linux/static/stable/x86_64/docker-18.09.3.tgz' 
    && tar -xvf splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 
    && tar -xvf docker-18.09.3.tgz  
    && rm -f splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 
    && rm -f docker-18.09.3.tgz

# С shell скриптами всё понятно, а вот inputs.conf, splunkclouduf.spl и first_start.sh нуждаются в пояснении. Об этом расскажу после source тэга.
COPY [ "inputs.conf", "docker-stats/props.conf", "/splunkforwarder/etc/system/local/" ]
COPY [ "docker-stats/docker_events.sh", "docker-stats/docker_inspect.sh", "docker-stats/docker_stats.sh", "docker-stats/docker_top.sh", "/splunkforwarder/bin/scripts/" ]
COPY splunkclouduf.spl /splunkclouduf.spl
COPY first_start.sh /splunkforwarder/bin/

#  Даём права на исполнение, добавляем пользователя и выполняем первоначальную настройку
RUN chmod +x /splunkforwarder/bin/scripts/*.sh 
    && groupadd -r splunk 
    && useradd -r -m -g splunk splunk 
    && echo "%sudo ALL=NOPASSWD:ALL" >> /etc/sudoers 
    && chown -R splunk:splunk $SPLUNK_HOME 
    && /splunkforwarder/bin/first_start.sh 
    && /splunkforwarder/bin/splunk install app /splunkclouduf.spl -auth admin:changeme 
    && /splunkforwarder/bin/splunk restart

# Копируем инит скрипты
COPY [ "init/entrypoint.sh", "init/checkstate.sh", "/sbin/" ]

# По желанию. Кому нужно локально иметь конфиги/логи, кому нет.
VOLUME [ "/splunkforwarder/etc", "/splunkforwarder/var" ]

HEALTHCHECK --interval=30s --timeout=30s --start-period=3m --retries=5 CMD /sbin/checkstate.sh || exit 1

ENTRYPOINT [ "/sbin/entrypoint.sh" ]
CMD [ "start-service" ]

Nitorina kini o wa ninu

akọkọ_bẹrẹ.sh

#!/usr/bin/expect -f
set timeout -1
spawn /splunkforwarder/bin/splunk start --accept-license
expect "Please enter an administrator username: "
send -- "adminr"
expect "Please enter a new password: "
send -- "changemer"
expect "Please confirm new password: "
send -- "changemer"
expect eof

Ni ibẹrẹ akọkọ, Splunk beere lọwọ rẹ lati fun u ni iwọle / ọrọ igbaniwọle, Ṣugbọn data yii ti lo Nikan lati ṣiṣẹ awọn aṣẹ iṣakoso fun fifi sori ẹrọ pato yẹn, iyẹn ni, inu eiyan naa. Ninu ọran tiwa, a kan fẹ lati lọlẹ apoti naa ki ohun gbogbo ba ṣiṣẹ ati awọn igi ṣan bi odo. Nitoribẹẹ, eyi jẹ koodu lile, ṣugbọn Emi ko rii awọn ọna miiran.

Siwaju ni ibamu si awọn akosile ti wa ni executed

/splunkforwarder/bin/splunk install app /splunkclouduf.spl -auth admin:changeme

splunkclouuf.spl - Eyi jẹ faili awọn iwe-ẹri fun Splunk Universal Forwarder, eyiti o le ṣe igbasilẹ lati wiwo wẹẹbu.

Nibo ni lati tẹ lati ṣe igbasilẹ (ninu awọn aworan)

Eyi jẹ ile-ipamọ deede ti o le jẹ ṣiṣi silẹ. Inu wa awọn iwe-ẹri ati ọrọ igbaniwọle kan fun sisopọ si SplunkCloud wa ati àbájade.conf pẹlu akojọ kan ti wa input instances. Faili yii yoo wulo titi ti o fi tun fi sori ẹrọ Splunk rẹ tabi ṣafikun oju-ọna igbewọle ti fifi sori ẹrọ ba wa ni agbegbe ile. Nitorina, ko si ohun ti ko tọ si pẹlu fifi kun sinu apo.

Ati ohun ti o kẹhin jẹ tun bẹrẹ. Bẹẹni, lati lo awọn ayipada, o nilo lati tun bẹrẹ.

Ninu wa awọn igbewọle.conf a fi awọn akọọlẹ ti a fẹ fi ranṣẹ si Splunk. Ko ṣe pataki lati ṣafikun faili yii si aworan ti, fun apẹẹrẹ, o pin awọn atunto nipasẹ puppet. Ohun kan ni pe Forwarder rii awọn atunto nigbati daemon ba bẹrẹ, bibẹẹkọ yoo nilo ./splunk tun bẹrẹ.

Iru awọn iwe afọwọkọ awọn iṣiro docker wo ni wọn jẹ? Nibẹ jẹ ẹya atijọ ojutu on Github lati outcoldman, awọn iwe afọwọkọ ti a ya lati ibẹ ati titunṣe lati ṣiṣẹ pẹlu awọn ẹya lọwọlọwọ ti Docker (ce-17.*) ati Splunk (7.*).

Pẹlu data ti o gba, o le kọ atẹle naa

dashboards: (awọn aworan meji)

Awọn koodu orisun fun awọn dashes wa ninu ọna asopọ ti a pese ni opin nkan naa. Jọwọ ṣe akiyesi pe awọn aaye yiyan 2 wa: 1 - yiyan atọka (ti a wa nipasẹ iboju-boju), yiyan agbalejo/apoti. O ṣeese o nilo lati ṣe imudojuiwọn iboju-itọka, da lori awọn orukọ ti o lo.

Ni ipari, Emi yoo fẹ lati fa ifojusi rẹ si iṣẹ naa bẹrẹ() в

aaye wiwọle.sh

start() {
    trap teardown EXIT
	if [ -z $SPLUNK_INDEX ]; then
	echo "'SPLUNK_INDEX' env variable is empty or not defined. Should be 'dev' or 'prd'." >&2
	exit 1
	else
	sed -e "s/@index@/$SPLUNK_INDEX/" -i ${SPLUNK_HOME}/etc/system/local/inputs.conf
	fi
	sed -e "s/@hostname@/$(cat /etc/hostname)/" -i ${SPLUNK_HOME}/etc/system/local/inputs.conf
    sh -c "echo 'starting' > /tmp/splunk-container.state"
	${SPLUNK_HOME}/bin/splunk start
    watch_for_failure
}

Ninu ọran mi, fun agbegbe kọọkan ati ohun elo kọọkan, jẹ ohun elo ninu apoti kan tabi ẹrọ agbalejo, a lo itọka lọtọ. Ni ọna yii, iyara wiwa kii yoo jiya nigbati ikojọpọ pataki ti data wa. Ofin ti o rọrun ni a lo lati lorukọ awọn atọka: _. Nitorinaa, ni ibere fun eiyan lati jẹ agbaye, ṣaaju ifilọlẹ daemon funrararẹ, a rọpo sed-th wildcard si awọn orukọ ti awọn ayika. Oniyipada orukọ ayika ti kọja nipasẹ awọn oniyipada ayika. Dun funny.

O tun tọ lati ṣe akiyesi pe fun idi kan Splunk ko ni ipa nipasẹ wiwa ti paramita docker hostname. Oun yoo tun fi agidi fi awọn igi ranṣẹ pẹlu id eiyan rẹ sinu aaye agbalejo. Bi ojutu, o le gbe soke / ati be be lo / hostname lati ẹrọ ogun ati ni ibẹrẹ ṣe awọn iyipada ti o jọra si awọn orukọ atọka.

Apeere docker-compose.yml

version: '2'
services:
  splunk-forwarder:
    image: "${IMAGE_REPO}/docker-stats-splunk-forwarder:${IMAGE_VERSION}"
    environment:
      SPLUNK_INDEX: ${ENVIRONMENT}
    volumes:
    - /etc/hostname:/etc/hostname:ro
    - /var/log:/var/log
    - /var/run/docker.sock:/var/run/docker.sock:ro

Abajade

Bẹẹni, boya ojutu naa ko dara julọ ati pe dajudaju kii ṣe gbogbo agbaye fun gbogbo eniyan, nitori ọpọlọpọ wa "koodu lile". Ṣugbọn ti o da lori rẹ, gbogbo eniyan le kọ aworan ti ara wọn ki o fi si ile-iṣẹ aladani wọn, ti, bi o ti ṣẹlẹ, o nilo Splunk Forwarder ni Docker.

Awọn ọna asopọ:

Solusan lati nkan naa
Ojutu lati ọdọ outcoldman ti o ṣe atilẹyin fun wa lati tun lo diẹ ninu awọn iṣẹ ṣiṣe
Ti. iwe fun eto soke Universal Forwarder

orisun: www.habr.com

Splunk Universal Forwarder ni docker bi a eto log-odè