Splunk Universal Forwarder a Docker azaman mai tattara log log

Splunk yana ɗaya daga cikin mafi yawan sanannun tarin log ɗin kasuwanci da samfuran bincike. Ko da a yanzu, lokacin da aka daina yin tallace-tallace a Rasha, wannan ba dalili ba ne don rubuta umarnin / yadda ake yin wannan samfurin.

Manufar: tattara tsarin rajistan ayyukan daga docker nodes a cikin Splunk ba tare da canza tsarin injin mai watsa shiri ba

Ina so in fara da tsarin hukuma, wanda yayi kama da ban mamaki lokacin amfani da Docker.
Hanyar zuwa tashar Docker
Me muke da shi:

1. Hoton Pullim

$ docker pull splunk/universalforwarder:latest

2. Fara akwati tare da sigogi masu mahimmanci

$ docker run -d  -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=<password>' splunk/universalforwarder:latest

3. Muna shiga cikin akwati

docker exec -it <container-id> /bin/bash

Na gaba, an umarce mu mu je wani sanannen adireshin a cikin takaddun.

Kuma saita kwandon bayan ya fara:

./splunk add forward-server <host name or ip address>:<listening port>
./splunk add monitor /var/log
./splunk restart

Jira Menene?

Amma abin mamaki bai ƙare a nan ba. Idan kun gudanar da akwati daga hoton hukuma a cikin yanayin mu'amala, zaku ga mai zuwa:

Dan takaici

$ docker run -it -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=password' splunk/universalforwarder:latest

PLAY [Run default Splunk provisioning] *******************************************************************************************************************************************************************************************************
Tuesday 09 April 2019  13:40:38 +0000 (0:00:00.096)       0:00:00.096 *********

TASK [Gathering Facts] ***********************************************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:39 +0000 (0:00:01.520)       0:00:01.616 *********

TASK [Get actual hostname] *******************************************************************************************************************************************************************************************************************
changed: [localhost]
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.599)       0:00:02.215 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.054)       0:00:02.270 *********

TASK [set_fact] ******************************************************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.075)       0:00:02.346 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.067)       0:00:02.413 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.060)       0:00:02.473 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.051)       0:00:02.525 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.056)       0:00:02.582 *********
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.216)       0:00:02.798 *********
included: /opt/ansible/roles/splunk_common/tasks/change_splunk_directory_owner.yml for localhost
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.087)       0:00:02.886 *********

TASK [splunk_common : Update Splunk directory owner] *****************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.324)       0:00:03.210 *********
included: /opt/ansible/roles/splunk_common/tasks/get_facts.yml for localhost
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.094)       0:00:03.305 *********

ну и так далее...

Mai girma. Hoton bashi da wani kayan tarihi. Wato duk lokacin da ka fara za a ɗauki lokaci don zazzage ma'ajin tare da binary, cire kaya da daidaitawa.
Me game da docker-way da duk wannan?

A'a na gode. Za mu ɗauki wata hanya ta daban. Idan muka yi duk waɗannan ayyuka a matakin taro fa? Sai mu tafi!

Domin kar a dade da yawa, zan nuna muku hoton karshe nan da nan:

Dockerfile

# Тут у кого какие предпочтения
FROM centos:7

# Задаём переменные, чтобы каждый раз при старте не указывать их
ENV SPLUNK_HOME /splunkforwarder
ENV SPLUNK_ROLE splunk_heavy_forwarder
ENV SPLUNK_PASSWORD changeme
ENV SPLUNK_START_ARGS --accept-license

# Ставим пакеты
# wget - чтобы скачать артефакты
# expect - понадобится для первоначального запуска Splunk на этапе сборки
# jq - используется в скриптах, которые собирают статистику докера
RUN yum install -y epel-release 
    && yum install -y wget expect jq

# Качаем, распаковываем, удаляем
RUN wget -O splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.2.4&product=universalforwarder&filename=splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz&wget=true' 
    && wget -O docker-18.09.3.tgz 'https://download.docker.com/linux/static/stable/x86_64/docker-18.09.3.tgz' 
    && tar -xvf splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 
    && tar -xvf docker-18.09.3.tgz  
    && rm -f splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 
    && rm -f docker-18.09.3.tgz

# С shell скриптами всё понятно, а вот inputs.conf, splunkclouduf.spl и first_start.sh нуждаются в пояснении. Об этом расскажу после source тэга.
COPY [ "inputs.conf", "docker-stats/props.conf", "/splunkforwarder/etc/system/local/" ]
COPY [ "docker-stats/docker_events.sh", "docker-stats/docker_inspect.sh", "docker-stats/docker_stats.sh", "docker-stats/docker_top.sh", "/splunkforwarder/bin/scripts/" ]
COPY splunkclouduf.spl /splunkclouduf.spl
COPY first_start.sh /splunkforwarder/bin/

#  Даём права на исполнение, добавляем пользователя и выполняем первоначальную настройку
RUN chmod +x /splunkforwarder/bin/scripts/*.sh 
    && groupadd -r splunk 
    && useradd -r -m -g splunk splunk 
    && echo "%sudo ALL=NOPASSWD:ALL" >> /etc/sudoers 
    && chown -R splunk:splunk $SPLUNK_HOME 
    && /splunkforwarder/bin/first_start.sh 
    && /splunkforwarder/bin/splunk install app /splunkclouduf.spl -auth admin:changeme 
    && /splunkforwarder/bin/splunk restart

# Копируем инит скрипты
COPY [ "init/entrypoint.sh", "init/checkstate.sh", "/sbin/" ]

# По желанию. Кому нужно локально иметь конфиги/логи, кому нет.
VOLUME [ "/splunkforwarder/etc", "/splunkforwarder/var" ]

HEALTHCHECK --interval=30s --timeout=30s --start-period=3m --retries=5 CMD /sbin/checkstate.sh || exit 1

ENTRYPOINT [ "/sbin/entrypoint.sh" ]
CMD [ "start-service" ]

Don haka abin da ya kunsa

farko_fara.sh

#!/usr/bin/expect -f
set timeout -1
spawn /splunkforwarder/bin/splunk start --accept-license
expect "Please enter an administrator username: "
send -- "adminr"
expect "Please enter a new password: "
send -- "changemer"
expect "Please confirm new password: "
send -- "changemer"
expect eof

A farkon farawa, Splunk yana tambayarka ka ba shi login/Password, AMMA ana amfani da wannan bayanan kawai don aiwatar da umarnin gudanarwa don wannan takamaiman shigarwa, wato, cikin akwati. A cikin yanayinmu, kawai muna so mu kaddamar da kwandon don komai ya yi aiki kuma katako yana gudana kamar kogi. Tabbas, wannan hardcode ne, amma ban sami wasu hanyoyi ba.

Ana aiwatar da gaba bisa ga rubutun

/splunkforwarder/bin/splunk install app /splunkclouduf.spl -auth admin:changeme

splunkclouuf.spl - Wannan fayil ne na takaddun shaida na Splunk Universal Forwarder, wanda za'a iya saukewa daga mahaɗin yanar gizo.

Inda za a danna don saukewa (a cikin hotuna)


Wannan rumbun adana bayanai ne na yau da kullun wanda za'a iya kwashewa. A ciki akwai takaddun shaida da kalmar sirri don haɗawa zuwa SplunkCloud da fitarwa.conf tare da jerin abubuwan shigar mu. Wannan fayil ɗin zai kasance mai dacewa har sai kun sake shigar da shigarwar Splunk ɗinku ko ƙara kumburin shigarwa idan shigarwar yana kan gaba. Saboda haka, babu laifi idan an ƙara shi a cikin akwati.

Kuma abu na ƙarshe shine sake farawa. Ee, don amfani da canje-canje, kuna buƙatar sake kunna shi.

A cikin mu abubuwan shiga.conf muna ƙara rajistan ayyukan da muke so mu aika zuwa Splunk. Ba lallai ba ne a ƙara wannan fayil ɗin zuwa hoton idan, alal misali, kuna rarraba saiti ta hanyar tsana. Abinda kawai shine Forwarder yana ganin saiti lokacin da daemon ya fara, in ba haka ba zai buƙaci ./splunk sake kunnawa.

Wane irin rubutun kididdiga ne? Akwai tsohuwar mafita akan Github daga outcoldman, An ɗauki rubutun daga can kuma an gyara su don aiki tare da nau'ikan Docker na yanzu (ce-17.*) da Splunk (7.*).

Tare da bayanan da aka samu, za ku iya gina waɗannan abubuwa

dashboards: (hotuna biyu)


Lambar tushe don dashes tana cikin hanyar haɗin da aka bayar a ƙarshen labarin. Lura cewa akwai filayen zaɓi guda 2: 1 - zaɓin fihirisa (nema ta hanyar abin rufe fuska), zaɓin masauki/kwantena. Wataƙila kuna buƙatar sabunta abin rufe fuska, dangane da sunayen da kuke amfani da su.

A ƙarshe, Ina so in jawo hankalin ku ga aikin fara() в

wurin shiga.sh

start() {
    trap teardown EXIT
	if [ -z $SPLUNK_INDEX ]; then
	echo "'SPLUNK_INDEX' env variable is empty or not defined. Should be 'dev' or 'prd'." >&2
	exit 1
	else
	sed -e "s/@index@/$SPLUNK_INDEX/" -i ${SPLUNK_HOME}/etc/system/local/inputs.conf
	fi
	sed -e "s/@hostname@/$(cat /etc/hostname)/" -i ${SPLUNK_HOME}/etc/system/local/inputs.conf
    sh -c "echo 'starting' > /tmp/splunk-container.state"
	${SPLUNK_HOME}/bin/splunk start
    watch_for_failure
}

A cikin yanayina, ga kowane yanayi da kowane mahallin mutum, zama aikace-aikace a cikin akwati ko na'ura mai watsa shiri, muna amfani da fihirisa daban. Ta wannan hanyar, saurin bincike ba zai sha wahala ba lokacin da akwai gagarumin tarin bayanai. Ana amfani da ƙa'ida mai sauƙi don suna fihirisa: _. Sabili da haka, domin akwati ya zama duniya, kafin ƙaddamar da daemon kanta, muna maye gurbin sed-th wildcard zuwa sunan muhalli. Ana wuce canjin sunan muhalli ta hanyar masu canjin yanayi. Sauti mai ban dariya.

Hakanan yana da mahimmanci a lura cewa saboda wasu dalilai kasancewar ma'aunin docker ba ya shafar Splunk sunan mai masauki. Har yanzu da taurin kai zai aika da gundumomi tare da id na akwati a cikin filin masaukin. A matsayin mafita, zaku iya hawa / sauransu / Sunan mai masauki daga na'ura mai watsa shiri kuma a farawa yin maye gurbin kama da sunayen fihirisa.

Misali docker-compose.yml

version: '2'
services:
  splunk-forwarder:
    image: "${IMAGE_REPO}/docker-stats-splunk-forwarder:${IMAGE_VERSION}"
    environment:
      SPLUNK_INDEX: ${ENVIRONMENT}
    volumes:
    - /etc/hostname:/etc/hostname:ro
    - /var/log:/var/log
    - /var/run/docker.sock:/var/run/docker.sock:ro

Sakamakon

Haka ne, watakila maganin ba shine manufa ba kuma tabbas ba kowa ba ne ga kowa, tun da akwai da yawa "hardcode". Amma bisa ga shi, kowa zai iya gina nasa hoton kuma ya sanya shi a cikin kayan aikin sa na sirri, idan, kamar yadda ya faru, kuna buƙatar Splunk Forwarder a Docker.

Tunani:

Magani daga labarin
Magani daga outcoldman wanda ya ƙarfafa mu mu sake amfani da wasu ayyukan
Na takardun don kafa Universal Forwarder

source: www.habr.com