🥇Splunk Universal Forwarder muDocker seSystem Log Collector

Splunk ndeimwe yeakawanda anozivikanwa ekutengesa log muunganidzwa uye zvigadzirwa zvekuongorora. Kunyange ikozvino, kana kutengesa kusingachaiti muRussia, ichi hachisi chikonzero chekunyora mirairo / sei-yechigadzirwa ichi.

Basa: unganidza matanda ehurongwa kubva docker node muSplunk pasina kushandura gadziriso yemuchina

Ndinoda kutanga nemaitiro epamutemo, ayo anotaridzika zvishoma kana uchishandisa Docker.
Batanidza kuDocker hub
Chii chatinacho:

1. Mufananidzo wePullim

$ docker pull splunk/universalforwarder:latest

2. Tanga mudziyo nematanho anodiwa

$ docker run -d  -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=<password>' splunk/universalforwarder:latest

3. Tinopinda mumudziyo

docker exec -it <container-id> /bin/bash

Tevere, tinokumbirwa kuenda kukero inozivikanwa mune zvinyorwa.

Uye gadzirisa mudziyo mushure mekunge watanga:


./splunk add forward-server <host name or ip address>:<listening port>
./splunk add monitor /var/log
./splunk restart

Mira. Chii?

Asi mashura acho haagumiri ipapo. Kana iwe ukamhanyisa mudziyo kubva pamufananidzo wepamutemo mune inopindirana modhi, iwe uchaona zvinotevera:

Kuodzwa mwoyo zvishoma


$ docker run -it -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=password' splunk/universalforwarder:latest

PLAY [Run default Splunk provisioning] *******************************************************************************************************************************************************************************************************
Tuesday 09 April 2019  13:40:38 +0000 (0:00:00.096)       0:00:00.096 *********

TASK [Gathering Facts] ***********************************************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:39 +0000 (0:00:01.520)       0:00:01.616 *********

TASK [Get actual hostname] *******************************************************************************************************************************************************************************************************************
changed: [localhost]
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.599)       0:00:02.215 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.054)       0:00:02.270 *********

TASK [set_fact] ******************************************************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.075)       0:00:02.346 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.067)       0:00:02.413 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.060)       0:00:02.473 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.051)       0:00:02.525 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.056)       0:00:02.582 *********
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.216)       0:00:02.798 *********
included: /opt/ansible/roles/splunk_common/tasks/change_splunk_directory_owner.yml for localhost
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.087)       0:00:02.886 *********

TASK [splunk_common : Update Splunk directory owner] *****************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.324)       0:00:03.210 *********
included: /opt/ansible/roles/splunk_common/tasks/get_facts.yml for localhost
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.094)       0:00:03.305 *********

ну и так далее...

Hukuru. Mufananidzo wacho hauna kana chigadzirwa. Ndokunge, pese paunotanga zvinotora nguva kurodha archive nemabhinari, unpack uye gadzirisa.
Zvakadini nezve docker-nzira uye zvese izvo?

Kwete ndatenda. Tichatora imwe nzira. Zvakadini kana tikaita mabasa ose aya pagungano? Ipapo ngatiende!

Kuti usanonoke nguva refu, ini ndinokuratidza mufananidzo wekupedzisira ipapo:

dockerfile

# Тут у кого какие предпочтения
FROM centos:7

# Задаём переменные, чтобы каждый раз при старте не указывать их
ENV SPLUNK_HOME /splunkforwarder
ENV SPLUNK_ROLE splunk_heavy_forwarder
ENV SPLUNK_PASSWORD changeme
ENV SPLUNK_START_ARGS --accept-license

# Ставим пакеты
# wget - чтобы скачать артефакты
# expect - понадобится для первоначального запуска Splunk на этапе сборки
# jq - используется в скриптах, которые собирают статистику докера
RUN yum install -y epel-release 
    && yum install -y wget expect jq

# Качаем, распаковываем, удаляем
RUN wget -O splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.2.4&product=universalforwarder&filename=splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz&wget=true' 
    && wget -O docker-18.09.3.tgz 'https://download.docker.com/linux/static/stable/x86_64/docker-18.09.3.tgz' 
    && tar -xvf splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 
    && tar -xvf docker-18.09.3.tgz  
    && rm -f splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 
    && rm -f docker-18.09.3.tgz

# С shell скриптами всё понятно, а вот inputs.conf, splunkclouduf.spl и first_start.sh нуждаются в пояснении. Об этом расскажу после source тэга.
COPY [ "inputs.conf", "docker-stats/props.conf", "/splunkforwarder/etc/system/local/" ]
COPY [ "docker-stats/docker_events.sh", "docker-stats/docker_inspect.sh", "docker-stats/docker_stats.sh", "docker-stats/docker_top.sh", "/splunkforwarder/bin/scripts/" ]
COPY splunkclouduf.spl /splunkclouduf.spl
COPY first_start.sh /splunkforwarder/bin/

#  Даём права на исполнение, добавляем пользователя и выполняем первоначальную настройку
RUN chmod +x /splunkforwarder/bin/scripts/*.sh 
    && groupadd -r splunk 
    && useradd -r -m -g splunk splunk 
    && echo "%sudo ALL=NOPASSWD:ALL" >> /etc/sudoers 
    && chown -R splunk:splunk $SPLUNK_HOME 
    && /splunkforwarder/bin/first_start.sh 
    && /splunkforwarder/bin/splunk install app /splunkclouduf.spl -auth admin:changeme 
    && /splunkforwarder/bin/splunk restart

# Копируем инит скрипты
COPY [ "init/entrypoint.sh", "init/checkstate.sh", "/sbin/" ]

# По желанию. Кому нужно локально иметь конфиги/логи, кому нет.
VOLUME [ "/splunkforwarder/etc", "/splunkforwarder/var" ]

HEALTHCHECK --interval=30s --timeout=30s --start-period=3m --retries=5 CMD /sbin/checkstate.sh || exit 1

ENTRYPOINT [ "/sbin/entrypoint.sh" ]
CMD [ "start-service" ]

Saka izvo zviri mukati

first_start.sh

#!/usr/bin/expect -f
set timeout -1
spawn /splunkforwarder/bin/splunk start --accept-license
expect "Please enter an administrator username: "
send -- "adminr"
expect "Please enter a new password: "
send -- "changemer"
expect "Please confirm new password: "
send -- "changemer"
expect eof

Pakutanga kwekutanga, Splunk anokukumbira kuti uipe yekupinda/password, ASI iyi data inoshandiswa chete kuita mirairo yekutonga kune iyo chaiyo yekuisa, kureva, mukati memudziyo. Kwatiri isu tinongoda kuvhura mudziyo kuti zvese zvishande uye matanda ayerera serwizi. Ehe, iyi hardcode, asi ini handisati ndawana chero dzimwe nzira.

Uyezve maererano ne script inoitwa

/splunkforwarder/bin/splunk install app /splunkclouduf.spl -auth admin:changeme

splunkclouduf.spl -Iri faira rezvitupa reSplunk Universal Forwarder, rinogona kutorwa kubva pawebhu interface.

Kwekudzvanya kudhawunirodha (mumifananidzo)

Iyi idura renguva dzose rinogona kuburitswa. Mukati mune zvitupa uye password yekubatanidza kune yedu SplunkCloud uye outputs.conf nerunyorwa rwezviitiko zvedu zvekupinza. Iyi faira ichave yakakosha kudzamara wadzosera yako Splunk kuisirwa kana kuwedzera inopinza node kana kuisirwa kuri pa-nzvimbo. Naizvozvo, hapana chakaipa nekuwedzera mukati memudziyo.

Uye chinhu chekupedzisira kutangazve. Hongu, kushandisa shanduko, unofanirwa kuitangazve.

Mune yedu inputs.conf tinowedzera matanda atinoda kutumira kuna Splunk. Hazvina kudikanwa kuwedzera iyi faira kumufananidzo kana, semuenzaniso, iwe uchigovera configs kuburikidza nepuppet. Chinhu chega ndechekuti Forwarder anoona magadzirirwo kana daemon yatanga, zvikasadaro ichada ./splunk restart.

Ndeapi mhando yezvinyorwa zvedocker stats? Pane mhinduro yekare paGithub kubva outcoldman, zvinyorwa zvakatorwa kubva ipapo uye zvakagadziridzwa kushanda neshanduro dzemazuva ano dzeDocker (ce-17.*) uye Splunk (7.*).

Ne data rakawanikwa, unogona kugadzira zvinotevera

dashboards: (mifananidzo miviri)

Iro kodhi yemadheshi iri mune chinongedzo chakapihwa pakupera kwechinyorwa. Ndapota cherechedza kuti kune 2 sarudza minda: 1 - index selection (yakatsvaga nemask), host/container kusarudzwa. Iwe ungangoda kugadzirisa index mask, zvichienderana nemazita aunoshandisa.

Mukupedzisa, ndinoda kukwevera pfungwa dzako kune basa kutanga () в

entrypoint.sh

start() {
    trap teardown EXIT
	if [ -z $SPLUNK_INDEX ]; then
	echo "'SPLUNK_INDEX' env variable is empty or not defined. Should be 'dev' or 'prd'." >&2
	exit 1
	else
	sed -e "s/@index@/$SPLUNK_INDEX/" -i ${SPLUNK_HOME}/etc/system/local/inputs.conf
	fi
	sed -e "s/@hostname@/$(cat /etc/hostname)/" -i ${SPLUNK_HOME}/etc/system/local/inputs.conf
    sh -c "echo 'starting' > /tmp/splunk-container.state"
	${SPLUNK_HOME}/bin/splunk start
    watch_for_failure
}

Mune yangu, kune imwe neimwe nharaunda uye yega yega mubatanidzwa, ingave application mumudziyo kana muchina wekutambira, isu tinoshandisa yakaparadzana index. Nenzira iyi, iyo yekumhanyisa yekutsvaga haizotambura kana paine kuunganidzwa kwakakosha kwedata. Mutemo wakapfava unoshandiswa kudoma ma indexes: _. Naizvozvo, kuitira kuti mudziyo uve wepasirese, tisati tatanga iyo daemon pachayo, tinoitsiva sed-th wildcard kune zita rezvakatipoteredza. Zita remamiriro ekunze rinopfuudzwa kuburikidza nemamiriro ekunze. Zvinonzwika zvinosetsa.

Izvo zvakakoshawo kuziva kuti nekuda kwechimwe chikonzero Splunk haina kukanganiswa nekuvapo kweiyo docker parameter hostname. Acharamba achiomesa musoro achitumira matanda neid yemudziyo wake mumunda wemauto. Semhinduro, unogona kukwira / etc / hostname kubva pamushini wekutambira uye pakutanga ita zvinotsiva zvakafanana nemazita e index.

Muenzaniso docker-compose.yml

version: '2'
services:
  splunk-forwarder:
    image: "${IMAGE_REPO}/docker-stats-splunk-forwarder:${IMAGE_VERSION}"
    environment:
      SPLUNK_INDEX: ${ENVIRONMENT}
    volumes:
    - /etc/hostname:/etc/hostname:ro
    - /var/log:/var/log
    - /var/run/docker.sock:/var/run/docker.sock:ro

Mugumisiro

Hongu, zvichida mhinduro haina kunaka uye zvechokwadi haisi yepasi rose kune wese, sezvo kune akawanda "hardcode". Asi zvichibva pazviri, munhu wese anogona kuvaka chifananidzo chavo uye nekuchiisa mune yavo yakavanzika artifactory, kana, sezvazvinoitika, iwe unoda Splunk Forwarder muDocker.

Mareferensi:

Mhinduro kubva kuchinyorwa
Mhinduro kubva kune outcoldman iyo yakatikurudzira kuti tishandise zvakare mamwe emashandiro
Of. zvinyorwa zvekugadzirisa Universal Forwarder

Source: www.habr.com

Splunk Universal Forwarder mu docker senge system yekuunganidza