🥇Splunk Universal Forwarder hauv Docker ua tus sau lub kaw lus

Splunk yog ib qho ntawm ob peb ntawm cov khoom lag luam pom zoo tshaj plaws thiab kev tshuaj xyuas cov khoom lag luam. Txawm tias tam sim no, thaum kev muag khoom tsis tau ua nyob rau hauv Russia, qhov no tsis yog vim li cas tsis sau cov lus qhia / yuav ua li cas rau cov khoom no.

Hom phiaj: sau cov ntaub ntawv kaw lus los ntawm docker nodes hauv Splunk yam tsis tau hloov lub tswv yim lub tshuab teeb tsa

Kuv xav pib nrog txoj hauv kev, uas zoo li txawv me ntsis thaum siv Docker.
Txuas rau Docker hub
Peb muaj dab tsi:

1. Pullim duab

$ docker pull splunk/universalforwarder:latest

2. Pib lub thawv nrog cov khoom tsim nyog

$ docker run -d  -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=<password>' splunk/universalforwarder:latest

3. Peb mus rau hauv lub thawv

docker exec -it <container-id> /bin/bash

Tom ntej no, peb raug hais kom mus rau qhov chaw nyob paub hauv cov ntaub ntawv.

Thiab configure lub thawv tom qab nws pib:


./splunk add forward-server <host name or ip address>:<listening port>
./splunk add monitor /var/log
./splunk restart

Tos. Dab tsi?

Tab sis cov surprises tsis xaus rau ntawd. Yog tias koj khiav lub thawv los ntawm cov duab nom tswv hauv kev sib tham sib, koj yuav pom cov hauv qab no:

Kev poob siab me ntsis


$ docker run -it -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=password' splunk/universalforwarder:latest

PLAY [Run default Splunk provisioning] *******************************************************************************************************************************************************************************************************
Tuesday 09 April 2019  13:40:38 +0000 (0:00:00.096)       0:00:00.096 *********

TASK [Gathering Facts] ***********************************************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:39 +0000 (0:00:01.520)       0:00:01.616 *********

TASK [Get actual hostname] *******************************************************************************************************************************************************************************************************************
changed: [localhost]
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.599)       0:00:02.215 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.054)       0:00:02.270 *********

TASK [set_fact] ******************************************************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.075)       0:00:02.346 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.067)       0:00:02.413 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.060)       0:00:02.473 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.051)       0:00:02.525 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.056)       0:00:02.582 *********
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.216)       0:00:02.798 *********
included: /opt/ansible/roles/splunk_common/tasks/change_splunk_directory_owner.yml for localhost
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.087)       0:00:02.886 *********

TASK [splunk_common : Update Splunk directory owner] *****************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.324)       0:00:03.210 *********
included: /opt/ansible/roles/splunk_common/tasks/get_facts.yml for localhost
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.094)       0:00:03.305 *********

ну и так далее...

Zoo heev. Daim duab tsis txawm muaj ib qho khoom qub. Ntawd yog, txhua zaus koj pib nws yuav siv sijhawm los rub tawm cov ntaub ntawv nrog binaries, tshem tawm thiab teeb tsa.
Yuav ua li cas txog docker-txoj kev thiab tag nrho cov ntawd?

Tsis ua tsaug. Peb yuav siv txoj kev sib txawv. Yuav ua li cas yog tias peb ua tag nrho cov haujlwm no nyob rau theem sib dhos? Ces cia mus!

Txhawm rau kom tsis txhob ncua sijhawm ntev, kuv yuav qhia koj cov duab kawg tam sim ntawd:

Dockerfile

# Тут у кого какие предпочтения
FROM centos:7

# Задаём переменные, чтобы каждый раз при старте не указывать их
ENV SPLUNK_HOME /splunkforwarder
ENV SPLUNK_ROLE splunk_heavy_forwarder
ENV SPLUNK_PASSWORD changeme
ENV SPLUNK_START_ARGS --accept-license

# Ставим пакеты
# wget - чтобы скачать артефакты
# expect - понадобится для первоначального запуска Splunk на этапе сборки
# jq - используется в скриптах, которые собирают статистику докера
RUN yum install -y epel-release 
    && yum install -y wget expect jq

# Качаем, распаковываем, удаляем
RUN wget -O splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.2.4&product=universalforwarder&filename=splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz&wget=true' 
    && wget -O docker-18.09.3.tgz 'https://download.docker.com/linux/static/stable/x86_64/docker-18.09.3.tgz' 
    && tar -xvf splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 
    && tar -xvf docker-18.09.3.tgz  
    && rm -f splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 
    && rm -f docker-18.09.3.tgz

# С shell скриптами всё понятно, а вот inputs.conf, splunkclouduf.spl и first_start.sh нуждаются в пояснении. Об этом расскажу после source тэга.
COPY [ "inputs.conf", "docker-stats/props.conf", "/splunkforwarder/etc/system/local/" ]
COPY [ "docker-stats/docker_events.sh", "docker-stats/docker_inspect.sh", "docker-stats/docker_stats.sh", "docker-stats/docker_top.sh", "/splunkforwarder/bin/scripts/" ]
COPY splunkclouduf.spl /splunkclouduf.spl
COPY first_start.sh /splunkforwarder/bin/

#  Даём права на исполнение, добавляем пользователя и выполняем первоначальную настройку
RUN chmod +x /splunkforwarder/bin/scripts/*.sh 
    && groupadd -r splunk 
    && useradd -r -m -g splunk splunk 
    && echo "%sudo ALL=NOPASSWD:ALL" >> /etc/sudoers 
    && chown -R splunk:splunk $SPLUNK_HOME 
    && /splunkforwarder/bin/first_start.sh 
    && /splunkforwarder/bin/splunk install app /splunkclouduf.spl -auth admin:changeme 
    && /splunkforwarder/bin/splunk restart

# Копируем инит скрипты
COPY [ "init/entrypoint.sh", "init/checkstate.sh", "/sbin/" ]

# По желанию. Кому нужно локально иметь конфиги/логи, кому нет.
VOLUME [ "/splunkforwarder/etc", "/splunkforwarder/var" ]

HEALTHCHECK --interval=30s --timeout=30s --start-period=3m --retries=5 CMD /sbin/checkstate.sh || exit 1

ENTRYPOINT [ "/sbin/entrypoint.sh" ]
CMD [ "start-service" ]

Yog li dab tsi muaj nyob rau hauv

first_start.sh

#!/usr/bin/expect -f
set timeout -1
spawn /splunkforwarder/bin/splunk start --accept-license
expect "Please enter an administrator username: "
send -- "adminr"
expect "Please enter a new password: "
send -- "changemer"
expect "Please confirm new password: "
send -- "changemer"
expect eof

Thaum pib thawj zaug, Splunk nug koj kom muab nws tus ID nkag mus / lo lus zais, Tab sis cov ntaub ntawv no tau siv tsuas los ua cov thawj coj hais kom ua rau qhov kev teeb tsa tshwj xeeb, uas yog, hauv lub thawv. Hauv peb qhov xwm txheej, peb tsuas yog xav tso lub thawv kom txhua yam ua haujlwm thiab cov cav ntws zoo li tus dej. Tau kawg, qhov no yog hardcode, tab sis kuv tsis tau pom lwm txoj hauv kev.

Ntxiv mus raws li tsab ntawv raug tua

/splunkforwarder/bin/splunk install app /splunkclouduf.spl -auth admin:changeme

ib splunkclouduf.spl - Qhov no yog cov ntaub ntawv pov thawj rau Splunk Universal Forwarder, uas tuaj yeem rub tawm los ntawm lub vev xaib interface.

Nyem qhov twg mus download tau (hauv cov duab)

Qhov no yog ib qho archive uas yuav tsum tau unpacked. Sab hauv yog daim ntawv pov thawj thiab tus password rau kev txuas mus rau peb SplunkCloud thiab outputs.conf nrog ib daim ntawv teev npe ntawm peb cov khoom siv. Cov ntaub ntawv no yuav muaj feem cuam tshuam txog thaum koj rov nruab koj qhov kev teeb tsa Splunk lossis ntxiv ib qho kev tawm tswv yim yog tias lub installation nyob ntawm qhov chaw. Yog li ntawd, tsis muaj dab tsi tsis ncaj ncees lawm nrog nws ntxiv rau hauv lub thawv.

Thiab qhov kawg yog rov pib dua. Yog lawm, txhawm rau siv cov kev hloov pauv, koj yuav tsum rov pib dua.

Hauv peb inputs.conf peb ntxiv cov cav uas peb xav xa mus rau Splunk. Nws tsis tas yuav ntxiv cov ntaub ntawv no rau daim duab yog tias, piv txwv li, koj faib cov teeb tsa ntawm tus menyuam roj hmab. Qhov tsuas yog qhov Forwarder pom cov configs thaum lub daemon pib, txwv tsis pub nws yuav xav tau ./splunk rov pib dua.

Dab tsi ntawm docker stats scripts yog lawv? Muaj ib qho kev daws teeb meem qub ntawm Github los ntawm tus neeg siab tawv, cov ntawv sau tau raug coj los ntawm qhov ntawd thiab hloov kho ua haujlwm nrog cov qauv tam sim no ntawm Docker (ce-17.*) thiab Splunk (7.*).

Nrog cov ntaub ntawv tau txais, koj tuaj yeem tsim cov hauv qab no

dashboards: (ob peb daim duab)

Lub hauv paus code rau dashes yog nyob rau hauv qhov link muab nyob rau hauv kawg ntawm tsab xov xwm. Thov nco ntsoov tias muaj 2 qhov chaw xaiv: 1 - index xaiv (nrhiav los ntawm daim npog ntsej muag), xaiv tus tswv tsev / thawv. Tej zaum koj yuav xav tau hloov kho daim npog qhov ncauj qhov ntswg, nyob ntawm cov npe koj siv.

Hauv kev xaus, kuv xav kos koj cov xim rau lub luag haujlwm pib() в

entrypoint.sh

start() {
    trap teardown EXIT
	if [ -z $SPLUNK_INDEX ]; then
	echo "'SPLUNK_INDEX' env variable is empty or not defined. Should be 'dev' or 'prd'." >&2
	exit 1
	else
	sed -e "s/@index@/$SPLUNK_INDEX/" -i ${SPLUNK_HOME}/etc/system/local/inputs.conf
	fi
	sed -e "s/@hostname@/$(cat /etc/hostname)/" -i ${SPLUNK_HOME}/etc/system/local/inputs.conf
    sh -c "echo 'starting' > /tmp/splunk-container.state"
	${SPLUNK_HOME}/bin/splunk start
    watch_for_failure
}

Hauv kuv qhov xwm txheej, rau txhua qhov chaw thiab txhua qhov chaw, yog nws daim ntawv thov hauv lub thawv lossis lub tshuab ua haujlwm, peb siv qhov ntsuas cais. Txoj kev no, kev tshawb nrhiav ceev yuav tsis raug kev txom nyem thaum muaj cov ntaub ntawv tseem ceeb. Ib txoj cai yooj yim yog siv rau npe indexes: _. Yog li ntawd, txhawm rau kom lub thawv ntim thoob ntiaj teb, ua ntej tso lub daemon nws tus kheej, peb hloov sed-th wildcard rau lub npe ntawm ib puag ncig. Ib puag ncig lub npe hloov pauv tau dhau los ntawm kev hloov pauv ib puag ncig. Suab funny.

Nws tseem tsim nyog sau cia tias vim qee yam Splunk tsis cuam tshuam los ntawm qhov muaj qhov ntsuas qhov ntsuas hostname. Nws tseem yuav tawv ncauj xa cov cav nrog tus id ntawm nws lub thawv rau hauv tus tswv teb. Raws li kev daws teeb meem, koj tuaj yeem mount / etc / hostname los ntawm tus tswv tshuab thiab thaum pib ua kev hloov pauv zoo ib yam li cov npe index.

Piv txwv li docker-compose.yml

version: '2'
services:
  splunk-forwarder:
    image: "${IMAGE_REPO}/docker-stats-splunk-forwarder:${IMAGE_VERSION}"
    environment:
      SPLUNK_INDEX: ${ENVIRONMENT}
    volumes:
    - /etc/hostname:/etc/hostname:ro
    - /var/log:/var/log
    - /var/run/docker.sock:/var/run/docker.sock:ro

Qhov no

Yog, tej zaum qhov kev daws teeb meem tsis yog qhov zoo tagnrho thiab yeej tsis yog universal rau txhua tus, vim muaj ntau yam "hardcode". Tab sis raws li nws, txhua tus tuaj yeem tsim lawv tus kheej cov duab thiab muab tso rau hauv lawv tus kheej artifactory, yog tias nws tshwm sim, koj xav tau Splunk Forwarder hauv Docker.

Links:

Kev daws los ntawm kab lus
Ib qho kev daws teeb meem los ntawm outcoldman uas tau tshoov peb kom rov siv qee qhov kev ua haujlwm
Ntawm. Cov ntaub ntawv rau kev teeb tsa Universal Forwarder

Tau qhov twg los: www.hab.com

Splunk Universal Forwarder nyob rau hauv Docker raws li lub kaw lus sau

Ntxiv ib saib Отменить ответ