Splunk yog ib qho ntawm ob peb ntawm cov khoom lag luam pom zoo tshaj plaws thiab kev tshuaj xyuas cov khoom lag luam. Txawm tias tam sim no, thaum kev muag khoom tsis tau ua nyob rau hauv Russia, qhov no tsis yog vim li cas tsis sau cov lus qhia / yuav ua li cas rau cov khoom no.
Hom phiaj: sau cov ntaub ntawv kaw lus los ntawm docker nodes hauv Splunk yam tsis tau hloov lub tswv yim lub tshuab teeb tsa
Kuv xav pib nrog txoj hauv kev, uas zoo li txawv me ntsis thaum siv Docker.
Peb muaj dab tsi:
1. Pullim duab
$ docker pull splunk/universalforwarder:latest2. Pib lub thawv nrog cov khoom tsim nyog
$ docker run -d -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=<password>' splunk/universalforwarder:latest3. Peb mus rau hauv lub thawv
docker exec -it <container-id> /bin/bashTom ntej no, peb raug hais kom mus rau qhov chaw nyob paub hauv cov ntaub ntawv.
Thiab configure lub thawv tom qab nws pib:
./splunk add forward-server <host name or ip address>:<listening port>
./splunk add monitor /var/log
./splunk restart
Tos. Dab tsi?
Tab sis cov surprises tsis xaus rau ntawd. Yog tias koj khiav lub thawv los ntawm cov duab nom tswv hauv kev sib tham sib, koj yuav pom cov hauv qab no:
Kev poob siab me ntsis
$ docker run -it -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=password' splunk/universalforwarder:latest
PLAY [Run default Splunk provisioning] *******************************************************************************************************************************************************************************************************
Tuesday 09 April 2019 13:40:38 +0000 (0:00:00.096) 0:00:00.096 *********
TASK [Gathering Facts] ***********************************************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019 13:40:39 +0000 (0:00:01.520) 0:00:01.616 *********
TASK [Get actual hostname] *******************************************************************************************************************************************************************************************************************
changed: [localhost]
Tuesday 09 April 2019 13:40:40 +0000 (0:00:00.599) 0:00:02.215 *********
Tuesday 09 April 2019 13:40:40 +0000 (0:00:00.054) 0:00:02.270 *********
TASK [set_fact] ******************************************************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019 13:40:40 +0000 (0:00:00.075) 0:00:02.346 *********
Tuesday 09 April 2019 13:40:40 +0000 (0:00:00.067) 0:00:02.413 *********
Tuesday 09 April 2019 13:40:40 +0000 (0:00:00.060) 0:00:02.473 *********
Tuesday 09 April 2019 13:40:40 +0000 (0:00:00.051) 0:00:02.525 *********
Tuesday 09 April 2019 13:40:40 +0000 (0:00:00.056) 0:00:02.582 *********
Tuesday 09 April 2019 13:40:41 +0000 (0:00:00.216) 0:00:02.798 *********
included: /opt/ansible/roles/splunk_common/tasks/change_splunk_directory_owner.yml for localhost
Tuesday 09 April 2019 13:40:41 +0000 (0:00:00.087) 0:00:02.886 *********
TASK [splunk_common : Update Splunk directory owner] *****************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019 13:40:41 +0000 (0:00:00.324) 0:00:03.210 *********
included: /opt/ansible/roles/splunk_common/tasks/get_facts.yml for localhost
Tuesday 09 April 2019 13:40:41 +0000 (0:00:00.094) 0:00:03.305 *********
ну и так далее...
Zoo heev. Daim duab tsis txawm muaj ib qho khoom qub. Ntawd yog, txhua zaus koj pib nws yuav siv sijhawm los rub tawm cov ntaub ntawv nrog binaries, tshem tawm thiab teeb tsa.
Yuav ua li cas txog docker-txoj kev thiab tag nrho cov ntawd?
Tsis ua tsaug. Peb yuav siv txoj kev sib txawv. Yuav ua li cas yog tias peb ua tag nrho cov haujlwm no nyob rau theem sib dhos? Ces cia mus!
Txhawm rau kom tsis txhob ncua sijhawm ntev, kuv yuav qhia koj cov duab kawg tam sim ntawd:
Dockerfile
# Тут у кого какие предпочтения
FROM centos:7
# Задаём переменные, чтобы каждый раз при старте не указывать их
ENV SPLUNK_HOME /splunkforwarder
ENV SPLUNK_ROLE splunk_heavy_forwarder
ENV SPLUNK_PASSWORD changeme
ENV SPLUNK_START_ARGS --accept-license
# Ставим пакеты
# wget - чтобы скачать артефакты
# expect - понадобится для первоначального запуска Splunk на этапе сборки
# jq - используется в скриптах, которые собирают статистику докера
RUN yum install -y epel-release
&& yum install -y wget expect jq
# Качаем, распаковываем, удаляем
RUN wget -O splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.2.4&product=universalforwarder&filename=splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz&wget=true'
&& wget -O docker-18.09.3.tgz 'https://download.docker.com/linux/static/stable/x86_64/docker-18.09.3.tgz'
&& tar -xvf splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz
&& tar -xvf docker-18.09.3.tgz
&& rm -f splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz
&& rm -f docker-18.09.3.tgz
# С shell скриптами всё понятно, а вот inputs.conf, splunkclouduf.spl и first_start.sh нуждаются в пояснении. Об этом расскажу после source тэга.
COPY [ "inputs.conf", "docker-stats/props.conf", "/splunkforwarder/etc/system/local/" ]
COPY [ "docker-stats/docker_events.sh", "docker-stats/docker_inspect.sh", "docker-stats/docker_stats.sh", "docker-stats/docker_top.sh", "/splunkforwarder/bin/scripts/" ]
COPY splunkclouduf.spl /splunkclouduf.spl
COPY first_start.sh /splunkforwarder/bin/
# Даём права на исполнение, добавляем пользователя и выполняем первоначальную настройку
RUN chmod +x /splunkforwarder/bin/scripts/*.sh
&& groupadd -r splunk
&& useradd -r -m -g splunk splunk
&& echo "%sudo ALL=NOPASSWD:ALL" >> /etc/sudoers
&& chown -R splunk:splunk $SPLUNK_HOME
&& /splunkforwarder/bin/first_start.sh
&& /splunkforwarder/bin/splunk install app /splunkclouduf.spl -auth admin:changeme
&& /splunkforwarder/bin/splunk restart
# Копируем инит скрипты
COPY [ "init/entrypoint.sh", "init/checkstate.sh", "/sbin/" ]
# По желанию. Кому нужно локально иметь конфиги/логи, кому нет.
VOLUME [ "/splunkforwarder/etc", "/splunkforwarder/var" ]
HEALTHCHECK --interval=30s --timeout=30s --start-period=3m --retries=5 CMD /sbin/checkstate.sh || exit 1
ENTRYPOINT [ "/sbin/entrypoint.sh" ]
CMD [ "start-service" ]Yog li dab tsi muaj nyob rau hauv
first_start.sh
#!/usr/bin/expect -f
set timeout -1
spawn /splunkforwarder/bin/splunk start --accept-license
expect "Please enter an administrator username: "
send -- "adminr"
expect "Please enter a new password: "
send -- "changemer"
expect "Please confirm new password: "
send -- "changemer"
expect eofThaum pib thawj zaug, Splunk nug koj kom muab nws tus ID nkag mus / lo lus zais, Tab sis cov ntaub ntawv no tau siv tsuas los ua cov thawj coj hais kom ua rau qhov kev teeb tsa tshwj xeeb, uas yog, hauv lub thawv. Hauv peb qhov xwm txheej, peb tsuas yog xav tso lub thawv kom txhua yam ua haujlwm thiab cov cav ntws zoo li tus dej. Tau kawg, qhov no yog hardcode, tab sis kuv tsis tau pom lwm txoj hauv kev.
Ntxiv mus raws li tsab ntawv raug tua
/splunkforwarder/bin/splunk install app /splunkclouduf.spl -auth admin:changemeib splunkclouduf.spl - Qhov no yog cov ntaub ntawv pov thawj rau Splunk Universal Forwarder, uas tuaj yeem rub tawm los ntawm lub vev xaib interface.
Nyem qhov twg mus download tau (hauv cov duab)
Qhov no yog ib qho archive uas yuav tsum tau unpacked. Sab hauv yog daim ntawv pov thawj thiab tus password rau kev txuas mus rau peb SplunkCloud thiab outputs.conf nrog ib daim ntawv teev npe ntawm peb cov khoom siv. Cov ntaub ntawv no yuav muaj feem cuam tshuam txog thaum koj rov nruab koj qhov kev teeb tsa Splunk lossis ntxiv ib qho kev tawm tswv yim yog tias lub installation nyob ntawm qhov chaw. Yog li ntawd, tsis muaj dab tsi tsis ncaj ncees lawm nrog nws ntxiv rau hauv lub thawv.
Thiab qhov kawg yog rov pib dua. Yog lawm, txhawm rau siv cov kev hloov pauv, koj yuav tsum rov pib dua.
Hauv peb inputs.conf peb ntxiv cov cav uas peb xav xa mus rau Splunk. Nws tsis tas yuav ntxiv cov ntaub ntawv no rau daim duab yog tias, piv txwv li, koj faib cov teeb tsa ntawm tus menyuam roj hmab. Qhov tsuas yog qhov Forwarder pom cov configs thaum lub daemon pib, txwv tsis pub nws yuav xav tau ./splunk rov pib dua.
Dab tsi ntawm docker stats scripts yog lawv? Muaj ib qho kev daws teeb meem qub ntawm Github los ntawm , cov ntawv sau tau raug coj los ntawm qhov ntawd thiab hloov kho ua haujlwm nrog cov qauv tam sim no ntawm Docker (ce-17.*) thiab Splunk (7.*).
Nrog cov ntaub ntawv tau txais, koj tuaj yeem tsim cov hauv qab no
dashboards: (ob peb daim duab)
Lub hauv paus code rau dashes yog nyob rau hauv qhov link muab nyob rau hauv kawg ntawm tsab xov xwm. Thov nco ntsoov tias muaj 2 qhov chaw xaiv: 1 - index xaiv (nrhiav los ntawm daim npog ntsej muag), xaiv tus tswv tsev / thawv. Tej zaum koj yuav xav tau hloov kho daim npog qhov ncauj qhov ntswg, nyob ntawm cov npe koj siv.
Hauv kev xaus, kuv xav kos koj cov xim rau lub luag haujlwm pib() в
entrypoint.sh
start() {
trap teardown EXIT
if [ -z $SPLUNK_INDEX ]; then
echo "'SPLUNK_INDEX' env variable is empty or not defined. Should be 'dev' or 'prd'." >&2
exit 1
else
sed -e "s/@index@/$SPLUNK_INDEX/" -i ${SPLUNK_HOME}/etc/system/local/inputs.conf
fi
sed -e "s/@hostname@/$(cat /etc/hostname)/" -i ${SPLUNK_HOME}/etc/system/local/inputs.conf
sh -c "echo 'starting' > /tmp/splunk-container.state"
${SPLUNK_HOME}/bin/splunk start
watch_for_failure
}Hauv kuv qhov xwm txheej, rau txhua qhov chaw thiab txhua qhov chaw, yog nws daim ntawv thov hauv lub thawv lossis lub tshuab ua haujlwm, peb siv qhov ntsuas cais. Txoj kev no, kev tshawb nrhiav ceev yuav tsis raug kev txom nyem thaum muaj cov ntaub ntawv tseem ceeb. Ib txoj cai yooj yim yog siv rau npe indexes: _. Yog li ntawd, txhawm rau kom lub thawv ntim thoob ntiaj teb, ua ntej tso lub daemon nws tus kheej, peb hloov sed-th wildcard rau lub npe ntawm ib puag ncig. Ib puag ncig lub npe hloov pauv tau dhau los ntawm kev hloov pauv ib puag ncig. Suab funny.
Nws tseem tsim nyog sau cia tias vim qee yam Splunk tsis cuam tshuam los ntawm qhov muaj qhov ntsuas qhov ntsuas hostname. Nws tseem yuav tawv ncauj xa cov cav nrog tus id ntawm nws lub thawv rau hauv tus tswv teb. Raws li kev daws teeb meem, koj tuaj yeem mount / etc / hostname los ntawm tus tswv tshuab thiab thaum pib ua kev hloov pauv zoo ib yam li cov npe index.
Piv txwv li docker-compose.yml
version: '2'
services:
splunk-forwarder:
image: "${IMAGE_REPO}/docker-stats-splunk-forwarder:${IMAGE_VERSION}"
environment:
SPLUNK_INDEX: ${ENVIRONMENT}
volumes:
- /etc/hostname:/etc/hostname:ro
- /var/log:/var/log
- /var/run/docker.sock:/var/run/docker.sock:roQhov no
Yog, tej zaum qhov kev daws teeb meem tsis yog qhov zoo tagnrho thiab yeej tsis yog universal rau txhua tus, vim muaj ntau yam "hardcode". Tab sis raws li nws, txhua tus tuaj yeem tsim lawv tus kheej cov duab thiab muab tso rau hauv lawv tus kheej artifactory, yog tias nws tshwm sim, koj xav tau Splunk Forwarder hauv Docker.
Links:
Tau qhov twg los: www.hab.com
