🥇Splunk Universal Forwarder-ni Docker-da tizim jurnali kollektori sifatida

Splunk - eng taniqli tijorat jurnallarini yig'ish va tahlil qilish mahsulotlaridan biri. Hozir ham, Rossiyada sotuvlar endi amalga oshirilmagan bo'lsa ham, bu ushbu mahsulot uchun ko'rsatmalar / qanday qilish kerakligini yozmaslik uchun sabab emas.

Maqsad: xost mashinasi konfiguratsiyasini o'zgartirmasdan Splunk-dagi docker tugunlaridan tizim jurnallarini yig'ing

Men rasmiy yondashuvdan boshlamoqchiman, bu Docker-dan foydalanishda biroz g'alati ko'rinadi.
Docker markaziga havola
Bizda nima bor:

1. Pullim tasviri

$ docker pull splunk/universalforwarder:latest

2. Idishni kerakli parametrlar bilan boshlang

$ docker run -d  -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=<password>' splunk/universalforwarder:latest

3. Biz konteynerga kiramiz

docker exec -it <container-id> /bin/bash

Keyinchalik, hujjatlardagi ma'lum manzilga o'tishimiz so'raladi.

Va konteynerni ishga tushirgandan so'ng sozlang:


./splunk add forward-server <host name or ip address>:<listening port>
./splunk add monitor /var/log
./splunk restart

Kutmoq. Nima?

Ammo kutilmagan hodisalar shu bilan tugamaydi. Agar siz konteynerni rasmiy tasvirdan interaktiv rejimda ishga tushirsangiz, quyidagilarni ko'rasiz:

Biroz umidsizlik


$ docker run -it -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=password' splunk/universalforwarder:latest

PLAY [Run default Splunk provisioning] *******************************************************************************************************************************************************************************************************
Tuesday 09 April 2019  13:40:38 +0000 (0:00:00.096)       0:00:00.096 *********

TASK [Gathering Facts] ***********************************************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:39 +0000 (0:00:01.520)       0:00:01.616 *********

TASK [Get actual hostname] *******************************************************************************************************************************************************************************************************************
changed: [localhost]
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.599)       0:00:02.215 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.054)       0:00:02.270 *********

TASK [set_fact] ******************************************************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.075)       0:00:02.346 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.067)       0:00:02.413 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.060)       0:00:02.473 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.051)       0:00:02.525 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.056)       0:00:02.582 *********
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.216)       0:00:02.798 *********
included: /opt/ansible/roles/splunk_common/tasks/change_splunk_directory_owner.yml for localhost
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.087)       0:00:02.886 *********

TASK [splunk_common : Update Splunk directory owner] *****************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.324)       0:00:03.210 *********
included: /opt/ansible/roles/splunk_common/tasks/get_facts.yml for localhost
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.094)       0:00:03.305 *********

ну и так далее...

Ajoyib. Rasmda hatto artefakt ham yo'q. Ya'ni, har safar boshlaganingizda arxivni ikkilik fayllar bilan yuklab olish, ochish va sozlash uchun vaqt kerak bo'ladi.
Docker-way va bularning barchasi haqida nima deyish mumkin?

Yo'q rahmat. Biz boshqa yo'ldan boramiz. Agar biz ushbu operatsiyalarning barchasini montaj bosqichida bajarsak nima bo'ladi? Keyin ketaylik!

Ko'p vaqtni kechiktirmaslik uchun men sizga darhol yakuniy rasmni ko'rsataman:

Docker fayli

# Тут у кого какие предпочтения
FROM centos:7

# Задаём переменные, чтобы каждый раз при старте не указывать их
ENV SPLUNK_HOME /splunkforwarder
ENV SPLUNK_ROLE splunk_heavy_forwarder
ENV SPLUNK_PASSWORD changeme
ENV SPLUNK_START_ARGS --accept-license

# Ставим пакеты
# wget - чтобы скачать артефакты
# expect - понадобится для первоначального запуска Splunk на этапе сборки
# jq - используется в скриптах, которые собирают статистику докера
RUN yum install -y epel-release 
    && yum install -y wget expect jq

# Качаем, распаковываем, удаляем
RUN wget -O splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.2.4&product=universalforwarder&filename=splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz&wget=true' 
    && wget -O docker-18.09.3.tgz 'https://download.docker.com/linux/static/stable/x86_64/docker-18.09.3.tgz' 
    && tar -xvf splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 
    && tar -xvf docker-18.09.3.tgz  
    && rm -f splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 
    && rm -f docker-18.09.3.tgz

# С shell скриптами всё понятно, а вот inputs.conf, splunkclouduf.spl и first_start.sh нуждаются в пояснении. Об этом расскажу после source тэга.
COPY [ "inputs.conf", "docker-stats/props.conf", "/splunkforwarder/etc/system/local/" ]
COPY [ "docker-stats/docker_events.sh", "docker-stats/docker_inspect.sh", "docker-stats/docker_stats.sh", "docker-stats/docker_top.sh", "/splunkforwarder/bin/scripts/" ]
COPY splunkclouduf.spl /splunkclouduf.spl
COPY first_start.sh /splunkforwarder/bin/

#  Даём права на исполнение, добавляем пользователя и выполняем первоначальную настройку
RUN chmod +x /splunkforwarder/bin/scripts/*.sh 
    && groupadd -r splunk 
    && useradd -r -m -g splunk splunk 
    && echo "%sudo ALL=NOPASSWD:ALL" >> /etc/sudoers 
    && chown -R splunk:splunk $SPLUNK_HOME 
    && /splunkforwarder/bin/first_start.sh 
    && /splunkforwarder/bin/splunk install app /splunkclouduf.spl -auth admin:changeme 
    && /splunkforwarder/bin/splunk restart

# Копируем инит скрипты
COPY [ "init/entrypoint.sh", "init/checkstate.sh", "/sbin/" ]

# По желанию. Кому нужно локально иметь конфиги/логи, кому нет.
VOLUME [ "/splunkforwarder/etc", "/splunkforwarder/var" ]

HEALTHCHECK --interval=30s --timeout=30s --start-period=3m --retries=5 CMD /sbin/checkstate.sh || exit 1

ENTRYPOINT [ "/sbin/entrypoint.sh" ]
CMD [ "start-service" ]

Xo'sh, unda nima bor

first_start.sh

#!/usr/bin/expect -f
set timeout -1
spawn /splunkforwarder/bin/splunk start --accept-license
expect "Please enter an administrator username: "
send -- "adminr"
expect "Please enter a new password: "
send -- "changemer"
expect "Please confirm new password: "
send -- "changemer"
expect eof

Birinchi ishga tushirishda Splunk sizdan unga login/parol berishingizni so'raydi, LEKIN bu ma'lumotlar ishlatiladi faqatgina muayyan o'rnatish uchun ma'muriy buyruqlarni bajarish uchun, ya'ni konteyner ichida. Bizning holatda, biz faqat konteynerni ishga tushirmoqchimiz, shunda hamma narsa ishlaydi va loglar daryo kabi oqadi. Albatta, bu qattiq kod, lekin men boshqa yo'llarni topa olmadim.

Keyinchalik skriptga muvofiq amalga oshiriladi

/splunkforwarder/bin/splunk install app /splunkclouduf.spl -auth admin:changeme

splunkclouduf.spl — Bu Splunk Universal Forwarder uchun hisob maʼlumotlari fayli boʻlib, uni veb-interfeysdan yuklab olish mumkin.

Yuklab olish uchun qayerga bosing (rasmlarda)

Bu oddiy arxiv bo'lib, uni ochish mumkin. Ichkarida bizning SplunkCloud va ulanish uchun sertifikatlar va parol mavjud outputs.conf kiritish misollarimiz ro'yxati bilan. Bu fayl Splunk o'rnatishingizni qayta o'rnatmaguningizcha yoki o'rnatish joyida bo'lsa, kiritish tugunini qo'shguningizcha tegishli bo'ladi. Shuning uchun uni idishning ichiga qo'shishning hech qanday yomon joyi yo'q.

Va oxirgi narsa - qayta ishga tushirish. Ha, o'zgarishlarni qo'llash uchun uni qayta ishga tushirishingiz kerak.

Bizning inputs.conf biz Splunk-ga yubormoqchi bo'lgan jurnallarni qo'shamiz. Agar, masalan, qo'g'irchoq orqali konfiguratsiyalarni tarqatsangiz, ushbu faylni rasmga qo'shish shart emas. Yagona narsa shundaki, ekspeditor demon ishga tushganda konfiguratsiyalarni ko'radi, aks holda unga kerak bo'ladi. ./splunk qayta ishga tushirish.

Ular qanday docker stats skriptlari? Github-da eski yechim mavjud outcoldman, skriptlar u yerdan olindi va Docker (ce-17.*) va Splunk (7.*) ning joriy versiyalari bilan ishlash uchun o'zgartirildi.

Olingan ma'lumotlar bilan siz quyidagilarni qurishingiz mumkin

asboblar paneli: (bir nechta rasm)

Chiziqlar uchun manba kodi maqolaning oxirida berilgan havolada. E'tibor bering, ikkita tanlangan maydon mavjud: 2 - indeks tanlash (niqob bo'yicha qidiriladi), xost/konteyner tanlash. Foydalanadigan nomlaringizga qarab indeks niqobini yangilashingiz kerak bo'ladi.

Xulosa qilib, men sizning e'tiboringizni funktsiyaga qaratmoqchiman start() в

entrypoint.sh

start() {
    trap teardown EXIT
	if [ -z $SPLUNK_INDEX ]; then
	echo "'SPLUNK_INDEX' env variable is empty or not defined. Should be 'dev' or 'prd'." >&2
	exit 1
	else
	sed -e "s/@index@/$SPLUNK_INDEX/" -i ${SPLUNK_HOME}/etc/system/local/inputs.conf
	fi
	sed -e "s/@hostname@/$(cat /etc/hostname)/" -i ${SPLUNK_HOME}/etc/system/local/inputs.conf
    sh -c "echo 'starting' > /tmp/splunk-container.state"
	${SPLUNK_HOME}/bin/splunk start
    watch_for_failure
}

Mening holimda, har bir muhit va har bir alohida ob'ekt uchun, xoh u konteynerdagi dastur yoki xoh xost mashinasi bo'lsin, biz alohida indeksdan foydalanamiz. Shunday qilib, ma'lumotlar sezilarli darajada to'planganda qidiruv tezligi pasaymaydi. Indekslarni nomlash uchun oddiy qoidadan foydalaniladi: _. Shuning uchun, konteyner universal bo'lishi uchun, demonning o'zini ishga tushirishdan oldin, biz almashtiramiz sed-atrof-muhit nomining joker belgisi. Atrof-muhit nomi o'zgaruvchisi muhit o'zgaruvchilari orqali uzatiladi. Kulgili eshitiladi.

Shuni ham ta'kidlash kerakki, ba'zi sabablarga ko'ra Splunk docker parametrining mavjudligiga ta'sir qilmaydi. hostname. U baribir o'jarlik bilan o'z konteynerining identifikatori bilan jurnallarni xost maydoniga yuboradi. Yechim sifatida siz o'rnatishingiz mumkin / etc / hostname xost mashinasidan va ishga tushirilganda indeks nomlariga o'xshash o'zgarishlarni amalga oshiring.

Misol docker-compose.yml

version: '2'
services:
  splunk-forwarder:
    image: "${IMAGE_REPO}/docker-stats-splunk-forwarder:${IMAGE_VERSION}"
    environment:
      SPLUNK_INDEX: ${ENVIRONMENT}
    volumes:
    - /etc/hostname:/etc/hostname:ro
    - /var/log:/var/log
    - /var/run/docker.sock:/var/run/docker.sock:ro

Xulosa

Ha, ehtimol, yechim ideal emas va, albatta, hamma uchun universal emas, chunki ko'p narsalar mavjud "qattiq kod". Ammo shunga asoslanib, har kim o'z imidjini yaratishi va uni shaxsiy artifakturasiga qo'yishi mumkin, agar sizga Docker-da Splunk Forwarder kerak bo'lsa.

Manbalar:

Maqoladan yechim
Outcoldmanning ba'zi funksiyalarni qayta ishlatishga ilhomlantirgan yechimi
Of. Universal Forwarderni sozlash uchun hujjatlar

Manba: www.habr.com

Splunk Universal Forwarder docker-da tizim jurnali kollektori sifatida

a Izoh qo'shish Bu savolga javob bekor