Pulogalamu ya ProHoster > Blog > Ulamuliro > Splunk Universal Forwarder mu docker ngati wosonkhanitsa logi

Splunk Universal Forwarder mu docker ngati wosonkhanitsa logi

Splunk Universal Forwarder mu docker ngati wosonkhanitsa logi

Splunk ndi imodzi mwazinthu zodziwika bwino zosonkhanitsira zolemba zamalonda ndi kusanthula. Ngakhale tsopano, pamene malonda sakupangidwanso ku Russia, ichi si chifukwa cholembera malangizo / momwe mungapangire mankhwalawa.

Cholinga: sonkhanitsani zipika zamakina kuchokera ku ma docker mu Splunk osasintha makina opangira

Ndikufuna kuyamba ndi njira yovomerezeka, yomwe imawoneka yachilendo mukamagwiritsa ntchito Docker.
Lumikizani ku Docker hub
Tili ndi chiyani:

1. Chithunzi cha Pullim

$ docker pull splunk/universalforwarder:latest

2. Yambani chidebe ndi magawo ofunikira

$ docker run -d  -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=<password>' splunk/universalforwarder:latest

3. Timalowa mu chidebe

docker exec -it <container-id> /bin/bash

Kenako, tikufunsidwa kupita ku adilesi yodziwika muzolemba.

Ndipo konzekerani chidebecho chikayamba:


./splunk add forward-server <host name or ip address>:<listening port>
./splunk add monitor /var/log
./splunk restart

Dikirani. Chani?

Koma zodabwitsazo sizimathera pamenepo. Ngati mutayendetsa chidebe kuchokera pa chithunzi chovomerezeka mumayendedwe ochezera, muwona zotsatirazi:

Kukhumudwa pang'ono


$ docker run -it -p 9997:9997 -e 'SPLUNK_START_ARGS=--accept-license' -e 'SPLUNK_PASSWORD=password' splunk/universalforwarder:latest

PLAY [Run default Splunk provisioning] *******************************************************************************************************************************************************************************************************
Tuesday 09 April 2019  13:40:38 +0000 (0:00:00.096)       0:00:00.096 *********

TASK [Gathering Facts] ***********************************************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:39 +0000 (0:00:01.520)       0:00:01.616 *********

TASK [Get actual hostname] *******************************************************************************************************************************************************************************************************************
changed: [localhost]
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.599)       0:00:02.215 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.054)       0:00:02.270 *********

TASK [set_fact] ******************************************************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.075)       0:00:02.346 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.067)       0:00:02.413 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.060)       0:00:02.473 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.051)       0:00:02.525 *********
Tuesday 09 April 2019  13:40:40 +0000 (0:00:00.056)       0:00:02.582 *********
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.216)       0:00:02.798 *********
included: /opt/ansible/roles/splunk_common/tasks/change_splunk_directory_owner.yml for localhost
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.087)       0:00:02.886 *********

TASK [splunk_common : Update Splunk directory owner] *****************************************************************************************************************************************************************************************
ok: [localhost]
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.324)       0:00:03.210 *********
included: /opt/ansible/roles/splunk_common/tasks/get_facts.yml for localhost
Tuesday 09 April 2019  13:40:41 +0000 (0:00:00.094)       0:00:03.305 *********

Π½Ρƒ ΠΈ Ρ‚Π°ΠΊ Π΄Π°Π»Π΅Π΅...

Zabwino. Chithunzichi chilibe chilichonse. Ndiye kuti, nthawi iliyonse mukayamba zimatenga nthawi kutsitsa zosungidwazo ndi ma binaries, kumasula ndikusintha.
Nanga bwanji za docker-way ndi zonsezo?

Ayi zikomo. Titenga njira ina. Nanga bwanji ngati titachita zonsezi panthawi ya msonkhano? Ndiye tiyeni!

Kuti ndisachedwe motalika, ndikuwonetsani chithunzi chomaliza nthawi yomweyo:

Dockerfile

# Π’ΡƒΡ‚ Ρƒ ΠΊΠΎΠ³ΠΎ ΠΊΠ°ΠΊΠΈΠ΅ прСдпочтСния
FROM centos:7

# Π—Π°Π΄Π°Ρ‘ΠΌ ΠΏΠ΅Ρ€Π΅ΠΌΠ΅Π½Π½Ρ‹Π΅, Ρ‡Ρ‚ΠΎΠ±Ρ‹ ΠΊΠ°ΠΆΠ΄Ρ‹ΠΉ Ρ€Π°Π· ΠΏΡ€ΠΈ стартС Π½Π΅ ΡƒΠΊΠ°Π·Ρ‹Π²Π°Ρ‚ΡŒ ΠΈΡ…
ENV SPLUNK_HOME /splunkforwarder
ENV SPLUNK_ROLE splunk_heavy_forwarder
ENV SPLUNK_PASSWORD changeme
ENV SPLUNK_START_ARGS --accept-license

# Π‘Ρ‚Π°Π²ΠΈΠΌ ΠΏΠ°ΠΊΠ΅Ρ‚Ρ‹
# wget - Ρ‡Ρ‚ΠΎΠ±Ρ‹ ΡΠΊΠ°Ρ‡Π°Ρ‚ΡŒ Π°Ρ€Ρ‚Π΅Ρ„Π°ΠΊΡ‚Ρ‹
# expect - понадобится для ΠΏΠ΅Ρ€Π²ΠΎΠ½Π°Ρ‡Π°Π»ΡŒΠ½ΠΎΠ³ΠΎ запуска Splunk Π½Π° этапС сборки
# jq - ΠΈΡΠΏΠΎΠ»ΡŒΠ·ΡƒΠ΅Ρ‚ΡΡ Π² скриптах, ΠΊΠΎΡ‚ΠΎΡ€Ρ‹Π΅ ΡΠΎΠ±ΠΈΡ€Π°ΡŽΡ‚ статистику Π΄ΠΎΠΊΠ΅Ρ€Π°
RUN yum install -y epel-release 
    && yum install -y wget expect jq

# ΠšΠ°Ρ‡Π°Π΅ΠΌ, распаковываСм, удаляСм
RUN wget -O splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.2.4&product=universalforwarder&filename=splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz&wget=true' 
    && wget -O docker-18.09.3.tgz 'https://download.docker.com/linux/static/stable/x86_64/docker-18.09.3.tgz' 
    && tar -xvf splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 
    && tar -xvf docker-18.09.3.tgz  
    && rm -f splunkforwarder-7.2.4-8a94541dcfac-Linux-x86_64.tgz 
    && rm -f docker-18.09.3.tgz

# Π‘ shell скриптами всё понятно, Π° Π²ΠΎΡ‚ inputs.conf, splunkclouduf.spl ΠΈ first_start.sh Π½ΡƒΠΆΠ΄Π°ΡŽΡ‚ΡΡ Π² пояснСнии. Об этом расскаТу послС source тэга.
COPY [ "inputs.conf", "docker-stats/props.conf", "/splunkforwarder/etc/system/local/" ]
COPY [ "docker-stats/docker_events.sh", "docker-stats/docker_inspect.sh", "docker-stats/docker_stats.sh", "docker-stats/docker_top.sh", "/splunkforwarder/bin/scripts/" ]
COPY splunkclouduf.spl /splunkclouduf.spl
COPY first_start.sh /splunkforwarder/bin/

#  Π”Π°Ρ‘ΠΌ ΠΏΡ€Π°Π²Π° Π½Π° исполнСниС, добавляСм ΠΏΠΎΠ»ΡŒΠ·ΠΎΠ²Π°Ρ‚Π΅Π»Ρ ΠΈ выполняСм ΠΏΠ΅Ρ€Π²ΠΎΠ½Π°Ρ‡Π°Π»ΡŒΠ½ΡƒΡŽ настройку
RUN chmod +x /splunkforwarder/bin/scripts/*.sh 
    && groupadd -r splunk 
    && useradd -r -m -g splunk splunk 
    && echo "%sudo ALL=NOPASSWD:ALL" >> /etc/sudoers 
    && chown -R splunk:splunk $SPLUNK_HOME 
    && /splunkforwarder/bin/first_start.sh 
    && /splunkforwarder/bin/splunk install app /splunkclouduf.spl -auth admin:changeme 
    && /splunkforwarder/bin/splunk restart

# ΠšΠΎΠΏΠΈΡ€ΡƒΠ΅ΠΌ ΠΈΠ½ΠΈΡ‚ скрипты
COPY [ "init/entrypoint.sh", "init/checkstate.sh", "/sbin/" ]

# По ТСланию. ΠšΠΎΠΌΡƒ Π½ΡƒΠΆΠ½ΠΎ локально ΠΈΠΌΠ΅Ρ‚ΡŒ ΠΊΠΎΠ½Ρ„ΠΈΠ³ΠΈ/Π»ΠΎΠ³ΠΈ, ΠΊΠΎΠΌΡƒ Π½Π΅Ρ‚.
VOLUME [ "/splunkforwarder/etc", "/splunkforwarder/var" ]

HEALTHCHECK --interval=30s --timeout=30s --start-period=3m --retries=5 CMD /sbin/checkstate.sh || exit 1

ENTRYPOINT [ "/sbin/entrypoint.sh" ]
CMD [ "start-service" ]

Ndiye zomwe zili mu

choyamba_yamba.sh

#!/usr/bin/expect -f
set timeout -1
spawn /splunkforwarder/bin/splunk start --accept-license
expect "Please enter an administrator username: "
send -- "adminr"
expect "Please enter a new password: "
send -- "changemer"
expect "Please confirm new password: "
send -- "changemer"
expect eof

Pachiyambi choyamba, Splunk akufunsani kuti mupereke malowedwe / mawu achinsinsi, KOMA detayi imagwiritsidwa ntchito okha kuti mupereke malamulo oyendetsera ntchitoyo, ndiye kuti, mkati mwa chidebecho. Kwa ife, timangofuna kuyambitsa chidebecho kuti zonse zigwire ntchito ndipo zipika ziziyenda ngati mtsinje. Inde, iyi ndi hardcode, koma sindinapeze njira zina.

Komanso malinga ndi script akuchitidwa

/splunkforwarder/bin/splunk install app /splunkclouduf.spl -auth admin:changeme

splunkclouduf.spl - Iyi ndi fayilo yotsimikizira za Splunk Universal Forwarder, yomwe imatha kutsitsidwa pa intaneti.

Komwe mungadina kuti mutsitse (pazithunzi)Splunk Universal Forwarder mu docker ngati wosonkhanitsa logi

Splunk Universal Forwarder mu docker ngati wosonkhanitsa logi
Uwu ndi nkhokwe yanthawi zonse yomwe imatha kumasulidwa. Mkati mwake muli ziphaso ndi mawu achinsinsi olumikizirana ndi SplunkCloud yathu ndi zotsatira.conf ndi mndandanda wa zochitika zathu. Fayiloyi idzakhala yofunikira mpaka mutakhazikitsanso kukhazikitsa kwa Splunk kapena kuwonjezera node yolowera ngati kukhazikitsa kuli pamalopo. Choncho, palibe cholakwika ndi kuwonjezera mkati mwa chidebe.

Ndipo chinthu chomaliza ndikuyambiranso. Inde, kuti mugwiritse ntchito zosinthazo, muyenera kuyiyambitsanso.

Mu wathu inputs.conf timawonjezera zipika zomwe tikufuna kutumiza ku Splunk. Sikoyenera kuwonjezera fayiloyi pachithunzichi ngati, mwachitsanzo, mumagawa ma configs kudzera pa chidole. Chokhacho ndi chakuti Forwarder amawona ma configs pamene daemon iyamba, mwinamwake idzafunika ./splunk restart.

Ndi zolembedwa zamtundu wanji za docker? Pali yankho lakale pa Github kuchokera outcoldman, zolembazo zinatengedwa kuchokera kumeneko ndikusinthidwa kuti zigwire ntchito ndi ma Docker (ce-17.*) ndi Splunk (7.*).

Ndi zomwe mwapeza, mutha kupanga zotsatirazi

dashboards: (zithunzi zingapo)Splunk Universal Forwarder mu docker ngati wosonkhanitsa logi

Splunk Universal Forwarder mu docker ngati wosonkhanitsa logi
Magwero a mizere ali mu ulalo womwe waperekedwa kumapeto kwa nkhaniyo. Chonde dziwani kuti pali magawo awiri osankhidwa: 2 - kusankha kwa index (kufufuzidwa ndi chigoba), kusankha kolandira / chotengera. Muyenera kusintha chigoba cha index, kutengera mayina omwe mumagwiritsa ntchito.

Pomaliza, ndikufuna ndikuwonetseni za ntchitoyi kuyamba () Π²

polowera.sh

start() {
    trap teardown EXIT
	if [ -z $SPLUNK_INDEX ]; then
	echo "'SPLUNK_INDEX' env variable is empty or not defined. Should be 'dev' or 'prd'." >&2
	exit 1
	else
	sed -e "s/@index@/$SPLUNK_INDEX/" -i ${SPLUNK_HOME}/etc/system/local/inputs.conf
	fi
	sed -e "s/@hostname@/$(cat /etc/hostname)/" -i ${SPLUNK_HOME}/etc/system/local/inputs.conf
    sh -c "echo 'starting' > /tmp/splunk-container.state"
	${SPLUNK_HOME}/bin/splunk start
    watch_for_failure
}

Kwa ine, pa chilengedwe chilichonse komanso gulu lililonse, kaya ndikugwiritsa ntchito mu chidebe kapena makina osungira, timagwiritsa ntchito index yosiyana. Mwanjira iyi, liwiro losaka silidzavutikira pakakhala kusonkhanitsa kwakukulu kwa data. Lamulo losavuta limagwiritsidwa ntchito kutchula ma index: _. Chifukwa chake, kuti chidebecho chikhale chapadziko lonse lapansi, tisanayambitse daemon yokha, timasintha ludzu-th wildcard ku dzina la chilengedwe. Kusintha kwa dzina la chilengedwe kumadutsa pazosintha zachilengedwe. Zikumveka zoseketsa.

Ndizoyeneranso kudziwa kuti pazifukwa zina Splunk samakhudzidwa ndi kupezeka kwa parameter ya docker dzina lake. Adzatumizabe zipika mouma khosi ndi id ya chidebe chake m'munda wa alendo. Monga yankho, mukhoza kukwera / etc / hostname kuchokera pamakina opangira komanso poyambira pangani zosintha zofanana ndi mayina a index.

Chitsanzo docker-compose.yml

version: '2'
services:
  splunk-forwarder:
    image: "${IMAGE_REPO}/docker-stats-splunk-forwarder:${IMAGE_VERSION}"
    environment:
      SPLUNK_INDEX: ${ENVIRONMENT}
    volumes:
    - /etc/hostname:/etc/hostname:ro
    - /var/log:/var/log
    - /var/run/docker.sock:/var/run/docker.sock:ro

Zotsatira

Inde, mwina yankho silili labwino ndipo ndithudi siliri la aliyense, popeza alipo ambiri "hardcode". Koma potengera izo, aliyense akhoza kupanga chithunzi chake ndikuchiyika muzojambula zawo zapadera, ngati, monga momwe zimachitikira, mukufunikira Splunk Forwarder ku Docker.

Zolemba:

Yankho kuchokera m'nkhaniyi
Yankho lochokera kwa outcoldman lomwe lidatilimbikitsa kuti tigwiritsenso ntchito zina
Za. zolemba zokhazikitsa Universal Forwarder

Source: www.habr.com

Kuwonjezera ndemanga