Ukumisela iqela le-Nomad usebenzisa i-Consul kunye nokudibanisa ne-Gitlab

Intshayelelo

Kutshanje, ukuthandwa kweKubernetes kuye kwakhula ngokukhawuleza - ngakumbi nangakumbi iiprojekthi ziyayiphumeza. Ndandifuna ukuchukumisa i-orchestrator efana ne-Nomad: ifanelekile kwiiprojekthi esele zisebenzisa ezinye izisombululo ezivela kwi-HashiCorp, umzekelo, i-Vault kunye ne-Consul, kunye neeprojekthi ngokwazo azinzima ngokweziseko zophuhliso. Esi sixhobo siya kuba nemiyalelo yokufaka i-Nomad, ukudibanisa i-nodes ezimbini kwi-cluster, kunye nokudibanisa i-Nomad kunye ne-Gitlab.

ibhentshi yovavanyo

Kancinci malunga nebhentshi yokuvavanya: iiseva ezintathu ezibonakalayo zisetyenziselwa iimpawu ze-2 CPU, i-RAM 4, i-50 Gb SSD, idityaniswe kwinethiwekhi yendawo eqhelekileyo. Amagama abo kunye needilesi ze-IP:

1. nomad-livelinux-01: 172.30.0.5
2. nomad-livelinux-02: 172.30.0.10
3. i-consul-livelinux-01: 172.30.0.15

Ukufakwa kukaNomad, u-Consul. Ukudala iqela le-Nomad

Masiqale ngofakelo olusisiseko. Nangona ukuseta bekulula, ndiza kuyichaza ngenxa yemfezeko yenqaku: yenziwe ngokusisiseko ukusuka kwidrafti kunye namanqaku okufikelela ngokukhawuleza xa kufuneka.

Ngaphambi kokuba siqale ukuziqhelanisa, siya kuxubusha inxalenye yethiyori, kuba kweli nqanaba kubalulekile ukuqonda isakhiwo sexesha elizayo.

Sineendawo ezimbini ze-nomad kwaye sifuna ukuzidibanisa zibe yimbumba, kwaye kwixesha elizayo siya kufuna ukulinganiswa kweqela elizenzekelayo - ngenxa yoku siya kufuna i-Consul. Ngesi sixhobo, ukudibanisa kunye nokongeza ii-nodes ezintsha kuba ngumsebenzi olula kakhulu: i-node ye-Nomad eyenziwe idibanisa kwi-agent ye-Consul, kwaye idibanisa kwiqela elikhoyo le-Nomad. Ngoko ke, ekuqaleni siza kufaka umncedisi we-Consul, siqwalasele ugunyaziso olusisiseko lwe-http lwephaneli yewebhu (ngaphandle kogunyaziso olungagqibekanga kwaye lunokufikelelwa kwidilesi yangaphandle), kunye ne-Consul agents ngokwabo kwiiseva ze-Nomad, emva koko. sizodlulela kuNomad qha.

Ukufakela izixhobo ze-HashiCorp kulula kakhulu: ngokusisiseko, sihambisa nje ifayile yokubini kwi-bin directory, simise ifayile yoqwalaselo yesixhobo, kwaye senze ifayile yayo yenkonzo.

Khuphela ifayile yokubini ye-Consul kwaye uyikhuphe kulawulo lwasekhaya lomsebenzisi:

root@consul-livelinux-01:~# wget https://releases.hashicorp.com/consul/1.5.0/consul_1.5.0_linux_amd64.zip
root@consul-livelinux-01:~# unzip consul_1.5.0_linux_amd64.zip
root@consul-livelinux-01:~# mv consul /usr/local/bin/

Ngoku sinayo i-consul binary esele ilungile ukulungiselela uqwalaselo olongezelelweyo.

Ukusebenzisana no-Consul, kufuneka senze isitshixo esisodwa ngokusebenzisa umyalelo we-keygen:

root@consul-livelinux-01:~# consul keygen

Masiqhubele phambili ukuseta uqwalaselo lwe-Consul, sisenza isilawuli /etc/consul.d/ ngesakhiwo esilandelayo:

/etc/consul.d/
├── bootstrap
│   └── config.json

Ulawulo lwe-bootstrap luya kuqulatha ifayile yoqwalaselo config.json - kuyo siya kuseka izicwangciso zeConsul. Imixholo yayo:

{
"bootstrap": true,
"server": true,
"datacenter": "dc1",
"data_dir": "/var/consul",
"encrypt": "your-key",
"log_level": "INFO",
"enable_syslog": true,
"start_join": ["172.30.0.15"]
}

Makhe sijonge izikhokelo eziphambili kunye neentsingiselo zazo ngokwahlukeneyo:

• I-bootstrap: yinyani. Sivumela ukongezwa okuzenzekelayo kweenodi ezintsha ukuba ziqhagamshelwe. Ndiyaqaphela ukuba asibonisi apha elona nani lichanekileyo leendawo ezilindelekileyo.
• Mncedisi: yinyani. Vula imo yeseva. U-Consul kulo matshini wenyani uya kusebenza njengomncedisi kuphela kunye nenkosi okwangoku, i-VM kaNomad iya kuba ngabathengi.
• nkcukacha:dc1. Chaza igama leziko ledatha ukwenza iqela. Kufuneka ifane kubo bobabini abathengi kunye nabancedisi.
• kubhala: isitshixo sakho. Isitshixo, ekufuneka kwakhona sibe yodwa kwaye ifanise kubo bonke abathengi kunye neeseva. Yenziwe ngokusebenzisa i-consul keygen umyalelo.
• qala_join. Kolu luhlu sibonisa uluhlu lweedilesi ze-IP apho uxhumano luya kwenziwa khona. Okwangoku sishiya idilesi yethu kuphela.

Kweli nqanaba sinokuqhuba i-consul sisebenzisa umgca womyalelo:

root@consul-livelinux-01:~# /usr/local/bin/consul agent -config-dir /etc/consul.d/bootstrap -ui

Le yindlela elungileyo yokulungisa ingxaki ngoku, nangona kunjalo, awuyi kukwazi ukusebenzisa le ndlela ngokuqhubekayo ngenxa yezizathu ezicacileyo. Masenze ifayile yenkonzo yokulawula u-Consul nge-systemd:

root@consul-livelinux-01:~# nano /etc/systemd/system/consul.service

Imixholo yefayile ye-consul.service:

[Unit]
Description=Consul Startup process
After=network.target
 
[Service]
Type=simple
ExecStart=/bin/bash -c '/usr/local/bin/consul agent -config-dir /etc/consul.d/bootstrap -ui' 
TimeoutStartSec=0
 
[Install]
WantedBy=default.target

Qalisa uMmeli nge-systemctl:

root@consul-livelinux-01:~# systemctl start consul

Makhe sijonge: inkonzo yethu kufuneka isebenze, kwaye ngokwenza umyalelo wamalungu e-consul kufuneka sibone iseva yethu:

root@consul-livelinux:/etc/consul.d# consul members
consul-livelinux    172.30.0.15:8301  alive   server  1.5.0  2         dc1  <all>

Inqanaba elilandelayo: ukufaka i-Nginx kunye nokuseta i-proxying kunye nogunyaziso lwe-http. Sifaka i-nginx ngokusebenzisa umphathi wepakethe kwaye kwi-/etc/nginx/site-enabled directory senza ifayile yoqwalaselo consul.conf ngeziqulatho zilandelayo:

upstream consul-auth {
    server localhost:8500;
}

server {

    server_name consul.doman.name;
    
    location / {
      proxy_pass http://consul-auth;
      proxy_set_header Host $host;
      auth_basic_user_file /etc/nginx/.htpasswd;
      auth_basic "Password-protected Area";
    }
}

Ungalibali ukwenza ifayile ye.htpasswd kwaye uvelise igama lomsebenzisi kunye negama lokugqitha kuyo. Le nto iyafuneka ukuze iphaneli yewebhu ingafumaneki kuye wonke umntu owazi isizinda sethu. Nangona kunjalo, xa siseta i-Gitlab, kuya kufuneka siyiyeke le nto - kungenjalo asizukwazi ukuthumela isicelo sethu kuNomad. Kwiprojekthi yam, zombini i-Gitlab kunye ne-Nomad zikwiwebhu engwevu kuphela, ngoko ke akukho ngxaki apha.

Kwiiseva ezimbini eziseleyo sifaka i-Consul agents ngokwemiyalelo elandelayo. Siphinda amanyathelo ngefayile yokubini:

root@nomad-livelinux-01:~# wget https://releases.hashicorp.com/consul/1.5.0/consul_1.5.0_linux_amd64.zip
root@nomad-livelinux-01:~# unzip consul_1.5.0_linux_amd64.zip
root@nomad-livelinux-01:~# mv consul /usr/local/bin/

Ngokufanisa kunye nomncedisi wangaphambili, senza uluhlu lweefayile zoqwalaselo /etc/consul.d kunye nesakhiwo esilandelayo:

/etc/consul.d/
├── client
│   └── config.json

Imixholo yefayile ye-config.json:

{
    "datacenter": "dc1",
    "data_dir": "/opt/consul",
    "log_level": "DEBUG",
    "node_name": "nomad-livelinux-01",
    "server": false,
    "encrypt": "your-private-key",
    "domain": "livelinux",
    "addresses": {
      "dns": "127.0.0.1",
      "https": "0.0.0.0",
      "grpc": "127.0.0.1",
      "http": "127.0.0.1"
    },
    "bind_addr": "172.30.0.5", # локальный адрес вм
    "start_join": ["172.30.0.15"], # удаленный адрес консул сервера
    "ports": {
      "dns": 53
     }

Gcina utshintsho kwaye uqhubele phambili ukuseta ifayile yenkonzo, imixholo yayo:

/etc/systemd/system/consul.service:

[Unit]
Description="HashiCorp Consul - A service mesh solution"
Documentation=https://www.consul.io/
Requires=network-online.target
After=network-online.target

[Service]
User=root
Group=root
ExecStart=/usr/local/bin/consul agent -config-dir=/etc/consul.d/client
ExecReload=/usr/local/bin/consul reload
KillMode=process
Restart=on-failure

[Install]
WantedBy=multi-user.target

Siqalisa i-consul kumncedisi. Ngoku, emva kokuqaliswa, kufuneka sibone inkonzo emiselweyo kumalungu e-nsul. Oku kuya kuthetha ukuba iqhagamshele ngempumelelo kwiqela njengomthengi. Phinda okufanayo kwiseva yesibini kwaye emva koko sinokuqala ukufaka kunye nokuqwalasela iNomad.

Ufakelo oluneenkcukacha ngakumbi lweNomad luchazwe kumaxwebhu asemthethweni. Kukho iindlela ezimbini zokufakela zemveli: ukukhuphela ifayile yokubini kunye nokuqokelela kwimvelaphi. Ndiza kukhetha indlela yokuqala.

Qaphela:: Iprojekthi ikhula ngokukhawuleza, uhlaziyo olutsha luhlala lukhutshwa. Mhlawumbi inguqulelo entsha iya kukhutshwa ngexesha eligqityiweyo eli nqaku. Ke, ngaphambi kokufunda, ndincoma ukujonga inguqulelo yangoku yeNomad okwangoku kwaye uyikhuphele.

root@nomad-livelinux-01:~# wget https://releases.hashicorp.com/nomad/0.9.1/nomad_0.9.1_linux_amd64.zip
root@nomad-livelinux-01:~# unzip nomad_0.9.1_linux_amd64.zip
root@nomad-livelinux-01:~# mv nomad /usr/local/bin/
root@nomad-livelinux-01:~# nomad -autocomplete-install
root@nomad-livelinux-01:~# complete -C /usr/local/bin/nomad nomad
root@nomad-livelinux-01:~# mkdir /etc/nomad.d

Emva kokukhulula, siya kufumana ifayile ye-binary ye-Nomad enobunzima be-65 MB - kufuneka ihanjiswe ku /usr/local/bin.

Masenze uvimba wedatha weNomad kwaye sihlele ifayile yenkonzo yayo (enokwenzeka ukuba ayizubakho ekuqaleni):

root@nomad-livelinux-01:~# mkdir --parents /opt/nomad
root@nomad-livelinux-01:~# nano /etc/systemd/system/nomad.service

Ncamathelisa le migca ilandelayo apho:

[Unit]
Description=Nomad
Documentation=https://nomadproject.io/docs/
Wants=network-online.target
After=network-online.target

[Service]
ExecReload=/bin/kill -HUP $MAINPID
ExecStart=/usr/local/bin/nomad agent -config /etc/nomad.d
KillMode=process
KillSignal=SIGINT
LimitNOFILE=infinity
LimitNPROC=infinity
Restart=on-failure
RestartSec=2
StartLimitBurst=3
StartLimitIntervalSec=10
TasksMax=infinity

[Install]
WantedBy=multi-user.target

Nangona kunjalo, asingxamanga ukuzisa i-nomad - asikayidali ifayile yayo yoqwalaselo:

root@nomad-livelinux-01:~# mkdir --parents /etc/nomad.d
root@nomad-livelinux-01:~# chmod 700 /etc/nomad.d
root@nomad-livelinux-01:~# nano /etc/nomad.d/nomad.hcl
root@nomad-livelinux-01:~# nano /etc/nomad.d/server.hcl

Uluhlu lokugqibela lolawulo luya kuba ngolu hlobo lulandelayo:

/etc/nomad.d/
├── nomad.hcl
└── server.hcl

Ifayile ye nomad.hcl kufuneka iqulathe uqwalaselo olulandelayo:

datacenter = "dc1"
data_dir = "/opt/nomad"

Imixholo yefayile ye-server.hcl:

server {
  enabled = true
  bootstrap_expect = 1
}

consul {
  address             = "127.0.0.1:8500"
  server_service_name = "nomad"
  client_service_name = "nomad-client"
  auto_advertise      = true
  server_auto_join    = true
  client_auto_join    = true
}

bind_addr = "127.0.0.1" 

advertise {
  http = "172.30.0.5"
}

client {
  enabled = true
}

Ungalibali ukutshintsha ifayile yoqwalaselo kwiseva yesibini - apho kuya kufuneka utshintshe ixabiso lomyalelo we-http.

Into yokugqibela kweli nqanaba kukuqwalasela i-Nginx ye-proxying kunye nokuseta ugunyaziso lwe-http. Imixholo yefayile nomad.conf:

upstream nomad-auth {
        server 172.30.0.5:4646;
}

server {

        server_name nomad.domain.name;
        
        location / {
	        proxy_pass http://nomad-auth;
	        proxy_set_header Host $host;
	        auth_basic_user_file /etc/nginx/.htpasswd;
		   auth_basic "Password-protected Area";
        }
        
}

Ngoku sinokufikelela kwiphaneli yewebhu ngokusebenzisa inethiwekhi yangaphandle. Qhagamshela kwaye uye kwiphepha labancedisi:

Umfanekiso 1. Uluhlu lwabancedisi kwiqela le-Nomad

Zombini iiseva ziboniswe ngempumelelo kwiqela lenjongo, siya kubona into enye kwimveliso yomyalelo wemo ye-nomad node:

Umfanekiso 2. Imveliso yomyalelo wemo ye-node ye-nomad

Kuthekani ngoConsul? Makhe sijonge. Yiya kwiqela lolawulo le-Consul, kwiphepha leendawo:

Umfanekiso 3. Uluhlu lweendawo zokuhlala kwiqela le-Consul

Ngoku sinoNomad olungisiweyo osebenza ngokubambisana no-Consul. Kwinqanaba lokugqibela, siya kufika kwindawo eyonwabisayo: ukuseta izikhongozeli zeDocker ukusuka eGitlab ukuya kuNomad, kwaye sithetha ngezinye zeempawu zayo ezahlukileyo.

Ukudala iGitlab Runner

Ukuhambisa imifanekiso ye-docker kwi-Nomad, siya kusebenzisa imbaleki eyahlukileyo kunye nefayile yokubini ye-Nomad ngaphakathi (apha, ngendlela, sinokuqaphela enye inkalo yezicelo ze-Hashicorp-ngayodwa ziyifayile yokubini enye). Yilayishe kulawulo lobaleki. Masenze iDockerfile elula kuyo ngomxholo olandelayo:

FROM alpine:3.9
RUN apk add --update --no-cache libc6-compat gettext
COPY nomad /usr/local/bin/nomad

Kwakwiprojekthi enye sidala .gitlab-ci.yml:

variables:
  DOCKER_IMAGE: nomad/nomad-deploy
  DOCKER_REGISTRY: registry.domain.name
 

stages:
  - build

build:
  stage: build
  image: ${DOCKER_REGISTRY}/nomad/alpine:3
  script:
    - tag=${DOCKER_REGISTRY}/${DOCKER_IMAGE}:latest
    - docker build --pull -t ${tag} -f Dockerfile .
    - docker push ${tag}

Ngenxa yoko, siya kuba nomfanekiso okhoyo wembaleki ye-Nomad kwi-Registry ye-Gitlab, ngoku sinokuya ngqo kwindawo yokugcina iprojekthi, senze iPipeline kwaye silungise umsebenzi we-Nomad.

Ukuseta iprojekthi

Masiqale ngefayile yomsebenzi kaNomad. Iprojekthi yam kweli nqaku iya kuba yeyokuqala: iya kubandakanya umsebenzi omnye. Imixholo ye-.gitlab-ci iya kuba ngolu hlobo lulandelayo:

variables:
  NOMAD_ADDR: http://nomad.address.service:4646
  DOCKER_REGISTRY: registry.domain.name
  DOCKER_IMAGE: example/project

stages:
  - build
  - deploy

build:
  stage: build
  image: ${DOCKER_REGISTRY}/nomad-runner/alpine:3
  script:
    - tag=${DOCKER_REGISTRY}/${DOCKER_IMAGE}:${CI_COMMIT_SHORT_SHA}
    - docker build --pull -t ${tag} -f Dockerfile .
    - docker push ${tag}


deploy:
  stage: deploy
  image: registry.example.com/nomad/nomad-runner:latest
  script:
    - envsubst '${CI_COMMIT_SHORT_SHA}' < project.nomad > job.nomad
    - cat job.nomad
    - nomad validate job.nomad
    - nomad plan job.nomad || if [ $? -eq 255 ]; then exit 255; else echo "success"; fi
    - nomad run job.nomad
  environment:
    name: production
  allow_failure: false
  when: manual

Apha ukuthunyelwa kwenzeka ngesandla, kodwa ungayiqwalasela ukutshintsha imixholo yolawulo lweprojekthi. Umbhobho unezigaba ezibini: ukuhlanganisa umfanekiso kunye nokuthunyelwa kwawo kwi-nomad. Kwinqanaba lokuqala, sihlanganisa umfanekiso we-docker kwaye siwutyhalele kwiRejistri yethu, kwaye okwesibini sisungula umsebenzi wethu eNomad.

job "monitoring-status" {
    datacenters = ["dc1"]
    migrate {
        max_parallel = 3
        health_check = "checks"
        min_healthy_time = "15s"
        healthy_deadline = "5m"
    }

    group "zhadan.ltd" {
        count = 1
        update {
            max_parallel      = 1
            min_healthy_time  = "30s"
            healthy_deadline  = "5m"
            progress_deadline = "10m"
            auto_revert       = true
        }
        task "service-monitoring" {
            driver = "docker"

            config {
                image = "registry.domain.name/example/project:${CI_COMMIT_SHORT_SHA}"
                force_pull = true
                auth {
                    username = "gitlab_user"
                    password = "gitlab_password"
                }
                port_map {
                    http = 8000
                }
            }
            resources {
                network {
                    port "http" {}
                }
            }
        }
    }
}

Nceda uqaphele ukuba ndineRejistri yabucala kwaye ukutsala ngempumelelo umfanekiso wedocker kufuneka ndingene kuwo. Esona sisombululo silungileyo kulo mzekelo kukufaka igama lokungena kunye negama lokugqitha kwiVault kwaye emva koko uyidibanise noNomad. UNomad ngokwemveli uyayixhasa iVault. Kodwa okokuqala, kwiVault ngokwayo, siza kufaka imigaqo-nkqubo eyimfuneko yeNomad, unokuzikhuphela:

# Download the policy and token role
$ curl https://nomadproject.io/data/vault/nomad-server-policy.hcl -O -s -L
$ curl https://nomadproject.io/data/vault/nomad-cluster-role.json -O -s -L

# Write the policy to Vault
$ vault policy write nomad-server nomad-server-policy.hcl

# Create the token role with Vault
$ vault write /auth/token/roles/nomad-cluster @nomad-cluster-role.json

Ngoku, emva kokwenza imigaqo-nkqubo eyimfuneko, siya kongeza ukudibanisa kunye neVault kwibhloko yomsebenzi kwifayile ye-job.nomad:

vault {
  enabled = true
  address = "https://vault.domain.name:8200"
  token = "token"
}

Ndisebenzisa ugunyaziso ngethokheni kwaye ndiyibhalisa ngokuthe ngqo apha, kukwakhona ukhetho lokuchaza umqondiso njengoguquguqukayo xa ndiqala iarhente ye-nomad:

$ VAULT_TOKEN=<token> nomad agent -config /path/to/config

Ngoku sinokusebenzisa izitshixo ngeVault. Umgaqo wokusebenza ulula: senza ifayile kumsebenzi weNomad oya kugcina amaxabiso ezinto eziguquguqukayo, umzekelo:

template {
                data = <<EOH
{{with secret "secrets/pipeline-keys"}}
REGISTRY_LOGIN="{{ .Data.REGISTRY_LOGIN }}"
REGISTRY_PASSWORD="{{ .Data.REGISTRY_LOGIN }}{{ end }}"

EOH
    destination = "secrets/service-name.env"
    env = true
}

Ngale ndlela ilula, ungaqwalasela ukuhanjiswa kwezikhongozeli kwiqela le-Nomad kwaye usebenze nayo kwixesha elizayo. Ndiza kuthetha ukuba kwinqanaba elithile ndiyavelana noNomad - ifanelekile ngakumbi kwiiprojekthi ezincinci apho i-Kubernetes inokubangela ubunzima obongezelelweyo kwaye ayiyi kuqonda amandla ayo apheleleyo. Kwaye, i-Nomad ifanelekile kwabaqalayo-kulula ukuyifaka kunye nokuqwalasela. Nangona kunjalo, xa ndivavanya iiprojekthi ezithile, ndidibana nengxaki kwiinguqulelo zayo zangaphambili - imisebenzi emininzi esisiseko ayikho okanye ayisebenzi ngokuchanekileyo. Nangona kunjalo, ndiyakholelwa ukuba i-Nomad iya kuqhubeka nokuphuhlisa kwaye kwixesha elizayo iya kufumana imisebenzi efunwa ngumntu wonke.

Umbhali: U-Ilya Andreev, uhlelwe ngu-Alexey Zhadan kunye neqela le-Live Linux


umthombo: www.habr.com