Kukhazikitsa gulu la Nomad pogwiritsa ntchito Consul ndikuphatikiza ndi Gitlab

Mau oyamba

Posachedwapa, kutchuka kwa Kubernetes kwakula mofulumira - ntchito zambiri zikukwaniritsa izo. Ndinkafuna kukhudza oimba ngati Nomad: ndiabwino kwa mapulojekiti omwe amagwiritsa ntchito kale njira zina kuchokera ku HashiCorp, mwachitsanzo, Vault ndi Consul, ndipo mapulojekitiwo sakhala ovuta pokhudzana ndi zomangamanga. Nkhaniyi idzakhala ndi malangizo oyika Nomad, kuphatikiza mfundo ziwiri kukhala gulu, komanso kuphatikiza Nomad ndi Gitlab.

benchi yoyesera

Pang'ono ndi benchi yoyesera: ma seva atatu omwe amagwiritsidwa ntchito ndi mawonekedwe a 2 CPU, 4 RAM, 50 Gb SSD, olumikizidwa kukhala maukonde wamba. Mayina awo ndi ma adilesi a IP:

1. nomad-livelinux-01: 172.30.0.5
2. nomad-livelinux-02: 172.30.0.10
3. Consul-livelinux-01: 172.30.0.15

Kukhazikitsidwa kwa Nomad, Consul. Kupanga gulu la Nomad

Tiyeni tiyambe ndi kukhazikitsa zofunika. Ngakhale kukhazikitsidwa kunali kosavuta, ndikulongosola chifukwa cha kukhulupirika kwa nkhaniyi: idapangidwa kuchokera ku zolembedwa ndi zolemba kuti mufike mwachangu pakafunika.

Tisanayambe kuchita, tidzakambirana gawo lachidziwitso, chifukwa panthawiyi ndikofunika kumvetsetsa dongosolo lamtsogolo.

Tili ndi ma node awiri oyendayenda ndipo tikufuna kuwaphatikiza kukhala gulu, ndipo mtsogolomu tidzafunikanso makulitsidwe am'magulu - chifukwa cha izi tidzafunika Consul. Ndi chida ichi, kusonkhanitsa ndi kuwonjezera ma node atsopano kumakhala ntchito yosavuta kwambiri: Node yopangidwa ya Nomad imagwirizanitsa ndi Consul agent, ndiyeno imagwirizanitsa ndi gulu la Nomad lomwe liripo. Chifukwa chake, poyambira tidzakhazikitsa seva ya Consul, sinthani chilolezo choyambira pa tsamba lawebusayiti (zilibe chilolezo mwachisawawa ndipo zitha kupezeka pa adilesi yakunja), komanso othandizira a Consul okha pa ma seva a Nomad, pambuyo pake. Tidzangopitirira ku Nomad.

Kuyika zida za HashiCorp ndikosavuta: kwenikweni, timangosuntha fayilo ya binary kupita ku bukhu la bin, kukhazikitsa fayilo yosinthira chida, ndikupanga fayilo yake yautumiki.

Tsitsani fayilo ya Binary ya Consul ndikuyimasulira m'ndandanda wanyumba ya wosuta:

root@consul-livelinux-01:~# wget https://releases.hashicorp.com/consul/1.5.0/consul_1.5.0_linux_amd64.zip
root@consul-livelinux-01:~# unzip consul_1.5.0_linux_amd64.zip
root@consul-livelinux-01:~# mv consul /usr/local/bin/

Tsopano tili ndi kazembe wopangidwa okonzeka kuti tikonzenso.

Kuti tigwire ntchito ndi Consul, tifunika kupanga kiyi yapadera pogwiritsa ntchito keygen command:

root@consul-livelinux-01:~# consul keygen

Tiyeni tipitirire kukhazikitsa kasinthidwe ka Consul, ndikupanga bukhu /etc/consul.d/ ndi dongosolo ili:

/etc/consul.d/
├── bootstrap
│   └── config.json

Buku la bootstrap lidzakhala ndi fayilo yokonzekera config.json - mmenemo tidzakhazikitsa Consul zoikamo. Zomwe zili mkati mwake:

{
"bootstrap": true,
"server": true,
"datacenter": "dc1",
"data_dir": "/var/consul",
"encrypt": "your-key",
"log_level": "INFO",
"enable_syslog": true,
"start_join": ["172.30.0.15"]
}

Tiyeni tiwone mayendedwe akulu ndi matanthauzo ake padera:

• bootstrap: zoona. Timathandizira kuwonjezera ma node atsopano ngati alumikizidwa. Ndikuwona kuti sitikuwonetsa pano chiwerengero chenicheni cha ma node omwe akuyembekezeka.
• seva: zoona. Yambitsani mawonekedwe a seva. Consul pamakina awa azikhala ngati seva yokhayo komanso master pakadali pano, VM ya Nomad ikhala makasitomala.
• datacenterndi: dc1. Tchulani dzina la data center kuti mupange cluster. Iyenera kukhala yofanana pa kasitomala ndi ma seva.
• chitetezo: kiyi yanu. Chinsinsi, chomwe chiyeneranso kukhala chapadera ndikufanana ndi makasitomala onse ndi ma seva. Amapangidwa pogwiritsa ntchito lamulo la consul keygen.
• yambitsani_join. Pamndandandawu tikuwonetsa mndandanda wa ma adilesi a IP omwe kulumikizana kudzapangidwira. Pakadali pano timasiya adilesi yathu yokha.

Pakadali pano titha kuyendetsa consul pogwiritsa ntchito mzere wolamula:

root@consul-livelinux-01:~# /usr/local/bin/consul agent -config-dir /etc/consul.d/bootstrap -ui

Iyi ndi njira yabwino yothetsera vutoli tsopano, komabe, simungathe kugwiritsa ntchito njirayi nthawi zonse pazifukwa zomveka. Tiyeni tipange fayilo yothandizira kuti tiziyang'anira Consul kudzera pa systemd:

root@consul-livelinux-01:~# nano /etc/systemd/system/consul.service

Zomwe zili mu fayilo ya consul.service:

[Unit]
Description=Consul Startup process
After=network.target
 
[Service]
Type=simple
ExecStart=/bin/bash -c '/usr/local/bin/consul agent -config-dir /etc/consul.d/bootstrap -ui' 
TimeoutStartSec=0
 
[Install]
WantedBy=default.target

Yambitsani Consul kudzera pa systemctl:

root@consul-livelinux-01:~# systemctl start consul

Tiyeni tiwone: ntchito yathu iyenera kukhala ikuyenda, ndipo pochita lamulo la mamembala a consul tiyenera kuwona seva yathu:

root@consul-livelinux:/etc/consul.d# consul members
consul-livelinux    172.30.0.15:8301  alive   server  1.5.0  2         dc1  <all>

Gawo lotsatira: kukhazikitsa Nginx ndikukhazikitsa proxying ndi chilolezo cha http. Timayika nginx kudzera mu woyang'anira phukusi ndipo mu /etc/nginx/sites-enabled directory timapanga fayilo yosinthira consul.conf ndi izi:

upstream consul-auth {
    server localhost:8500;
}

server {

    server_name consul.doman.name;
    
    location / {
      proxy_pass http://consul-auth;
      proxy_set_header Host $host;
      auth_basic_user_file /etc/nginx/.htpasswd;
      auth_basic "Password-protected Area";
    }
}

Musaiwale kupanga fayilo ya .htpasswd ndikupanga dzina lolowera ndi mawu achinsinsi ake. Izi ndizofunikira kuti tsamba lawebusayiti lisapezeke kwa aliyense amene amadziwa dera lathu. Komabe, tikakhazikitsa Gitlab, tidzayenera kusiya izi - apo ayi sitidzatha kutumiza ntchito yathu ku Nomad. Mu pulojekiti yanga, onse a Gitlab ndi Nomad ali pa intaneti yotuwa, ndiye palibe vuto pano.

Pa ma seva awiri otsalawo timayika ma Consul agents motsatira malangizo awa. Timabwereza masitepe ndi fayilo ya binary:

root@nomad-livelinux-01:~# wget https://releases.hashicorp.com/consul/1.5.0/consul_1.5.0_linux_amd64.zip
root@nomad-livelinux-01:~# unzip consul_1.5.0_linux_amd64.zip
root@nomad-livelinux-01:~# mv consul /usr/local/bin/

Poyerekeza ndi seva yapitayi, timapanga chikwatu cha mafayilo osinthika /etc/consul.d ndi dongosolo ili:

/etc/consul.d/
├── client
│   └── config.json

Zomwe zili mu fayilo ya config.json:

{
    "datacenter": "dc1",
    "data_dir": "/opt/consul",
    "log_level": "DEBUG",
    "node_name": "nomad-livelinux-01",
    "server": false,
    "encrypt": "your-private-key",
    "domain": "livelinux",
    "addresses": {
      "dns": "127.0.0.1",
      "https": "0.0.0.0",
      "grpc": "127.0.0.1",
      "http": "127.0.0.1"
    },
    "bind_addr": "172.30.0.5", # локальный адрес вм
    "start_join": ["172.30.0.15"], # удаленный адрес консул сервера
    "ports": {
      "dns": 53
     }

Sungani zosinthazo ndikupitilira kukhazikitsa fayilo yautumiki, zomwe zili mkati mwake:

/etc/systemd/system/consul.service:

[Unit]
Description="HashiCorp Consul - A service mesh solution"
Documentation=https://www.consul.io/
Requires=network-online.target
After=network-online.target

[Service]
User=root
Group=root
ExecStart=/usr/local/bin/consul agent -config-dir=/etc/consul.d/client
ExecReload=/usr/local/bin/consul reload
KillMode=process
Restart=on-failure

[Install]
WantedBy=multi-user.target

Timatsegula consul pa seva. Tsopano, mutatha kukhazikitsa, tiyenera kuwona ntchito yokhazikitsidwa mwa mamembala a nsul. Izi zikutanthauza kuti yalumikizana bwino ndi gulu ngati kasitomala. Bwerezani zomwezo pa seva yachiwiri ndipo pambuyo pake tikhoza kuyamba kukhazikitsa ndi kukonza Nomad.

Kuyika kwatsatanetsatane kwa Nomad kukufotokozedwa m'malemba ake ovomerezeka. Pali njira ziwiri zokhazikitsira zachikhalidwe: kutsitsa fayilo ya binary ndikulemba kuchokera kugwero. Ndisankha njira yoyamba.

ndemanga: Ntchitoyi ikukula mwachangu, zosintha zatsopano zimatulutsidwa nthawi zambiri. Mwinamwake Baibulo latsopano lidzatulutsidwa pamene nkhaniyi idzamalizidwa. Chifukwa chake, ndisanawerenge, ndikupangira kuyang'ana mtundu waposachedwa wa Nomad pakadali pano ndikutsitsa.

root@nomad-livelinux-01:~# wget https://releases.hashicorp.com/nomad/0.9.1/nomad_0.9.1_linux_amd64.zip
root@nomad-livelinux-01:~# unzip nomad_0.9.1_linux_amd64.zip
root@nomad-livelinux-01:~# mv nomad /usr/local/bin/
root@nomad-livelinux-01:~# nomad -autocomplete-install
root@nomad-livelinux-01:~# complete -C /usr/local/bin/nomad nomad
root@nomad-livelinux-01:~# mkdir /etc/nomad.d

Pambuyo pomasula, tidzalandira fayilo ya binary ya Nomad yolemera 65 MB - iyenera kusamutsidwa ku /usr/local/bin.

Tiyeni tipange chikwatu cha Nomad ndikusintha fayilo yake yautumiki (mwina sizingakhalepo pachiyambi):

root@nomad-livelinux-01:~# mkdir --parents /opt/nomad
root@nomad-livelinux-01:~# nano /etc/systemd/system/nomad.service

Matani mizere iyi pamenepo:

[Unit]
Description=Nomad
Documentation=https://nomadproject.io/docs/
Wants=network-online.target
After=network-online.target

[Service]
ExecReload=/bin/kill -HUP $MAINPID
ExecStart=/usr/local/bin/nomad agent -config /etc/nomad.d
KillMode=process
KillSignal=SIGINT
LimitNOFILE=infinity
LimitNPROC=infinity
Restart=on-failure
RestartSec=2
StartLimitBurst=3
StartLimitIntervalSec=10
TasksMax=infinity

[Install]
WantedBy=multi-user.target

Komabe, sitikufulumira kuyambitsa nomad - sitinapange fayilo yake yosinthira:

root@nomad-livelinux-01:~# mkdir --parents /etc/nomad.d
root@nomad-livelinux-01:~# chmod 700 /etc/nomad.d
root@nomad-livelinux-01:~# nano /etc/nomad.d/nomad.hcl
root@nomad-livelinux-01:~# nano /etc/nomad.d/server.hcl

Mapangidwe omaliza a chikwatu adzakhala motere:

/etc/nomad.d/
├── nomad.hcl
└── server.hcl

Fayilo ya nomad.hcl iyenera kukhala ndi masinthidwe awa:

datacenter = "dc1"
data_dir = "/opt/nomad"

Zomwe zili mu fayilo ya seva.hcl:

server {
  enabled = true
  bootstrap_expect = 1
}

consul {
  address             = "127.0.0.1:8500"
  server_service_name = "nomad"
  client_service_name = "nomad-client"
  auto_advertise      = true
  server_auto_join    = true
  client_auto_join    = true
}

bind_addr = "127.0.0.1" 

advertise {
  http = "172.30.0.5"
}

client {
  enabled = true
}

Musaiwale kusintha fayilo yosinthira pa seva yachiwiri - pamenepo muyenera kusintha mtengo wa malangizo a http.

Chinthu chotsiriza pa nthawi ino ndikukonza Nginx kwa proxying ndi kukhazikitsa http chilolezo. Zomwe zili mufayilo ya nomad.conf:

upstream nomad-auth {
        server 172.30.0.5:4646;
}

server {

        server_name nomad.domain.name;
        
        location / {
	        proxy_pass http://nomad-auth;
	        proxy_set_header Host $host;
	        auth_basic_user_file /etc/nginx/.htpasswd;
		   auth_basic "Password-protected Area";
        }
        
}

Tsopano titha kulowa pagulu lawebusayiti kudzera pa netiweki yakunja. Lumikizani ndikupita kutsamba la seva:

Chithunzi 1. Mndandanda wa ma seva mu gulu la Nomad

Ma seva onsewa akuwonetsedwa bwino pagawo, tiwona zomwezo pakutulutsa kwa lamulo la nomad node:

Chithunzi 2. Kutulutsa kwa nomad node status command

Nanga Consul? Tiyeni tiwone. Pitani ku gulu lowongolera la Consul, patsamba la node:

Chithunzi 3. Mndandanda wa node mu gulu la Consul

Tsopano tili ndi Nomad yokonzekera yogwira ntchito limodzi ndi Consul. Pomaliza, tifika ku gawo losangalatsa: kukhazikitsa zotengera za Docker kuchokera ku Gitlab kupita ku Nomad, ndikukambirananso zina mwazinthu zake zapadera.

Kupanga Gitlab Runner

Kuti titumizire zithunzi za docker ku Nomad, tidzagwiritsa ntchito wothamanga wosiyana ndi fayilo ya binary ya Nomad mkati (apa, mwa njira, titha kuzindikira mbali ina ya mapulogalamu a Hashicorp - payekhapayekha ndi fayilo imodzi ya binary). Kwezani izo kwa wothamanga chikwatu. Tiyeni tipange Dockerfile yosavuta yake ndi izi:

FROM alpine:3.9
RUN apk add --update --no-cache libc6-compat gettext
COPY nomad /usr/local/bin/nomad

Muntchito yomweyi timapanga .gitlab-ci.yml:

variables:
  DOCKER_IMAGE: nomad/nomad-deploy
  DOCKER_REGISTRY: registry.domain.name
 

stages:
  - build

build:
  stage: build
  image: ${DOCKER_REGISTRY}/nomad/alpine:3
  script:
    - tag=${DOCKER_REGISTRY}/${DOCKER_IMAGE}:latest
    - docker build --pull -t ${tag} -f Dockerfile .
    - docker push ${tag}

Zotsatira zake, tidzakhala ndi chithunzi chopezeka cha wothamanga wa Nomad mu Gitlab Registry, tsopano titha kupita molunjika kumalo osungirako polojekiti, kupanga Pipeline ndikukonzekera ntchito ya Nomad.

Kukonzekera kwa polojekiti

Tiyeni tiyambe ndi fayilo ya ntchito ya Nomad. Ntchito yanga m'nkhaniyi ikhala yakale kwambiri: ikhala ndi ntchito imodzi. Zomwe zili mu .gitlab-ci zidzakhala motere:

variables:
  NOMAD_ADDR: http://nomad.address.service:4646
  DOCKER_REGISTRY: registry.domain.name
  DOCKER_IMAGE: example/project

stages:
  - build
  - deploy

build:
  stage: build
  image: ${DOCKER_REGISTRY}/nomad-runner/alpine:3
  script:
    - tag=${DOCKER_REGISTRY}/${DOCKER_IMAGE}:${CI_COMMIT_SHORT_SHA}
    - docker build --pull -t ${tag} -f Dockerfile .
    - docker push ${tag}


deploy:
  stage: deploy
  image: registry.example.com/nomad/nomad-runner:latest
  script:
    - envsubst '${CI_COMMIT_SHORT_SHA}' < project.nomad > job.nomad
    - cat job.nomad
    - nomad validate job.nomad
    - nomad plan job.nomad || if [ $? -eq 255 ]; then exit 255; else echo "success"; fi
    - nomad run job.nomad
  environment:
    name: production
  allow_failure: false
  when: manual

Apa kutumizidwa kumachitika pamanja, koma mutha kuyikonza kuti musinthe zomwe zili mu bukhu la polojekiti. Pipeline imakhala ndi magawo awiri: kusonkhanitsa zithunzi ndi kutumizidwa ku nomad. Pa gawo loyamba, timasonkhanitsa chithunzi cha docker ndikuchikankhira mu Registry yathu, ndipo chachiwiri timayambitsa ntchito yathu ku Nomad.

job "monitoring-status" {
    datacenters = ["dc1"]
    migrate {
        max_parallel = 3
        health_check = "checks"
        min_healthy_time = "15s"
        healthy_deadline = "5m"
    }

    group "zhadan.ltd" {
        count = 1
        update {
            max_parallel      = 1
            min_healthy_time  = "30s"
            healthy_deadline  = "5m"
            progress_deadline = "10m"
            auto_revert       = true
        }
        task "service-monitoring" {
            driver = "docker"

            config {
                image = "registry.domain.name/example/project:${CI_COMMIT_SHORT_SHA}"
                force_pull = true
                auth {
                    username = "gitlab_user"
                    password = "gitlab_password"
                }
                port_map {
                    http = 8000
                }
            }
            resources {
                network {
                    port "http" {}
                }
            }
        }
    }
}

Chonde dziwani kuti ndili ndi Registry yachinsinsi ndikukoka bwino chithunzi cha docker ndiyenera kulowamo. Yankho labwino kwambiri pankhaniyi ndikulowetsa malowedwe ndi mawu achinsinsi mu Vault ndikuphatikiza ndi Nomad. Nomad mbadwa amathandizira Vault. Koma choyamba, tiyeni tiyike mfundo zofunika za Nomad mu Vault yokha; atha kutsitsidwa:

# Download the policy and token role
$ curl https://nomadproject.io/data/vault/nomad-server-policy.hcl -O -s -L
$ curl https://nomadproject.io/data/vault/nomad-cluster-role.json -O -s -L

# Write the policy to Vault
$ vault policy write nomad-server nomad-server-policy.hcl

# Create the token role with Vault
$ vault write /auth/token/roles/nomad-cluster @nomad-cluster-role.json

Tsopano, titapanga mfundo zofunika, tiwonjezera kuphatikiza ndi Vault mu block block mu fayilo ya job.nomad:

vault {
  enabled = true
  address = "https://vault.domain.name:8200"
  token = "token"
}

Ndimagwiritsa ntchito chilolezo ndi chizindikiro ndikulembetsa pano, palinso mwayi wofotokozera chizindikirocho ngati chosinthika poyambitsa nomad wothandizira:

$ VAULT_TOKEN=<token> nomad agent -config /path/to/config

Tsopano titha kugwiritsa ntchito makiyi ndi Vault. Mfundo yogwira ntchito ndiyosavuta: timapanga fayilo mu Nomad ntchito yomwe imasunga zosintha, mwachitsanzo:

template {
                data = <<EOH
{{with secret "secrets/pipeline-keys"}}
REGISTRY_LOGIN="{{ .Data.REGISTRY_LOGIN }}"
REGISTRY_PASSWORD="{{ .Data.REGISTRY_LOGIN }}{{ end }}"

EOH
    destination = "secrets/service-name.env"
    env = true
}

Ndi njira yosavuta iyi, mutha kukonza zotumizira zotengera kugulu la Nomad ndikugwira nawo ntchito mtsogolo. Ndikunena kuti pamlingo wina ndimamvera chisoni Nomad - ndizoyeneranso ntchito zazing'ono pomwe Kubernetes angayambitse zovuta zina ndipo sangazindikire kuthekera kwake konse. Kuphatikiza apo, Nomad ndiyabwino kwa oyamba kumene - ndiyosavuta kukhazikitsa ndikusintha. Komabe, poyesa ntchito zina, ndimakumana ndi vuto ndi matembenuzidwe ake oyambirira - ntchito zambiri zoyambira sizimakhalapo kapena sizigwira ntchito moyenera. Komabe, ndikukhulupirira kuti Nomad ipitilira kukula ndipo mtsogolomo ipeza ntchito zomwe aliyense amafunikira.

Wolemba: Ilya Andreev, wolembedwa ndi Alexey Zhadan ndi gulu la Live Linux


Source: www.habr.com