ProHoster > Блог > Kev tswj hwm > Teeb tsa Nomad pawg siv Consul thiab koom nrog Gitlab

Teeb tsa Nomad pawg siv Consul thiab koom nrog Gitlab

Taw qhia

Tsis ntev los no, qhov nrov ntawm Kubernetes tau loj hlob sai heev - ntau thiab ntau qhov haujlwm tau siv nws. Kuv xav kov ntawm tus kws ntaus suab paj nruag zoo li Nomad: nws yog qhov zoo tshaj plaws rau cov haujlwm uas twb tau siv lwm cov kev daws teeb meem los ntawm HashiCorp, piv txwv li, Vault thiab Consul, thiab cov haujlwm lawv tus kheej tsis yog qhov nyuaj ntawm cov txheej txheem. Cov khoom no yuav muaj cov lus qhia rau kev txhim kho Nomad, sib txuas ob lub rau hauv ib pawg, nrog rau kev sib koom ua ke Nomad nrog Gitlab.

Teeb tsa Nomad pawg siv Consul thiab koom nrog Gitlab

Test sawv

Ib me ntsis txog lub rooj zaum sim: peb lub virtual servers siv nrog cov yam ntxwv ntawm 2 CPU, 4 RAM, 50 Gb SSD, koom ua ke rau hauv ib qho chaw hauv zos. Lawv cov npe thiab IP chaw nyob:

  1. nomad-livelinux-01: 172.30.0.5
  2. nomad-livelinux-02: 172.30.0.10
  3. consul-livelinux-01: 172.30.0.15

Installation ntawm Nomad, Consul. Tsim ib pawg Nomad

Cia peb pib nrog qhov yooj yim installation. Txawm hais tias qhov teeb tsa tau yooj yim, kuv yuav piav qhia nws rau kev ncaj ncees ntawm tsab xov xwm: nws yog qhov tseem ceeb tsim los ntawm cov ntawv sau thiab sau ntawv kom nkag tau sai thaum xav tau.

Ua ntej peb pib xyaum, peb yuav tham txog qhov theoretical ib feem, vim hais tias nyob rau theem no nws yog ib qho tseem ceeb kom nkag siab txog cov qauv yav tom ntej.

Peb muaj ob lub nomad nodes thiab peb xav muab lawv mus rau hauv ib pawg, thiab yav tom ntej peb kuj yuav xav tau ib tug tsis siv neeg pab pawg - rau qhov no peb yuav tsum tau Consul. Nrog rau cov cuab yeej no, pawg thiab ntxiv cov nodes tshiab dhau los ua ib txoj haujlwm yooj yim heev: tus tsim Nomad node txuas mus rau tus neeg sawv cev Consul, thiab tom qab ntawd txuas mus rau pawg Nomad uas twb muaj lawm. Yog li ntawd, thaum pib peb yuav nruab Consul server, teeb tsa kev tso cai http yooj yim rau lub vev xaib vaj huam sib luag (nws tsis muaj kev tso cai los ntawm lub neej ntawd thiab tuaj yeem nkag mus rau ntawm qhov chaw nyob sab nraud), nrog rau cov neeg sawv cev Consul lawv tus kheej ntawm Nomad servers, tom qab ntawd Peb tsuas yog mus rau Nomad.

Txhim kho HashiCorp cov cuab yeej yooj yim heev: qhov tseem ceeb, peb tsuas yog txav cov ntaub ntawv binary mus rau hauv phau ntawv teev npe, teeb tsa lub cuab yeej cov ntaub ntawv teeb tsa, thiab tsim nws cov ntaub ntawv pabcuam.

Download Consul binary file thiab unpack nws mus rau hauv tus neeg siv lub tsev directory:

root@consul-livelinux-01:~# wget https://releases.hashicorp.com/consul/1.5.0/consul_1.5.0_linux_amd64.zip
root@consul-livelinux-01:~# unzip consul_1.5.0_linux_amd64.zip
root@consul-livelinux-01:~# mv consul /usr/local/bin/

Tam sim no peb muaj npaj txhij-ua consul binary rau ntxiv configuration.

Txhawm rau ua haujlwm nrog Consul, peb yuav tsum tsim tus yuam sij tshwj xeeb siv cov lus txib keygen:

root@consul-livelinux-01:~# consul keygen

Cia peb mus rau kev teeb tsa Consul configuration, tsim ib daim ntawv teev npe /etc/consul.d/ nrog cov qauv hauv qab no:

/etc/consul.d/
├── bootstrap
│   └── config.json

Lub bootstrap directory yuav muaj cov ntaub ntawv teeb tsa config.json - hauv nws peb yuav teeb tsa Consul chaw. Nws cov ntsiab lus:

{
"bootstrap": true,
"server": true,
"datacenter": "dc1",
"data_dir": "/var/consul",
"encrypt": "your-key",
"log_level": "INFO",
"enable_syslog": true,
"start_join": ["172.30.0.15"]
}

Cia peb saib cov lus qhia tseem ceeb thiab lawv lub ntsiab lus sib cais:

  • bootstrap: tseeb. Peb pab kom tsis siv neeg ntxiv ntawm cov nodes tshiab yog tias lawv txuas nrog. Kuv nco ntsoov tias peb tsis qhia qhov tseeb ntawm qhov xav tau ntawm no.
  • neeg rau zaub mov: tseeb. Qhib hom server. Consul ntawm lub tshuab virtual no yuav ua raws li tib neeg rau zaub mov thiab tus tswv tam sim no, Nomad's VM yuav yog cov neeg siv khoom.
  • datacenterib: dc1. Qhia lub npe ntawm qhov chaw khaws ntaub ntawv los tsim cov pawg. Nws yuav tsum zoo ib yam ntawm ob tus neeg siv khoom thiab cov servers.
  • encrypt:koj-key. Qhov tseem ceeb, uas tseem yuav tsum tau tshwj xeeb thiab sib tw ntawm txhua tus neeg siv khoom thiab cov servers. Tsim siv tus consul keygen hais kom ua.
  • pib_join. Hauv daim ntawv teev npe no peb qhia cov npe ntawm IP chaw nyob uas qhov kev sib txuas yuav ua. Tam sim no peb tsuas yog tso peb tus kheej qhov chaw nyob.

Hauv qhov no, peb tuaj yeem khiav consul siv kab hais kom ua:

root@consul-livelinux-01:~# /usr/local/bin/consul agent -config-dir /etc/consul.d/bootstrap -ui

Qhov no yog ib txoj hauv kev zoo rau kev debug tam sim no, txawm li cas los xij, koj yuav tsis tuaj yeem siv txoj kev no ua ntu zus rau qhov laj thawj pom tseeb. Cia peb tsim cov ntaub ntawv pabcuam los tswj Consul ntawm systemd:

root@consul-livelinux-01:~# nano /etc/systemd/system/consul.service

Cov ntsiab lus ntawm consul.service file: 

[Unit]
Description=Consul Startup process
After=network.target
 
[Service]
Type=simple
ExecStart=/bin/bash -c '/usr/local/bin/consul agent -config-dir /etc/consul.d/bootstrap -ui' 
TimeoutStartSec=0
 
[Install]
WantedBy=default.target

Launch Consul ntawm systemctl:

root@consul-livelinux-01:~# systemctl start consul

Cia peb xyuas: peb cov kev pabcuam yuav tsum tau ua haujlwm, thiab los ntawm kev ua raws li cov tswvcuab hauv pawg tswj hwm peb yuav tsum pom peb cov server:

root@consul-livelinux:/etc/consul.d# consul members
consul-livelinux    172.30.0.15:8301  alive   server  1.5.0  2         dc1  <all>

Cov theem tom ntej: txhim kho Nginx thiab teeb tsa proxying thiab http kev tso cai. Peb nruab nginx los ntawm tus thawj tswj pob thiab hauv /etc/nginx/sites-enabled directory peb tsim cov ntaub ntawv consul.conf nrog cov ntsiab lus hauv qab no:

upstream consul-auth {
    server localhost:8500;
}

server {

    server_name consul.doman.name;
    
    location / {
      proxy_pass http://consul-auth;
      proxy_set_header Host $host;
      auth_basic_user_file /etc/nginx/.htpasswd;
      auth_basic "Password-protected Area";
    }
}

Tsis txhob hnov ​​​​qab tsim cov ntaub ntawv .htpasswd thiab tsim ib tus neeg siv lub npe thiab tus password rau nws. Cov khoom no yuav tsum tau kom lub vev xaib vaj huam sib luag tsis muaj rau txhua tus neeg uas paub peb lub npe. Txawm li cas los xij, thaum teeb tsa Gitlab, peb yuav tsum tso tseg qhov no - txwv tsis pub peb yuav tsis tuaj yeem xa peb daim ntawv thov mus rau Nomad. Hauv kuv qhov project, ob qho tib si Gitlab thiab Nomad tsuas yog nyob rau ntawm lub vev xaib grey, yog li tsis muaj teeb meem zoo li no.

Ntawm ob lub servers ntxiv peb nruab Consul tus neeg sawv cev raws li cov lus qhia hauv qab no. Peb rov ua cov kauj ruam nrog cov ntaub ntawv binary:

root@nomad-livelinux-01:~# wget https://releases.hashicorp.com/consul/1.5.0/consul_1.5.0_linux_amd64.zip
root@nomad-livelinux-01:~# unzip consul_1.5.0_linux_amd64.zip
root@nomad-livelinux-01:~# mv consul /usr/local/bin/

Los ntawm kev sib piv nrog cov neeg rau zaub mov dhau los, peb tsim cov npe rau cov ntaub ntawv teeb tsa /etc/consul.d nrog cov qauv hauv qab no:

/etc/consul.d/
├── client
│   └── config.json

Cov ntsiab lus ntawm cov ntaub ntawv config.json:

{
    "datacenter": "dc1",
    "data_dir": "/opt/consul",
    "log_level": "DEBUG",
    "node_name": "nomad-livelinux-01",
    "server": false,
    "encrypt": "your-private-key",
    "domain": "livelinux",
    "addresses": {
      "dns": "127.0.0.1",
      "https": "0.0.0.0",
      "grpc": "127.0.0.1",
      "http": "127.0.0.1"
    },
    "bind_addr": "172.30.0.5", # локальный адрес вм
    "start_join": ["172.30.0.15"], # удаленный адрес консул сервера
    "ports": {
      "dns": 53
     }

Txuag cov kev hloov pauv thiab txav mus rau kev teeb tsa cov ntaub ntawv pabcuam, nws cov ntsiab lus:

/etc/systemd/system/consul.service:

[Unit]
Description="HashiCorp Consul - A service mesh solution"
Documentation=https://www.consul.io/
Requires=network-online.target
After=network-online.target

[Service]
User=root
Group=root
ExecStart=/usr/local/bin/consul agent -config-dir=/etc/consul.d/client
ExecReload=/usr/local/bin/consul reload
KillMode=process
Restart=on-failure

[Install]
WantedBy=multi-user.target

Peb tso tus consul ntawm lub server. Tam sim no, tom qab tso tawm, peb yuav tsum pom cov kev pabcuam teeb tsa hauv nsul cov tswv cuab. Qhov no yuav txhais tau tias nws tau ua tiav txuas nrog pawg ua tus neeg siv khoom. Rov ua qhov qub ntawm tus neeg rau zaub mov thib ob thiab tom qab ntawd peb tuaj yeem pib txhim kho thiab teeb tsa Nomad.

Cov ncauj lus kom ntxaws ntxiv ntawm Nomad tau piav qhia hauv nws cov ntaub ntawv raug cai. Muaj ob txoj kev teeb tsa ib txwm muaj: rub tawm cov ntaub ntawv binary thiab sau los ntawm qhov chaw. Kuv mam li xaiv thawj txoj kev.

Примечание: Qhov project tab tom txhim kho sai heev, cov kev hloov tshiab tshiab feem ntau tso tawm. Tej zaum ib tug tshiab version yuav raug tso tawm los ntawm lub sij hawm no tsab xov xwm tiav. Yog li ntawd, ua ntej nyeem ntawv, kuv pom zoo kom kuaj xyuas Nomad version tam sim no thiab rub tawm nws. 

root@nomad-livelinux-01:~# wget https://releases.hashicorp.com/nomad/0.9.1/nomad_0.9.1_linux_amd64.zip
root@nomad-livelinux-01:~# unzip nomad_0.9.1_linux_amd64.zip
root@nomad-livelinux-01:~# mv nomad /usr/local/bin/
root@nomad-livelinux-01:~# nomad -autocomplete-install
root@nomad-livelinux-01:~# complete -C /usr/local/bin/nomad nomad
root@nomad-livelinux-01:~# mkdir /etc/nomad.d

Tom qab unpacking, peb yuav tau txais Nomad binary cov ntaub ntawv hnyav 65 MB - nws yuav tsum tau tsiv mus rau /usr/local/bin.

Cia peb tsim cov ntaub ntawv teev npe rau Nomad thiab kho nws cov ntaub ntawv pabcuam (nws feem ntau yuav tsis muaj nyob rau thaum pib):

root@nomad-livelinux-01:~# mkdir --parents /opt/nomad
root@nomad-livelinux-01:~# nano /etc/systemd/system/nomad.service

Muab cov kab hauv qab no tso rau hauv:

[Unit]
Description=Nomad
Documentation=https://nomadproject.io/docs/
Wants=network-online.target
After=network-online.target

[Service]
ExecReload=/bin/kill -HUP $MAINPID
ExecStart=/usr/local/bin/nomad agent -config /etc/nomad.d
KillMode=process
KillSignal=SIGINT
LimitNOFILE=infinity
LimitNPROC=infinity
Restart=on-failure
RestartSec=2
StartLimitBurst=3
StartLimitIntervalSec=10
TasksMax=infinity

[Install]
WantedBy=multi-user.target

Txawm li cas los xij, peb tsis maj nrawm rau tso tawm nomad - peb tseem tsis tau tsim nws cov ntaub ntawv teeb tsa:

root@nomad-livelinux-01:~# mkdir --parents /etc/nomad.d
root@nomad-livelinux-01:~# chmod 700 /etc/nomad.d
root@nomad-livelinux-01:~# nano /etc/nomad.d/nomad.hcl
root@nomad-livelinux-01:~# nano /etc/nomad.d/server.hcl

Qhov kawg directory qauv yuav raws li nram no:

/etc/nomad.d/
├── nomad.hcl
└── server.hcl

Cov ntaub ntawv nomad.hcl yuav tsum muaj cov kev teeb tsa hauv qab no:

datacenter = "dc1"
data_dir = "/opt/nomad"

Cov ntsiab lus ntawm cov ntaub ntawv server.hcl:

server {
  enabled = true
  bootstrap_expect = 1
}

consul {
  address             = "127.0.0.1:8500"
  server_service_name = "nomad"
  client_service_name = "nomad-client"
  auto_advertise      = true
  server_auto_join    = true
  client_auto_join    = true
}

bind_addr = "127.0.0.1" 

advertise {
  http = "172.30.0.5"
}

client {
  enabled = true
}

Tsis txhob hnov ​​​​qab hloov cov ntaub ntawv teeb tsa ntawm lub server thib ob - nyob ntawd koj yuav tsum tau hloov tus nqi ntawm http cov lus qhia.

Qhov kawg ntawm theem no yog teeb tsa Nginx rau kev tso npe thiab teeb tsa http kev tso cai. Cov ntsiab lus ntawm nomad.conf cov ntaub ntawv:

upstream nomad-auth {
        server 172.30.0.5:4646;
}

server {

        server_name nomad.domain.name;
        
        location / {
	        proxy_pass http://nomad-auth;
	        proxy_set_header Host $host;
	        auth_basic_user_file /etc/nginx/.htpasswd;
		   auth_basic "Password-protected Area";
        }
        
}

Tam sim no peb tuaj yeem nkag mus rau lub vev xaib vaj huam sib luag ntawm lub network sab nraud. Txuas thiab mus rau nplooj ntawv servers:

Teeb tsa Nomad pawg siv Consul thiab koom nrog Gitlab
Duab 1. Cov npe ntawm cov servers hauv Nomad pawg

Ob lub servers tau ua tiav tiav nyob rau hauv lub vaj huam sib luag, peb yuav pom tib yam hauv cov zis ntawm nomad node raws li txoj cai hais kom ua:

Teeb tsa Nomad pawg siv Consul thiab koom nrog Gitlab
Duab 2. Tso zis ntawm nomad node raws li txoj cai hais kom ua

Yuav ua li cas yog Consul? Cia peb saib. Mus rau Consul tswj vaj huam sib luag, mus rau nplooj ntawv nodes:
Teeb tsa Nomad pawg siv Consul thiab koom nrog Gitlab
Duab 3. Cov npe ntawm cov nodes hauv pawg Consul

Tam sim no peb tau npaj Nomad ua haujlwm nrog Consul. Nyob rau theem kawg, peb yuav tau mus rau qhov kev lom zem: teeb tsa kev xa khoom ntawm Docker ntim los ntawm Gitlab rau Nomad, thiab tseem tham txog qee yam ntawm nws lwm yam tshwj xeeb.

Tsim Gitlab Runner

Txhawm rau xa cov duab docker rau Nomad, peb yuav siv cov neeg khiav dej num sib cais nrog Nomad binary cov ntaub ntawv sab hauv (ntawm no, los ntawm txoj kev, peb tuaj yeem nco txog lwm yam ntawm Hashicorp daim ntawv thov - ib tus zuj zus lawv yog ib daim ntawv binary nkaus xwb). Upload nws mus rau lub runner directory. Cia peb tsim Dockerfile yooj yim rau nws nrog cov ntsiab lus hauv qab no:


FROM alpine:3.9
RUN apk add --update --no-cache libc6-compat gettext
COPY nomad /usr/local/bin/nomad

Hauv tib qhov project peb tsim .gitlab-ci.yml:

variables:
  DOCKER_IMAGE: nomad/nomad-deploy
  DOCKER_REGISTRY: registry.domain.name
 

stages:
  - build

build:
  stage: build
  image: ${DOCKER_REGISTRY}/nomad/alpine:3
  script:
    - tag=${DOCKER_REGISTRY}/${DOCKER_IMAGE}:latest
    - docker build --pull -t ${tag} -f Dockerfile .
    - docker push ${tag}

Raws li qhov tshwm sim, peb yuav muaj cov duab muaj nyob ntawm Nomad khiav hauv Gitlab Registry, tam sim no peb tuaj yeem mus ncaj qha mus rau qhov chaw cia khoom, tsim Pipeline thiab teeb tsa Nomad txoj haujlwm nomad.

Kev teeb tsa qhov project

Cia peb pib nrog txoj haujlwm cov ntaub ntawv rau Nomad. Kuv qhov project hauv kab lus no yuav yog qhov tseem ceeb heev: nws yuav muaj ib txoj haujlwm. Cov ntsiab lus ntawm .gitlab-ci yuav ua raws li hauv qab no:

variables:
  NOMAD_ADDR: http://nomad.address.service:4646
  DOCKER_REGISTRY: registry.domain.name
  DOCKER_IMAGE: example/project

stages:
  - build
  - deploy

build:
  stage: build
  image: ${DOCKER_REGISTRY}/nomad-runner/alpine:3
  script:
    - tag=${DOCKER_REGISTRY}/${DOCKER_IMAGE}:${CI_COMMIT_SHORT_SHA}
    - docker build --pull -t ${tag} -f Dockerfile .
    - docker push ${tag}


deploy:
  stage: deploy
  image: registry.example.com/nomad/nomad-runner:latest
  script:
    - envsubst '${CI_COMMIT_SHORT_SHA}' < project.nomad > job.nomad
    - cat job.nomad
    - nomad validate job.nomad
    - nomad plan job.nomad || if [ $? -eq 255 ]; then exit 255; else echo "success"; fi
    - nomad run job.nomad
  environment:
    name: production
  allow_failure: false
  when: manual

Ntawm no qhov kev xa tawm tshwm sim manually, tab sis koj tuaj yeem teeb tsa nws los hloov cov ntsiab lus ntawm qhov project directory. Cov raj xa dej muaj ob theem: kev sib dhos duab thiab nws qhov kev xa mus rau cov nom tswv. Hauv thawj theem, peb sib sau ua ke ib daim duab docker thiab thawb nws mus rau hauv peb Kev Sau Npe, thiab thaum thib ob peb pib peb txoj haujlwm hauv Nomad. 

job "monitoring-status" {
    datacenters = ["dc1"]
    migrate {
        max_parallel = 3
        health_check = "checks"
        min_healthy_time = "15s"
        healthy_deadline = "5m"
    }

    group "zhadan.ltd" {
        count = 1
        update {
            max_parallel      = 1
            min_healthy_time  = "30s"
            healthy_deadline  = "5m"
            progress_deadline = "10m"
            auto_revert       = true
        }
        task "service-monitoring" {
            driver = "docker"

            config {
                image = "registry.domain.name/example/project:${CI_COMMIT_SHORT_SHA}"
                force_pull = true
                auth {
                    username = "gitlab_user"
                    password = "gitlab_password"
                }
                port_map {
                    http = 8000
                }
            }
            resources {
                network {
                    port "http" {}
                }
            }
        }
    }
}

Thov nco ntsoov tias kuv muaj tus kheej Registry thiab kom ua tiav cov duab docker kuv yuav tsum tau nkag mus rau nws. Qhov kev daws teeb meem zoo tshaj plaws hauv qhov no yog nkag mus rau tus ID nkag mus thiab lo lus zais rau hauv Vault thiab tom qab ntawd ua ke nrog Nomad. Nomad ib txwm txhawb nqa Vault. Tab sis ua ntej, cia peb nruab cov cai tsim nyog rau Nomad hauv Vault nws tus kheej; lawv tuaj yeem rub tawm:

# Download the policy and token role
$ curl https://nomadproject.io/data/vault/nomad-server-policy.hcl -O -s -L
$ curl https://nomadproject.io/data/vault/nomad-cluster-role.json -O -s -L

# Write the policy to Vault
$ vault policy write nomad-server nomad-server-policy.hcl

# Create the token role with Vault
$ vault write /auth/token/roles/nomad-cluster @nomad-cluster-role.json

Tam sim no, tau tsim cov cai tsim nyog, peb yuav ntxiv kev koom ua ke nrog Vault hauv cov haujlwm thaiv hauv cov ntaub ntawv job.nomad:

vault {
  enabled = true
  address = "https://vault.domain.name:8200"
  token = "token"
}

Kuv siv kev tso cai los ntawm token thiab sau npe ncaj qha rau ntawm no, kuj tseem muaj cov kev xaiv ntawm kev qhia lub token ua qhov sib txawv thaum pib tus neeg sawv cev nomad:

$ VAULT_TOKEN=<token> nomad agent -config /path/to/config

Tam sim no peb tuaj yeem siv cov yuam sij nrog Vault. Lub hauv paus ntsiab lus ntawm kev ua haujlwm yog qhov yooj yim: peb tsim cov ntaub ntawv hauv Nomad txoj haujlwm uas yuav khaws cov txiaj ntsig ntawm qhov sib txawv, piv txwv li:

template {
                data = <<EOH
{{with secret "secrets/pipeline-keys"}}
REGISTRY_LOGIN="{{ .Data.REGISTRY_LOGIN }}"
REGISTRY_PASSWORD="{{ .Data.REGISTRY_LOGIN }}{{ end }}"

EOH
    destination = "secrets/service-name.env"
    env = true
}

Nrog rau txoj hauv kev yooj yim no, koj tuaj yeem teeb tsa kev xa khoom ntim rau Nomad pawg thiab ua haujlwm nrog nws yav tom ntej. Kuv yuav hais tias rau qee qhov kuv ua siab ntev rau Nomad - nws yog qhov tsim nyog rau cov haujlwm me me uas Kubernetes tuaj yeem ua rau muaj kev nyuaj ntxiv thiab yuav tsis paub nws lub peev xwm tag nrho. Ntxiv rau, Nomad yog zoo meej rau cov pib tshiab - nws yooj yim rau nruab thiab teeb tsa. Txawm li cas los xij, thaum kuaj ntawm qee qhov haujlwm, kuv ntsib teeb meem nrog nws cov qauv thaum ntxov - ntau lub luag haujlwm yooj yim tsuas yog tsis nyob ntawd lossis lawv tsis ua haujlwm raug. Txawm li cas los xij, kuv ntseeg tias Nomad yuav txhim kho txuas ntxiv thiab yav tom ntej nws yuav tau txais cov haujlwm uas txhua tus xav tau.

Sau: Ilya Andreev, kho los ntawm Alexey Zhadan thiab pab neeg nyob Linux


Tau qhov twg los: www.hab.com

Ntxiv ib saib