ProHoster > Blog > Tsamaiso > Ho theha sehlopha sa Nomad ho sebelisa Consul le ho kopanya le Gitlab

Ho theha sehlopha sa Nomad ho sebelisa Consul le ho kopanya le Gitlab

Selelekela

Haufinyane tjena, botumo ba Kubernetes bo ntse bo eketseha ka potlako - merero e mengata e ntse e e kenya ts'ebetsong. Ke ne ke batla ho ama 'mino oa liletsa o kang Nomad: e nepahetse bakeng sa merero e seng e ntse e sebelisa litharollo tse ling tse tsoang ho HashiCorp, mohlala, Vault le Consul,' me merero ka boeona ha e rarahane ho latela mekhoa ea motheo. Boitsebiso bona bo tla ba le litaelo tsa ho kenya Nomad, ho kopanya li-node tse peli ka sehlopha, hammoho le ho kopanya Nomad le Gitlab.

Ho theha sehlopha sa Nomad ho sebelisa Consul le ho kopanya le Gitlab

teko ea teko

Hanyenyane ka benche ea teko: li-server tse tharo tsa sebele li sebelisoa ka litšobotsi tsa 2 CPU, 4 RAM, 50 Gb SSD, tse kopantsoeng ho ba marang-rang a tloaelehileng a sebakeng seo. Mabitso a bona le liaterese tsa IP:

  1. nomad-livelinux-01: 172.30.0.5
  2. nomad-livelinux-02: 172.30.0.10
  3. Consul-livelinux-01: 172.30.0.15

Ho kenngoa ha Nomad, Consul. Ho theha sehlopha sa Nomad

A re qaleng ka ho instola tse motheo. Leha ho hlophisoa ho ne ho le bonolo, ke tla e hlalosa molemong oa bots'epehi ba sengoloa: ha e le hantle se entsoe ho tsoa ho lingoloa le lintlha bakeng sa ho fihlella kapele ha ho hlokahala.

Pele re qala ho ikoetlisa, re tla tšohla karolo ea theory, hobane sethaleng sena ke habohlokoa ho utloisisa sebopeho sa nakong e tlang.

Re na le li-node tse peli tsa nomad 'me re batla ho li kopanya hore e be sehlopha,' me nakong e tlang re tla boela re hloke "automatically cluster scaling" - bakeng sa sena re tla hloka Consul. Ka sesebelisoa sena, ho kopanya le ho eketsa li-node tse ncha ho fetoha mosebetsi o bonolo haholo: node e bōpiloeng ea Nomad e hokahanya le moemeli oa Consul, ebe e hokahanya le sehlopha se teng sa Nomad. Ka hona, qalong re tla kenya seva sa Consul, re hlophise tumello ea mantlha ea http bakeng sa sebaka sa marang-rang (ha se na tumello ka boiketsetso mme se ka fumaneha atereseng ea kantle), hammoho le baemeli ba Consul ka bobona ho li-server tsa Nomad, kamora moo. re tla fetela ho Nomad feela.

Ho kenya lisebelisoa tsa HashiCorp ho bonolo haholo: ha e le hantle, re tsamaisa faele ea binary bukeng ea bin, re theha faele ea tlhophiso ea sesebelisoa, 'me re thehe faele ea eona ea ts'ebeletso.

Khoasolla faele ea binary ea Consul 'me u e lokolle bukeng ea lapeng ea mosebelisi:

root@consul-livelinux-01:~# wget https://releases.hashicorp.com/consul/1.5.0/consul_1.5.0_linux_amd64.zip
root@consul-livelinux-01:~# unzip consul_1.5.0_linux_amd64.zip
root@consul-livelinux-01:~# mv consul /usr/local/bin/

Hona joale re na le "consul binary" e lokiselitsoeng hantle bakeng sa tlhophiso e eketsehileng.

Ho sebetsa le Consul, re hloka ho theha senotlolo se ikhethileng re sebelisa taelo ea keygen:

root@consul-livelinux-01:~# consul keygen

Ha re tsoeleng pele ho theha tlhophiso ea Consul, ho theha directory /etc/consul.d/ ka sebopeho se latelang:

/etc/consul.d/
├── bootstrap
│   └── config.json

Buka ea bootstrap e tla ba le faele ea configuration config.json - ho eona re tla beha litlhophiso tsa Consul. Litaba tsa eona:

{
"bootstrap": true,
"server": true,
"datacenter": "dc1",
"data_dir": "/var/consul",
"encrypt": "your-key",
"log_level": "INFO",
"enable_syslog": true,
"start_join": ["172.30.0.15"]
}

Ha re shebeng litaelo tsa mantlha le meelelo ea tsona ka thoko:

  • bootstrap: nnete. Re nolofalletsa ho eketsa li-node tse ncha haeba li hokahane. Kea hlokomela hore ha re bontše mona palo e nepahetseng ea li-node tse lebelletsoeng.
  • seva: nnete. Lumella mokhoa oa seva. Consul mochining ona o sebetsang e tla sebetsa e le eona feela seva le master hajoale, VM ea Nomad e tla ba bareki.
  • Datacenterke: dc1. Hlalosa lebitso la setsi sa data ho theha sehlopha. E tlameha ho tšoana ho bareki le li-server.
  • ho kenyeletsa: senotlolo sa hau. Senotlolo, seo le sona se tlamehang ho ikhetha le ho bapisa bareki bohle le li-server. E hlahisoa ho sebelisoa taelo ea consul keygen.
  • qala_join. Lethathamong lena re bonts'a lethathamo la liaterese tsa IP tseo khokahano e tla etsoa ho tsona. Hajoale re siea aterese ea rona feela.

Mothating ona re ka matha consul re sebelisa mola oa taelo:

root@consul-livelinux-01:~# /usr/local/bin/consul agent -config-dir /etc/consul.d/bootstrap -ui

Ena ke mokhoa o motle oa ho lokisa bothata hona joale, leha ho le joalo, u ke ke ua khona ho sebelisa mokhoa ona kamehla ka mabaka a hlakileng. Ha re theheng faele ea litšebeletso ho laola Consul ka systemd:

root@consul-livelinux-01:~# nano /etc/systemd/system/consul.service

Litaba tsa faele ea consul.service: 

[Unit]
Description=Consul Startup process
After=network.target
 
[Service]
Type=simple
ExecStart=/bin/bash -c '/usr/local/bin/consul agent -config-dir /etc/consul.d/bootstrap -ui' 
TimeoutStartSec=0
 
[Install]
WantedBy=default.target

Qala Consul ka systemctl:

root@consul-livelinux-01:~# systemctl start consul

Ha re hlahlobeng: tšebeletso ea rona e tlameha ho sebetsa, 'me ka ho phethahatsa taelo ea litho tsa consul re lokela ho bona seva sa rona:

root@consul-livelinux:/etc/consul.d# consul members
consul-livelinux    172.30.0.15:8301  alive   server  1.5.0  2         dc1  <all>

Mohato o latelang: ho kenya Nginx le ho theha proxying le tumello ea http. Re kenya nginx ka mookameli oa sephutheloana mme ka har'a /etc/nginx/site-enabled directory re theha faele ea consul.conf e nang le litaba tse latelang:

upstream consul-auth {
    server localhost:8500;
}

server {

    server_name consul.doman.name;
    
    location / {
      proxy_pass http://consul-auth;
      proxy_set_header Host $host;
      auth_basic_user_file /etc/nginx/.htpasswd;
      auth_basic "Password-protected Area";
    }
}

U se ke ua lebala ho etsa faele ea .htpasswd le ho hlahisa lebitso la hau le password bakeng sa eona. Ntho ena ea hlokahala hore sebaka sa marang-rang se se fumanehe ho bohle ba tsebang sebaka sa rona. Leha ho le joalo, ha re theha Gitlab, re tla tlameha ho lahla sena - ho seng joalo re ke ke ra khona ho isa kopo ea rona ho Nomad. Morerong oa ka, Gitlab le Nomad li ho webosaete ea bohlooho feela, kahoo ha ho na bothata bo joalo mona.

Ho li-server tse peli tse setseng re kenya baemeli ba Consul ho latela litaelo tse latelang. Re pheta mehato ka faele ea binary:

root@nomad-livelinux-01:~# wget https://releases.hashicorp.com/consul/1.5.0/consul_1.5.0_linux_amd64.zip
root@nomad-livelinux-01:~# unzip consul_1.5.0_linux_amd64.zip
root@nomad-livelinux-01:~# mv consul /usr/local/bin/

Ka papiso le seva e fetileng, re theha directory bakeng sa lifaele tsa tlhophiso /etc/consul.d ka sebopeho se latelang:

/etc/consul.d/
├── client
│   └── config.json

Litaba tsa faele ea config.json:

{
    "datacenter": "dc1",
    "data_dir": "/opt/consul",
    "log_level": "DEBUG",
    "node_name": "nomad-livelinux-01",
    "server": false,
    "encrypt": "your-private-key",
    "domain": "livelinux",
    "addresses": {
      "dns": "127.0.0.1",
      "https": "0.0.0.0",
      "grpc": "127.0.0.1",
      "http": "127.0.0.1"
    },
    "bind_addr": "172.30.0.5", # локальный адрес вм
    "start_join": ["172.30.0.15"], # удаленный адрес консул сервера
    "ports": {
      "dns": 53
     }

Boloka liphetoho 'me u tsoele pele ho seta faele ea litšebeletso, likahare tsa eona:

/etc/systemd/system/consul.service:

[Unit]
Description="HashiCorp Consul - A service mesh solution"
Documentation=https://www.consul.io/
Requires=network-online.target
After=network-online.target

[Service]
User=root
Group=root
ExecStart=/usr/local/bin/consul agent -config-dir=/etc/consul.d/client
ExecReload=/usr/local/bin/consul reload
KillMode=process
Restart=on-failure

[Install]
WantedBy=multi-user.target

Re qala consul ho seva. Joale, kamora ho qala, re lokela ho bona ts'ebeletso e hlophisitsoeng ho litho tsa nsul. Sena se tla bolela hore e hokahane ka katleho le sehlopha joalo ka moreki. Pheta se tšoanang ho seva sa bobeli mme ka mor'a moo re ka qala ho kenya le ho lokisa Nomad.

Ho kenya Nomad ka botlalo ho hlalositsoe litokomaneng tsa eona tsa semmuso. Ho na le mekhoa e 'meli ea setso ea ho kenya: ho khoasolla faele ea binary le ho bokella ho tsoa mohloling. Ke tla khetha mokhoa oa pele.

mantsoe: Morero o ntse o tsoela pele ka potlako haholo, hangata ho lokolloa lintlafatso tse ncha. Mohlomong phetolelo e ncha e tla lokolloa ha sengoloa sena se phetheloa. Ka hona, pele ke bala, ke khothaletsa ho sheba mofuta oa hajoale oa Nomad hona joale le ho o jarolla. 

root@nomad-livelinux-01:~# wget https://releases.hashicorp.com/nomad/0.9.1/nomad_0.9.1_linux_amd64.zip
root@nomad-livelinux-01:~# unzip nomad_0.9.1_linux_amd64.zip
root@nomad-livelinux-01:~# mv nomad /usr/local/bin/
root@nomad-livelinux-01:~# nomad -autocomplete-install
root@nomad-livelinux-01:~# complete -C /usr/local/bin/nomad nomad
root@nomad-livelinux-01:~# mkdir /etc/nomad.d

Kamora ho manolla, re tla fumana faele ea binary ea Nomad e boima ba 65 MB - e tlameha ho isoa ho /usr/local/bin.

Ha re theheng lethathamo la data bakeng sa Nomad mme re hlophise faele ea eona ea ts'ebeletso (mohlomong e ke ke ea ba teng qalong):

root@nomad-livelinux-01:~# mkdir --parents /opt/nomad
root@nomad-livelinux-01:~# nano /etc/systemd/system/nomad.service

Beha mela e latelang moo:

[Unit]
Description=Nomad
Documentation=https://nomadproject.io/docs/
Wants=network-online.target
After=network-online.target

[Service]
ExecReload=/bin/kill -HUP $MAINPID
ExecStart=/usr/local/bin/nomad agent -config /etc/nomad.d
KillMode=process
KillSignal=SIGINT
LimitNOFILE=infinity
LimitNPROC=infinity
Restart=on-failure
RestartSec=2
StartLimitBurst=3
StartLimitIntervalSec=10
TasksMax=infinity

[Install]
WantedBy=multi-user.target

Leha ho le joalo, ha re potlakele ho qala nomad - ha re so thehe faele ea eona ea tlhophiso:

root@nomad-livelinux-01:~# mkdir --parents /etc/nomad.d
root@nomad-livelinux-01:~# chmod 700 /etc/nomad.d
root@nomad-livelinux-01:~# nano /etc/nomad.d/nomad.hcl
root@nomad-livelinux-01:~# nano /etc/nomad.d/server.hcl

Sebopeho sa ho qetela sa directory se tla ba ka tsela e latelang:

/etc/nomad.d/
├── nomad.hcl
└── server.hcl

Faele ea nomad.hcl e tlameha ho ba le litlhophiso tse latelang:

datacenter = "dc1"
data_dir = "/opt/nomad"

Likahare tsa faele ea server.hcl:

server {
  enabled = true
  bootstrap_expect = 1
}

consul {
  address             = "127.0.0.1:8500"
  server_service_name = "nomad"
  client_service_name = "nomad-client"
  auto_advertise      = true
  server_auto_join    = true
  client_auto_join    = true
}

bind_addr = "127.0.0.1" 

advertise {
  http = "172.30.0.5"
}

client {
  enabled = true
}

U se ke ua lebala ho fetola faele ea tlhophiso ho seva sa bobeli - moo o tla hloka ho fetola boleng ba taelo ea http.

Ntho ea ho qetela mothating ona ke ho lokisa Nginx bakeng sa proxying le ho theha tumello ea http. Litaba tsa faele ea nomad.conf:

upstream nomad-auth {
        server 172.30.0.5:4646;
}

server {

        server_name nomad.domain.name;
        
        location / {
	        proxy_pass http://nomad-auth;
	        proxy_set_header Host $host;
	        auth_basic_user_file /etc/nginx/.htpasswd;
		   auth_basic "Password-protected Area";
        }
        
}

Hona joale re khona ho fihlella marang-rang a marang-rang ka marang-rang a kantle. Hokela 'me u ee leqepheng la li-server:

Ho theha sehlopha sa Nomad ho sebelisa Consul le ho kopanya le Gitlab
Setšoantšo sa 1. Lethathamo la li-server ho sehlopha sa Nomad

Li-server ka bobeli li bonts'oa ka katleho phanele, re tla bona ntho e ts'oanang ho tlhahiso ea taelo ea boemo ba nomad node:

Ho theha sehlopha sa Nomad ho sebelisa Consul le ho kopanya le Gitlab
Setšoantšo sa 2. Sephetho sa taelo ea boemo ba node ea nomad

Ho thoe'ng ka Consul? Ha re shebeng. Eya ho sehlopha sa taolo sa Consul, leqepheng la nodes:
Ho theha sehlopha sa Nomad ho sebelisa Consul le ho kopanya le Gitlab
Setšoantšo sa 3. Lethathamo la li-node sehlopheng sa Consul

Hona joale re na le Nomad e lokiselitsoeng e sebetsang hammoho le Consul. Boemong ba ho qetela, re tla fihla karolong e monate: ho theha lijana tsa Docker ho tloha Gitlab ho ea Nomad, le ho bua ka tse ling tsa likarolo tsa eona tse ikhethang.

Ho theha Gitlab Runner

Ho romela litšoantšo tsa docker ho Nomad, re tla sebelisa semathi se arohaneng le faele ea binary ea Nomad ka hare (mona, ka tsela, re ka hlokomela tšobotsi e 'ngoe ea likopo tsa Hashicorp - ka bomong ke faele e le' ngoe ea binary). E kenye bukeng ea semathi. Ha re e etsetseng Dockerfile e bonolo ka litaba tse latelang:


FROM alpine:3.9
RUN apk add --update --no-cache libc6-compat gettext
COPY nomad /usr/local/bin/nomad

Mosebetsing o tšoanang re theha .gitlab-ci.yml:

variables:
  DOCKER_IMAGE: nomad/nomad-deploy
  DOCKER_REGISTRY: registry.domain.name
 

stages:
  - build

build:
  stage: build
  image: ${DOCKER_REGISTRY}/nomad/alpine:3
  script:
    - tag=${DOCKER_REGISTRY}/${DOCKER_IMAGE}:latest
    - docker build --pull -t ${tag} -f Dockerfile .
    - docker push ${tag}

Ka lebaka leo, re tla ba le setšoantšo se fumanehang sa semathi sa Nomad ho Registry ea Gitlab, joale re ka ea ka ho toba sebakeng sa polokelo ea morero, ra theha Pipeline le ho lokisa mosebetsi oa Nomad.

Ho hlophisoa ha morero

Ha re qale ka faele ea mosebetsi bakeng sa Nomad. Morero oa ka sehloohong sena e tla ba oa khale haholo: o tla ba le mosebetsi o le mong. Litaba tsa .gitlab-ci li tla ba ka tsela e latelang:

variables:
  NOMAD_ADDR: http://nomad.address.service:4646
  DOCKER_REGISTRY: registry.domain.name
  DOCKER_IMAGE: example/project

stages:
  - build
  - deploy

build:
  stage: build
  image: ${DOCKER_REGISTRY}/nomad-runner/alpine:3
  script:
    - tag=${DOCKER_REGISTRY}/${DOCKER_IMAGE}:${CI_COMMIT_SHORT_SHA}
    - docker build --pull -t ${tag} -f Dockerfile .
    - docker push ${tag}


deploy:
  stage: deploy
  image: registry.example.com/nomad/nomad-runner:latest
  script:
    - envsubst '${CI_COMMIT_SHORT_SHA}' < project.nomad > job.nomad
    - cat job.nomad
    - nomad validate job.nomad
    - nomad plan job.nomad || if [ $? -eq 255 ]; then exit 255; else echo "success"; fi
    - nomad run job.nomad
  environment:
    name: production
  allow_failure: false
  when: manual

Mona ho tsamaisoa ho etsahala ka letsoho, empa u ka e hlophisa ho fetola litaba tsa buka ea morero. Pipeline e na le mekhahlelo e 'meli: kopano ea litšoantšo le ho romelloa ha eona ho nomad. Mothating oa pele, re bokella setšoantšo sa docker ebe re se sutumelletsa ho Registry ea rona, mme ka lekhetlo la bobeli re qala mosebetsi oa rona ho Nomad. 

job "monitoring-status" {
    datacenters = ["dc1"]
    migrate {
        max_parallel = 3
        health_check = "checks"
        min_healthy_time = "15s"
        healthy_deadline = "5m"
    }

    group "zhadan.ltd" {
        count = 1
        update {
            max_parallel      = 1
            min_healthy_time  = "30s"
            healthy_deadline  = "5m"
            progress_deadline = "10m"
            auto_revert       = true
        }
        task "service-monitoring" {
            driver = "docker"

            config {
                image = "registry.domain.name/example/project:${CI_COMMIT_SHORT_SHA}"
                force_pull = true
                auth {
                    username = "gitlab_user"
                    password = "gitlab_password"
                }
                port_map {
                    http = 8000
                }
            }
            resources {
                network {
                    port "http" {}
                }
            }
        }
    }
}

Ka kopo elelloa hore ke na le Registry ea poraefete le ho hula setšoantšo sa docker ka katleho ke hloka ho kena ho sona. Tharollo e molemohali tabeng ena ke ho kenya ho kena le password ho Vault ebe o e kopanya le Nomad. Nomad ka tlhaho o tšehetsa Vault. Empa pele, ho Vault ka boeona, re tla kenya maano a hlokahalang bakeng sa Nomad, o ka a khoasolla:

# Download the policy and token role
$ curl https://nomadproject.io/data/vault/nomad-server-policy.hcl -O -s -L
$ curl https://nomadproject.io/data/vault/nomad-cluster-role.json -O -s -L

# Write the policy to Vault
$ vault policy write nomad-server nomad-server-policy.hcl

# Create the token role with Vault
$ vault write /auth/token/roles/nomad-cluster @nomad-cluster-role.json

Joale, ha re se re thehile maano a hlokahalang, re tla eketsa kopanyo le Vault sebakeng sa mosebetsi ho file ea job.nomad:

vault {
  enabled = true
  address = "https://vault.domain.name:8200"
  token = "token"
}

Ke sebelisa tumello ka tokeneng mme ke e ngolisa ka kotloloho mona, ho boetse ho na le khetho ea ho hlakisa lets'oao joalo ka phetoho ha ke qala moemeli oa nomad:

$ VAULT_TOKEN=<token> nomad agent -config /path/to/config

Joale re ka sebelisa linotlolo ka Vault. Molao-motheo oa ts'ebetso o bonolo: re theha faele mosebetsing oa Nomad o tla boloka boleng ba mefuta e fapaneng, mohlala:

template {
                data = <<EOH
{{with secret "secrets/pipeline-keys"}}
REGISTRY_LOGIN="{{ .Data.REGISTRY_LOGIN }}"
REGISTRY_PASSWORD="{{ .Data.REGISTRY_LOGIN }}{{ end }}"

EOH
    destination = "secrets/service-name.env"
    env = true
}

Ka mokhoa ona o bonolo, o ka hlophisa phepelo ea lijana ho sehlopha sa Nomad mme o sebetse le eona nakong e tlang. Ke tla re ho isa tekanyong e itseng ke utloela Nomad bohloko - e loketse haholoanyane merero e menyenyane moo Kubernetes e ka bakang ho rarahana ho eketsehileng 'me e ke ke ea hlokomela bokhoni ba eona bo feletseng. Hape, Nomad e nepahetse bakeng sa ba qalang — ho bonolo ho e kenya le ho e hlophisa. Leha ho le joalo, ha ke etsa liteko mererong e meng, ke kopana le bothata ka liphetolelo tsa eona tsa pele - mesebetsi e mengata ea mantlha ha e eo kapa ha e sebetse hantle. Leha ho le joalo, ke lumela hore Nomad e tla tsoela pele ho nts'etsapele mme nakong e tlang e tla fumana mesebetsi eo motho e mong le e mong a e hlokang.

Mongoli: Ilya Andreev, e hlophisitsoeng ke Alexey Zhadan le sehlopha sa Live Linux


Source: www.habr.com

Eketsa ka tlhaloso