🥇 Nyefee ngwa site na iji Docker Swarm

Sistemụ ntụnye ọdịnaya vidiyo dị n'ịntanetị anyị na-arụ ọrụ bụ mmepe azụmahịa mechiri emechi ma bụrụ teknụzụ nwere ọtụtụ akụkụ nke ihe nwe ya na ihe mepere emepe. Ebumnuche nke ide edemede a bụ ịkọwa mmejuputa usoro nchịkọta docker swarm maka ikpo okwu na-emepe emepe, na-akpaghasị usoro ọrụ nke usoro anyị n'okpuru obere oge. A na-ekewa akụkọ a na-egosi n'uche gị ụzọ abụọ. Akụkụ nke mbụ na-akọwa CI/CD tupu iji docker swarm, na akụkụ nke abụọ na-akọwa usoro nke mmejuputa ya. Ndị na-enweghị mmasị ịgụ akụkụ nke mbụ nwere ike ịga n'ihu na nke abụọ n'enweghị nsogbu.

Nkebi nke XNUMX

N'otu oge, ọ dị mkpa ịtọlite usoro CI/CD ngwa ngwa o kwere mee. Otu n'ime ọnọdụ ahụ abụghị iji Docker maka ntinye A na-emepụta akụrụngwa maka ọtụtụ ihe kpatara ya:

maka ịrụ ọrụ ntụkwasị obi ma kwụsie ike nke akụrụngwa na Production (ya bụ, n'ezie, ihe achọrọ ka ịghara iji virtualization)
Ndị mmepe na-eduga achọghị ịrụ ọrụ na Docker (ọ dị ịtụnanya, mana otu ahụ ka ọ dị)
maka echiche echiche nke njikwa R&D

Akụrụngwa, nchịkọta na ihe mbụ chọrọ maka MVP bụ ndị a:

4 Intel® X5650 sava nwere Debian (otu igwe dị ike karịa maka mmepe)
A na-eme mmepe nke akụrụngwa omenala gị na C++, Python3
Ihe eji eme ihe nke atọ nke atọ: Kafka, Clickhouse, Airflow, Redis, Grafana, Postgresql, Mysql,…
Pipeline maka iwulite na ịnwale ihe dị iche iche maka ndozi na ntọhapụ

Otu n'ime ajụjụ mbụ ekwesịrị idozi na mbido mbụ bụ ka a ga-esi tinye ngwa ọdịnala na gburugburu ebe ọ bụla (CI/CD).

Anyị kpebiri ịwụnye ihe ndị ọzọ n'usoro ma melite ha n'usoro. Ngwa omenala emepụtara na C++ ma ọ bụ Python nwere ike ibunye n'ọtụtụ ụzọ. N'ime ha, dịka ọmụmaatụ: ịmepụta ngwugwu usoro, na-eziga ha na ebe nchekwa nke ihe oyiyi anakọtara na ntinye ha na-esote na sava. Maka ihe a na-amaghị ama, a họọrọ usoro ọzọ, ya bụ: iji CI, a na-achịkọta faịlụ ndị nwere ike ime ngwa ngwa, a na-emepụta gburugburu ebe obibi ọrụ, py modules from requirements.txt na-ezigara, na-ezigara ihe ndị a niile na nhazi, scripts na. gburugburu ngwa ngwa na sava. Na-esote, ewepụtara ngwa site na onye ọrụ mebere na-enweghị ikike nchịkwa.

A họọrọ Gitlab-CI ka usoro CI/CD. Pipeline nke si na ya pụta dị ka nke a:

N'usoro, gitlab-ci.yml dị ka nke a:

---
variables:
  # минимальная версия ЦПУ на серверах, где разворачивается кластер
  CMAKE_CPUTYPE: "westmere"

  DEBIAN: "MYREGISTRY:5000/debian:latest"

before_script:
  - eval $(ssh-agent -s)
  - ssh-add <(echo "$SSH_PRIVATE_KEY")
  - mkdir -p ~/.ssh && echo -e "Host *ntStrictHostKeyChecking nonn" > ~/.ssh/config

stages:
  - build
  - testing
  - deploy

debug.debian:
  stage: build
  image: $DEBIAN
  script:
    - cd builds/release && ./build.sh
    paths:
      - bin/
      - builds/release/bin/
    when: always
release.debian:
  stage: build
  image: $DEBIAN
  script:
    - cd builds/release && ./build.sh
    paths:
      - bin/
      - builds/release/bin/
    when: always

## testing stage
tests.codestyle:
  stage: testing
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - /bin/bash run_tests.sh -t codestyle -b "${CI_COMMIT_REF_NAME}_codestyle"
tests.debug.debian:
  stage: testing
  image: $DEBIAN
  dependencies:
    - debug.debian
  script:
    - /bin/bash run_tests.sh -e codestyle/test_pylint.py -b "${CI_COMMIT_REF_NAME}_debian_debug"
  artifacts:
    paths:
      - run_tests/username/
    when: always
    expire_in: 1 week
tests.release.debian:
  stage: testing
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - /bin/bash run_tests.sh -e codestyle/test_pylint.py -b "${CI_COMMIT_REF_NAME}_debian_release"
  artifacts:
    paths:
      - run_tests/username/
    when: always
    expire_in: 1 week

## staging stage
deploy_staging:
  stage: deploy
  environment: staging
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - cd scripts/deploy/ &&
        python3 createconfig.py -s $CI_ENVIRONMENT_NAME &&
        /bin/bash install_venv.sh -d -r ../../requirements.txt &&
        python3 prepare_init.d.py &&
        python3 deploy.py -s $CI_ENVIRONMENT_NAME
  when: manual

Ọ dị mma ịmara na a na-eme mgbakọ na nnwale na onyonyo nke ya, ebe etinyerelarị ngwugwu usoro niile dị mkpa na ntọala ndị ọzọ.

Ọ bụ ezie na nke ọ bụla n'ime edemede ndị a na ọrụ na-adọrọ mmasị n'ụzọ nke ya, n'ezie agaghị m ekwu banyere ha; ịkọwa nke ọ bụla n'ime ha ga-ewe oge buru ibu ma nke a abụghị nzube nke isiokwu ahụ. Ka m dọọrọ uche gị gaa n'eziokwu ahụ bụ na ogbo mbunye nwere usoro nke edemede ọkpụkpọ:

mepụtaconfig.py - mepụta faịlụ settings.ini nwere ntọala maka akụrụngwa dị na gburugburu ebe dị iche iche maka ntinye na-esote (mmepụta, Mmepụta, Nnwale, ...)
install_venv.sh - na-emepụta ebe mebere py components n'ime akwụkwọ ndekọ aha ma detuo ya na sava dịpụrụ adịpụ
kwadebe_init.d.py - na-akwado scripts maka mmalite nkwụsị akụrụngwa dabere na ndebiri
tinye.py - deploys na malitegharịa ọhụrụ components

Oge gafere. A na-eji mmalite na mmepụta dochie ọkwa nhazi. Agbakwunyela nkwado maka ngwaahịa a na nkesa ọzọ (CentOS). Agbakwunyere sava anụ ahụ 5 dị ike yana iri na abuo mebere. Ma ọ bịara na-esiwanye ike maka ndị mmepe na ndị nyocha ịnwale ọrụ ha na gburugburu ma ọ bụ obere nso na steeti ọrụ. N'oge a, ọ bịara doo anya na ọ gaghị ekwe omume ime na-enweghị ya ...

Nkebi nke II

Yabụ, ụyọkọ anyị bụ usoro dị egwu nke ihe mejupụtara ihe iri na abuo nke Dockerfiles kọwaraghị ya. Ị nwere ike hazie ya maka itinye ya na gburugburu ebe obibi naanị n'ozuzu ya. Ọrụ anyị bụ ibunye ụyọkọ ahụ n'ime ebe a na-eme atụmatụ iji nwalee ya tupu nnwale tupu ahapụpụta ya.

N'usoro iwu, enwere ike inwe ọtụtụ ụyọkọ na-arụ ọrụ n'otu oge: ka enwere ọrụ na steeti emechara ma ọ bụ na-eru nso. Ike nke sava dị n'aka anyị na-enye anyị ohere ịme ọtụtụ ụyọkọ na sava ọ bụla. Ụyọkọ nhazi ọ bụla ga-anọpụ iche (e kwesịghị inwe nkpuchi na ọdụ ụgbọ mmiri, akwụkwọ ndekọ aha, wdg).

Ihe onwunwe anyị kacha baa uru bụ oge anyị, ma anyị enweghị ọtụtụ ihe.

Maka nmalite ngwa ngwa, anyị họọrọ Docker Swarm n'ihi ịdị mfe ya na ụkpụrụ ụlọ na-agbanwe agbanwe. Ihe mbụ anyị mere bụ ịmepụta onye njikwa na ọtụtụ ọnụ na sava dịpụrụ adịpụ:

$ docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
kilqc94pi2upzvabttikrfr5d     nop-test-1     Ready               Active                                  19.03.2
jilwe56pl2zvabupryuosdj78     nop-test-2     Ready               Active                                  19.03.2
j5a4yz1kr2xke6b1ohoqlnbq5 *   nop-test-3     Ready               Active              Leader              19.03.2

Ọzọ, anyị mepụtara netwọk:


$ docker network create --driver overlay --subnet 10.10.10.0/24 nw_swarm

Ọzọ, anyị jikọtara Gitlab-CI na Swarm nodes n'ihe gbasara njikwa anya nke ọnụ site na CI: ịwụnye asambodo, ịtọ mgbanwe nzuzo, yana ịtọlite ọrụ Docker na sava njikwa. Nke a otu isiokwu zoputara anyi otutu oge.

Ọzọ, anyị gbakwunyere ọrụ maka ịmepụta na ibibi nchịkọta na .gitlab-ci .yml.

agbakwunyere ọtụtụ ọrụ na .gitlab-ci .yml

## staging stage
deploy_staging:
  stage: testing
  before_script:
    - echo "override global 'before_script'"
  image: "REGISTRY:5000/docker:latest"
  environment: staging
  dependencies: []
  variables:
    DOCKER_CERT_PATH: "/certs"
    DOCKER_HOST: tcp://10.50.173.107:2376
    DOCKER_TLS_VERIFY: 1
    CI_BIN_DEPENDENCIES_JOB: "release.centos.7"
  script:
    - mkdir -p $DOCKER_CERT_PATH
    - echo "$TLSCACERT" > $DOCKER_CERT_PATH/ca.pem
    - echo "$TLSCERT" > $DOCKER_CERT_PATH/cert.pem
    - echo "$TLSKEY" > $DOCKER_CERT_PATH/key.pem
    - docker stack deploy -c docker-compose.yml ${CI_ENVIRONMENT_NAME}_${CI_COMMIT_REF_NAME} --with-registry-auth
    - rm -rf $DOCKER_CERT_PATH
  when: manual

## stop staging stage
stop_staging:
  stage: testing
  before_script:
    - echo "override global 'before_script'"
  image: "REGISTRY:5000/docker:latest"
  environment: staging
  dependencies: []
  variables:
    DOCKER_CERT_PATH: "/certs"
    DOCKER_HOST: tcp://10.50.173.107:2376
    DOCKER_TLS_VERIFY: 1
  script:
    - mkdir -p $DOCKER_CERT_PATH
    - echo "$TLSCACERT" > $DOCKER_CERT_PATH/ca.pem
    - echo "$TLSCERT" > $DOCKER_CERT_PATH/cert.pem
    - echo "$TLSKEY" > $DOCKER_CERT_PATH/key.pem
    - docker stack rm ${CI_ENVIRONMENT_NAME}_${CI_COMMIT_REF_NAME}
    # TODO: need check that stopped
  when: manual

Site na mpempe koodu ahụ dị n'elu, o doro anya na agbakwunyere bọtịnụ abụọ na Pipelines (deploy_staging, stop_staging) nke chọrọ ọrụ aka.

Aha nchịkọta ahụ dabara na aha alaka ụlọ ọrụ na nke a pụrụ iche kwesịrị ezuru. Ọrụ ndị dị na ngwugwu ahụ na-enweta adreesị IP pụrụ iche, yana ọdụ ụgbọ mmiri, akwụkwọ ndekọ aha, wdg. ga-anọpụ iche, mana otu ihe ahụ site na nchịkọta ruo na nchịkọta (ebe ọ bụ na faịlụ nhazi bụ otu maka nchịkọta niile) - nke ahụ bụ ihe anyị chọrọ. Anyị na-ebuga tojupụtara (ụyọkọ) na-eji docker-compose.yml, nke na-akọwa ụyọkọ anyị.

docker-compose.yml

---
version: '3'

services:
  userprop:
    image: redis:alpine
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:
  celery_bcd:
    image: redis:alpine
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  schedulerdb:
    image: mariadb:latest
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
      MYSQL_DATABASE: schedulerdb
      MYSQL_USER: ****
      MYSQL_PASSWORD: ****
    command: ['--character-set-server=utf8mb4', '--collation-server=utf8mb4_unicode_ci', '--explicit_defaults_for_timestamp=1']
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  celerydb:
    image: mariadb:latest
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
      MYSQL_DATABASE: celerydb
      MYSQL_USER: ****
      MYSQL_PASSWORD: ****
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  cluster:
    image: $CENTOS7
    environment:
      - CENTOS
      - CI_ENVIRONMENT_NAME
      - CI_API_V4_URL
      - CI_REPOSITORY_URL
      - CI_PROJECT_ID
      - CI_PROJECT_URL
      - CI_PROJECT_PATH
      - CI_PROJECT_NAME
      - CI_COMMIT_REF_NAME
      - CI_BIN_DEPENDENCIES_JOB
    command: >
      sudo -u myusername -H /bin/bash -c ". /etc/profile &&
        mkdir -p /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME &&
        cd /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME &&
            git clone -b $CI_COMMIT_REF_NAME $CI_REPOSITORY_URL . &&
            curl $CI_API_V4_URL/projects/$CI_PROJECT_ID/jobs/artifacts/$CI_COMMIT_REF_NAME/download?job=$CI_BIN_DEPENDENCIES_JOB -o artifacts.zip &&
            unzip artifacts.zip ;
        cd /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME/scripts/deploy/ &&
            python3 createconfig.py -s $CI_ENVIRONMENT_NAME &&
            /bin/bash install_venv.sh -d -r ../../requirements.txt &&
            python3 prepare_init.d.py &&
            python3 deploy.py -s $CI_ENVIRONMENT_NAME"
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    tty: true
    stdin_open: true
    networks:
      nw_swarm:

networks:
  nw_swarm:
    external: true

N'ebe a, ị ga-ahụ na ejikọrọ ihe ndị ahụ site na otu netwọk (nw_swarm) ma nwee ike ịnweta ibe ha.

Ngwa sistemu (dabere na redis, mysql) kewapụrụ na ọdọ mmiri izugbe nke ngwa ọdịnala (na atụmatụ, a na-ekewakwa ngwa ọdịnala dị ka ọrụ). Usoro mbufe nke ụyọkọ anyị dị ka ibufe CMD n'otu nnukwu ihe oyiyi ahaziri ahazi na, n'ozuzu, ọ dịghị ihe dị iche na ntinye aka nke akọwara na Nkebi nke Mbụ. M ga-emesi ndịiche ahụ ike:

git clone... - anyị nwetara faịlụ ndị dị mkpa iji rụọ ọrụ mbugharị (createconfig.py, install_venv.sh, wdg)
curl...&& tọpu ya... - budata ma budata ihe arụrụ arụ (ihe ejikọtara)

Enwere naanị otu nsogbu a na-akọwabeghị: akụrụngwa nwere interface webụ anaghị enweta ya site na ihe nchọgharị ndị mmepe. Anyị na-eji reverse proxy dozie nsogbu a:

Na .gitlab-ci.yml, mgbe ibugharị ụyọkọ ụyọkọ ahụ, gbakwunye ahịrị maka ibugharị nhazi (nke, mgbe emechara, naanị na-emelite nhazi ya (na-emepụta faịlụ nhazi nginx ọhụrụ dị ka template: /etc/nginx/conf.d) /${CI_COMMIT_REF_NAME}.conf) - hụ koodu docker-compose-nginx.yml)

    - docker stack deploy -c docker-compose-nginx.yml ${CI_ENVIRONMENT_NAME} --with-registry-auth

docker-na-ede-nginx.yml

---
version: '3'

services:
  nginx:
    image: nginx:latest
    environment:
      CI_COMMIT_REF_NAME: ${CI_COMMIT_REF_NAME}
      NGINX_CONFIG: |-
            server {
                listen 8080;
                server_name staging_${CI_COMMIT_REF_NAME}_cluster.dev;

                location / {
                    proxy_pass http://staging_${CI_COMMIT_REF_NAME}_cluster:8080;
                }
            }
            server {
                listen 5555;
                server_name staging_${CI_COMMIT_REF_NAME}_cluster.dev;

                location / {
                    proxy_pass http://staging_${CI_COMMIT_REF_NAME}_cluster:5555;
                }
            }
    volumes:
      - /tmp/staging/nginx:/etc/nginx/conf.d
    command:
      /bin/bash -c "echo -e "$$NGINX_CONFIG" > /etc/nginx/conf.d/${CI_COMMIT_REF_NAME}.conf;
        nginx -g "daemon off;";
        /etc/init.d/nginx reload"
    ports:
      - 8080:8080
      - 5555:5555
      - 3000:3000
      - 443:443
      - 80:80
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

networks:
  nw_swarm:
    external: true

Na kọmpụta ndị nrụpụta, melite /etc/hosts; tọọ url ka ọ bụrụ nginx:

10.50.173.106 staging_BRANCH-1831_cluster.dev
Yabụ, ewebatala ụyọkọ staging dịpụrụ adịpụ na ndị mmepe nwere ike ịmalite ha n'ụdị ọ bụla zuru oke iji nwalee ọrụ ha.

Atụmatụ ọdịnihu:

Kewapụ ihe anyị dị ka ọrụ
Mepụta Dockerfile maka nke ọ bụla
Chọpụta ọnụ ọnụ na-ebuchaghị ibu na akpaghị aka
Ezipụta ọnụ ọnụ site na iji ndebiri aha (kama iji id dị ka ọ dị na akụkọ ahụ)
Tinye nlele na ebibiwo ngwugwu ahụ
...

Daalụ pụrụ iche maka ederede.

isi: www.habr.com

Tinye Docker Swarm ngwa

Nkebi nke XNUMX

Nkebi nke II

Tinye a comment Отменить ответ