🥇 Ran awọn ohun elo ṣiṣẹ ni lilo Docker Swarm

Eto iṣeduro akoonu fidio ori ayelujara ti a n ṣiṣẹ lori jẹ idagbasoke iṣowo pipade ati imọ-ẹrọ jẹ iṣupọ paati pupọ ti ohun-ini ati awọn paati orisun ṣiṣi. Idi ti kikọ nkan yii ni lati ṣapejuwe imuse ti eto ikojọpọ swarm docker fun pẹpẹ ti o ti ṣeto, laisi idalọwọduro iṣan-iṣẹ ti iṣeto ti awọn ilana wa labẹ awọn ipo akoko to lopin. Itan-akọọlẹ ti a gbekalẹ si akiyesi rẹ ti pin si awọn ẹya meji. Apa akọkọ ṣe apejuwe CI/CD ṣaaju lilo docker swarm, ati apakan keji ṣe apejuwe ilana ti imuse rẹ. Awọn ti ko nifẹ lati ka apakan akọkọ le lọ lailewu si ekeji.

Ьmi

Ni ẹẹkan, iwulo wa lati ṣeto ilana CI/CD ni yarayara bi o ti ṣee. Ọkan ninu awọn ipo kii ṣe lati lo Docker fun imuṣiṣẹ Awọn eroja ti wa ni idagbasoke fun awọn idi pupọ:

fun igbẹkẹle diẹ sii ati iduroṣinṣin ti awọn paati ni iṣelọpọ (ie, ni pataki, ibeere naa lati maṣe lo agbara agbara)
Awọn olupilẹṣẹ asiwaju ko fẹ lati ṣiṣẹ pẹlu Docker (ajeji, ṣugbọn iyẹn ni bi o ti ri)
fun arojinle idi ti R&D isakoso

Awọn amayederun, akopọ ati awọn ibeere ibẹrẹ isunmọ fun MVP jẹ bi atẹle:

Awọn olupin Intel® X4 5650 pẹlu Debian (ẹrọ ti o lagbara diẹ sii fun idagbasoke)
Idagbasoke ti ara rẹ aṣa irinše ti wa ni ti gbe jade ni C ++, Python3
Awọn irinṣẹ ẹgbẹ kẹta akọkọ ti a lo: Kafka, Clickhouse, Airflow, Redis, Grafana, Postgresql, Mysql,…
Awọn paipu fun kikọ ati idanwo awọn paati lọtọ fun yokokoro ati itusilẹ

Ọkan ninu awọn ibeere akọkọ ti o nilo lati yanju ni ipele ibẹrẹ ni bii awọn paati aṣa yoo ṣe ran lọ si agbegbe eyikeyi (CI/CD).

A pinnu lati fi sori ẹrọ awọn paati ẹnikẹta ni ọna ṣiṣe ati mu wọn dojuiwọn ni ọna ṣiṣe. Awọn ohun elo aṣa ti o dagbasoke ni C ++ tabi Python ni a le gbe lọ ni awọn ọna pupọ. Lara wọn, fun apẹẹrẹ: ṣiṣẹda awọn idii eto, fifiranṣẹ wọn si ibi ipamọ ti awọn aworan ti a gba ati fifi sori wọn atẹle lori awọn olupin. Fun idi kan ti a ko mọ tẹlẹ, ọna miiran ti yan, eyun: lilo CI, awọn faili ṣiṣe ohun elo jẹ akopọ, ṣẹda agbegbe iṣẹ akanṣe foju kan, awọn modulu py lati awọn ibeere.txt ti fi sori ẹrọ, ati pe gbogbo awọn ohun elo wọnyi ni a firanṣẹ pẹlu awọn atunto, awọn iwe afọwọkọ ati agbegbe ohun elo ti o tẹle si awọn olupin. Nigbamii ti, awọn ohun elo ṣe ifilọlẹ lati ọdọ olumulo foju kan laisi awọn ẹtọ alabojuto.

Gitlab-CI ti yan bi eto CI/CD. Pipeline ti o yọrisi dabi nkan bi eyi:

Ni igbekalẹ, gitlab-ci.yml dabi eyi:

---
variables:
  # минимальная версия ЦПУ на серверах, где разворачивается кластер
  CMAKE_CPUTYPE: "westmere"

  DEBIAN: "MYREGISTRY:5000/debian:latest"

before_script:
  - eval $(ssh-agent -s)
  - ssh-add <(echo "$SSH_PRIVATE_KEY")
  - mkdir -p ~/.ssh && echo -e "Host *ntStrictHostKeyChecking nonn" > ~/.ssh/config

stages:
  - build
  - testing
  - deploy

debug.debian:
  stage: build
  image: $DEBIAN
  script:
    - cd builds/release && ./build.sh
    paths:
      - bin/
      - builds/release/bin/
    when: always
release.debian:
  stage: build
  image: $DEBIAN
  script:
    - cd builds/release && ./build.sh
    paths:
      - bin/
      - builds/release/bin/
    when: always

## testing stage
tests.codestyle:
  stage: testing
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - /bin/bash run_tests.sh -t codestyle -b "${CI_COMMIT_REF_NAME}_codestyle"
tests.debug.debian:
  stage: testing
  image: $DEBIAN
  dependencies:
    - debug.debian
  script:
    - /bin/bash run_tests.sh -e codestyle/test_pylint.py -b "${CI_COMMIT_REF_NAME}_debian_debug"
  artifacts:
    paths:
      - run_tests/username/
    when: always
    expire_in: 1 week
tests.release.debian:
  stage: testing
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - /bin/bash run_tests.sh -e codestyle/test_pylint.py -b "${CI_COMMIT_REF_NAME}_debian_release"
  artifacts:
    paths:
      - run_tests/username/
    when: always
    expire_in: 1 week

## staging stage
deploy_staging:
  stage: deploy
  environment: staging
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - cd scripts/deploy/ &&
        python3 createconfig.py -s $CI_ENVIRONMENT_NAME &&
        /bin/bash install_venv.sh -d -r ../../requirements.txt &&
        python3 prepare_init.d.py &&
        python3 deploy.py -s $CI_ENVIRONMENT_NAME
  when: manual

O tọ lati ṣe akiyesi pe apejọ ati idanwo ni a ṣe lori aworan tirẹ, nibiti gbogbo awọn idii eto pataki ti fi sii tẹlẹ ati awọn eto miiran ti ṣe.

Botilẹjẹpe ọkọọkan awọn iwe afọwọkọ wọnyi ni awọn iṣẹ jẹ iwunilori ni ọna tirẹ, dajudaju Emi kii yoo sọrọ nipa wọn; ṣapejuwe ọkọọkan wọn yoo gba akoko pupọ ati eyi kii ṣe idi ti nkan naa. Jẹ ki n fa akiyesi rẹ nikan si otitọ pe ipele imuṣiṣẹ ni lẹsẹsẹ ti awọn iwe afọwọkọ pipe:

ṣẹdaconfig.py - ṣẹda faili settings.ini pẹlu awọn eto fun awọn paati ni awọn agbegbe oriṣiriṣi fun imuṣiṣẹ atẹle (Isọjade, iṣelọpọ, Idanwo,…)
install_venv.sh - ṣẹda agbegbe foju kan fun awọn paati py ninu iwe ilana kan pato ati daakọ si awọn olupin latọna jijin
mura_init.d.py - ngbaradi awọn iwe afọwọkọ fun awọn paati iduro-ibẹrẹ ti o da lori awoṣe
ransogun.py - ransogun ati ki o tun titun irinše

Akoko ti kọja. Ipele ipele ti rọpo nipasẹ iṣelọpọ ati iṣelọpọ. Atilẹyin fun ọja naa ti ṣafikun lori pinpin diẹ sii (CentOS). Awọn olupin ti ara 5 ti o lagbara miiran ati awọn foju foju mejila ni a ṣafikun. Ati pe o nira pupọ si fun awọn idagbasoke ati awọn oludanwo lati ṣe idanwo awọn iṣẹ ṣiṣe wọn ni agbegbe diẹ sii tabi kere si isunmọ si ipo iṣẹ. Ni akoko yii o han gbangba pe ko ṣee ṣe lati ṣe laisi rẹ…

Apá II

Nitorinaa, iṣupọ wa jẹ eto iyalẹnu ti awọn paati mejila mejila ti ko ṣe apejuwe nipasẹ Dockerfiles. O le tunto rẹ fun imuṣiṣẹ si agbegbe kan pato nikan ni gbogbogbo. Iṣẹ-ṣiṣe wa ni lati ran iṣupọ naa lọ si agbegbe idasile lati ṣe idanwo ṣaaju idanwo-itusilẹ ṣaaju.

Ni imọ-jinlẹ, ọpọlọpọ awọn iṣupọ le wa ni igbakanna ti n ṣiṣẹ: bi ọpọlọpọ awọn iṣẹ ṣiṣe wa ni ipo ti o pari tabi ti o sunmọ ipari. Agbara ti awọn olupin ti o wa ni ipamọ wa gba wa laaye lati ṣiṣe awọn iṣupọ pupọ lori olupin kọọkan. Iṣupọ iṣeto kọọkan gbọdọ jẹ sọtọ (ko yẹ ki o wa ni agbekọja ni awọn ebute oko oju omi, awọn ilana, ati bẹbẹ lọ).

Ohun elo ti o niyelori julọ ni akoko wa, ati pe a ko ni pupọ ninu rẹ.

Fun ibẹrẹ iyara, a yan Docker Swarm nitori ayedero rẹ ati faaji rọ. Ohun akọkọ ti a ṣe ni ṣẹda oluṣakoso ati ọpọlọpọ awọn apa lori awọn olupin latọna jijin:

$ docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
kilqc94pi2upzvabttikrfr5d     nop-test-1     Ready               Active                                  19.03.2
jilwe56pl2zvabupryuosdj78     nop-test-2     Ready               Active                                  19.03.2
j5a4yz1kr2xke6b1ohoqlnbq5 *   nop-test-3     Ready               Active              Leader              19.03.2

Nigbamii, a ṣẹda nẹtiwọki kan:


$ docker network create --driver overlay --subnet 10.10.10.0/24 nw_swarm

Nigbamii ti, a sopọ Gitlab-CI ati awọn apa Swarm ni awọn ofin ti iṣakoso latọna jijin ti awọn apa lati CI: fifi sori awọn iwe-ẹri, ṣeto awọn oniyipada aṣiri, ati tun ṣeto iṣẹ Docker lori olupin iṣakoso. Eyi nkan ti o ti fipamọ wa kan pupo ti akoko.

Nigbamii ti, a ṣafikun awọn iṣẹ fun ṣiṣẹda ati iparun akopọ ni .gitlab-ci .yml.

Ọpọlọpọ awọn iṣẹ diẹ sii ni a ti ṣafikun si .gitlab-ci .yml

## staging stage
deploy_staging:
  stage: testing
  before_script:
    - echo "override global 'before_script'"
  image: "REGISTRY:5000/docker:latest"
  environment: staging
  dependencies: []
  variables:
    DOCKER_CERT_PATH: "/certs"
    DOCKER_HOST: tcp://10.50.173.107:2376
    DOCKER_TLS_VERIFY: 1
    CI_BIN_DEPENDENCIES_JOB: "release.centos.7"
  script:
    - mkdir -p $DOCKER_CERT_PATH
    - echo "$TLSCACERT" > $DOCKER_CERT_PATH/ca.pem
    - echo "$TLSCERT" > $DOCKER_CERT_PATH/cert.pem
    - echo "$TLSKEY" > $DOCKER_CERT_PATH/key.pem
    - docker stack deploy -c docker-compose.yml ${CI_ENVIRONMENT_NAME}_${CI_COMMIT_REF_NAME} --with-registry-auth
    - rm -rf $DOCKER_CERT_PATH
  when: manual

## stop staging stage
stop_staging:
  stage: testing
  before_script:
    - echo "override global 'before_script'"
  image: "REGISTRY:5000/docker:latest"
  environment: staging
  dependencies: []
  variables:
    DOCKER_CERT_PATH: "/certs"
    DOCKER_HOST: tcp://10.50.173.107:2376
    DOCKER_TLS_VERIFY: 1
  script:
    - mkdir -p $DOCKER_CERT_PATH
    - echo "$TLSCACERT" > $DOCKER_CERT_PATH/ca.pem
    - echo "$TLSCERT" > $DOCKER_CERT_PATH/cert.pem
    - echo "$TLSKEY" > $DOCKER_CERT_PATH/key.pem
    - docker stack rm ${CI_ENVIRONMENT_NAME}_${CI_COMMIT_REF_NAME}
    # TODO: need check that stopped
  when: manual

Lati ajẹkù koodu ti o wa loke o han gbangba pe awọn bọtini meji ti ṣafikun si Pipelines (deploy_staging, stop_staging) ti o nilo iṣe afọwọṣe.

Orukọ akopọ ibaamu orukọ ẹka ati iyasọtọ yii yẹ ki o to. Awọn iṣẹ inu akopọ gba awọn adiresi IP alailẹgbẹ, ati awọn ebute oko oju omi, awọn ilana, ati bẹbẹ lọ. yoo ya sọtọ, ṣugbọn kanna lati akopọ si akopọ (niwọn igba ti faili iṣeto jẹ kanna fun gbogbo awọn akopọ) - iyẹn ni a fẹ. A ran awọn akopọ (iṣupọ) lilo docker-compose.yml, eyi ti o ṣe apejuwe iṣupọ wa.

docker-compose.yml

---
version: '3'

services:
  userprop:
    image: redis:alpine
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:
  celery_bcd:
    image: redis:alpine
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  schedulerdb:
    image: mariadb:latest
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
      MYSQL_DATABASE: schedulerdb
      MYSQL_USER: ****
      MYSQL_PASSWORD: ****
    command: ['--character-set-server=utf8mb4', '--collation-server=utf8mb4_unicode_ci', '--explicit_defaults_for_timestamp=1']
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  celerydb:
    image: mariadb:latest
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
      MYSQL_DATABASE: celerydb
      MYSQL_USER: ****
      MYSQL_PASSWORD: ****
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  cluster:
    image: $CENTOS7
    environment:
      - CENTOS
      - CI_ENVIRONMENT_NAME
      - CI_API_V4_URL
      - CI_REPOSITORY_URL
      - CI_PROJECT_ID
      - CI_PROJECT_URL
      - CI_PROJECT_PATH
      - CI_PROJECT_NAME
      - CI_COMMIT_REF_NAME
      - CI_BIN_DEPENDENCIES_JOB
    command: >
      sudo -u myusername -H /bin/bash -c ". /etc/profile &&
        mkdir -p /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME &&
        cd /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME &&
            git clone -b $CI_COMMIT_REF_NAME $CI_REPOSITORY_URL . &&
            curl $CI_API_V4_URL/projects/$CI_PROJECT_ID/jobs/artifacts/$CI_COMMIT_REF_NAME/download?job=$CI_BIN_DEPENDENCIES_JOB -o artifacts.zip &&
            unzip artifacts.zip ;
        cd /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME/scripts/deploy/ &&
            python3 createconfig.py -s $CI_ENVIRONMENT_NAME &&
            /bin/bash install_venv.sh -d -r ../../requirements.txt &&
            python3 prepare_init.d.py &&
            python3 deploy.py -s $CI_ENVIRONMENT_NAME"
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    tty: true
    stdin_open: true
    networks:
      nw_swarm:

networks:
  nw_swarm:
    external: true

Nibi o le rii pe awọn paati naa ni asopọ nipasẹ nẹtiwọọki kan (nw_swarm) ati pe o wa si ara wọn.

Awọn paati eto (da lori redis, mysql) ti yapa lati adagun gbogbogbo ti awọn paati aṣa (ninu awọn ero, awọn paati aṣa tun pin bi awọn iṣẹ). Ipele imuṣiṣẹ ti iṣupọ wa dabi gbigbe CMD si aworan atunto nla kan wa ati, ni gbogbogbo, ko yatọ si imuṣiṣẹ ti a ṣalaye ni Apá I. Emi yoo tẹnumọ awọn iyatọ:

git clone... - a gba awọn faili pataki lati ṣe imuṣiṣẹ (createconfig.py, install_venv.sh, bbl)
curl...&& unzip... - ṣe igbasilẹ ati ṣii awọn ohun-ọṣọ kikọ (awọn ohun elo ti a ṣajọ)

Iṣoro kan ṣoṣo ti a ko ṣalaye sibẹsibẹ: awọn paati ti o ni wiwo wẹẹbu ko ni iraye si lati awọn aṣawakiri ti awọn olupilẹṣẹ. A yanju iṣoro yii nipa lilo aṣoju yiyipada, nitorinaa:

Ni .gitlab-ci.yml, lẹhin gbigbe akopọ iṣupọ, ṣafikun laini kan fun gbigbe iwọntunwọnsi (eyiti, nigbati o ba ṣe, ṣe imudojuiwọn iṣeto ni nikan (ṣẹda awọn faili atunto nginx tuntun ni ibamu si awoṣe: /etc/nginx/conf.d) /${CI_COMMIT_REF_NAME}.conf) - wo koodu docker-compose-nginx.yml)

    - docker stack deploy -c docker-compose-nginx.yml ${CI_ENVIRONMENT_NAME} --with-registry-auth

docker-kọ-nginx.yml

---
version: '3'

services:
  nginx:
    image: nginx:latest
    environment:
      CI_COMMIT_REF_NAME: ${CI_COMMIT_REF_NAME}
      NGINX_CONFIG: |-
            server {
                listen 8080;
                server_name staging_${CI_COMMIT_REF_NAME}_cluster.dev;

                location / {
                    proxy_pass http://staging_${CI_COMMIT_REF_NAME}_cluster:8080;
                }
            }
            server {
                listen 5555;
                server_name staging_${CI_COMMIT_REF_NAME}_cluster.dev;

                location / {
                    proxy_pass http://staging_${CI_COMMIT_REF_NAME}_cluster:5555;
                }
            }
    volumes:
      - /tmp/staging/nginx:/etc/nginx/conf.d
    command:
      /bin/bash -c "echo -e "$$NGINX_CONFIG" > /etc/nginx/conf.d/${CI_COMMIT_REF_NAME}.conf;
        nginx -g "daemon off;";
        /etc/init.d/nginx reload"
    ports:
      - 8080:8080
      - 5555:5555
      - 3000:3000
      - 443:443
      - 80:80
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

networks:
  nw_swarm:
    external: true

Lori awọn kọnputa idagbasoke, imudojuiwọn /etc/hosts; ṣeto url si nginx:

10.50.173.106 staging_BRANCH-1831_cluster.dev
Nitorinaa, imuṣiṣẹ ti awọn iṣupọ itage ti o ya sọtọ ti ni imuse ati pe awọn olupilẹṣẹ le ṣe ifilọlẹ wọn ni iwọn eyikeyi ti o to lati ṣe idanwo awọn iṣẹ ṣiṣe wọn.

Awọn ero iwaju:

Yatọ awọn paati wa bi awọn iṣẹ
Ṣẹda Dockerfile fun ọkọọkan
Ṣe awari awọn apa ti o kojọpọ kere si ninu akopọ
Pato awọn apa nipa lilo awoṣe orukọ (dipo lilo id bi ninu nkan naa)
Ṣafikun ayẹwo pe akopọ ti run
...

O ṣeun pataki fun nkan.

orisun: www.habr.com

Ran awọn ohun elo ṣiṣẹ nipa lilo Docker Swarm

Ьmi

Apá II

Fi ọrọìwòye kun Fagilee esi