🥇 Sanya apps tare da Docker Swarm

Tsarin shawarwarin abun cikin bidiyo na kan layi da muke aiki da shi shine rufaffiyar ci gaban kasuwanci kuma ta fasaha ce ta ƙunshi abubuwa da yawa na abubuwan mallakar mallaka da buɗaɗɗen tushe. Makasudin rubuta wannan labarin shine don bayyana aiwatar da tsarin tari na docker swarm don wurin tsarawa ba tare da ɓata ingantaccen tsarin tafiyar da ayyukanmu cikin ƙayyadadden lokaci ba. Labarin da aka gabatar muku ya kasu kashi biyu. Kashi na farko yana bayanin CI / CD kafin amfani da docker swarm, na biyu kuma yana bayanin tsarin aiwatar da shi. Wadanda ba su da sha'awar karanta kashi na farko za su iya ci gaba zuwa na biyu lafiya.

Kashi na XNUMX

Komawa cikin nisa, shekara mai nisa, ya zama dole don saita tsarin CI / CD da sauri. Ɗaya daga cikin sharuɗɗan shine rashin amfani da Docker domin turawa abubuwan da aka haɓaka saboda dalilai da yawa:

don ƙarin ingantaccen aiki da kwanciyar hankali na abubuwan haɗin gwiwa a cikin Production (wato, a zahiri, buƙatun kada a yi amfani da haɓakawa)
manyan masu haɓakawa ba sa son yin aiki tare da Docker (m, amma haka yake)
bisa ga la'akarin akida na gudanarwa na R&D

An gabatar da ababen more rayuwa, tari da kimanin buƙatun farko na MVP kamar haka:

4 Intel® X5650 sabobin tare da Debian (an inganta injin mafi ƙarfi)
Ana aiwatar da haɓaka abubuwan haɗin kai na al'ada a cikin C ++, Python3
Babban kayan aikin ɓangare na uku da aka yi amfani da su: Kafka, Clickhouse, Airflow, Redis, Grafana, Postgresql, Mysql,…
Bututu don ginawa da gwada abubuwan da aka gyara daban don gyarawa da saki

Ɗaya daga cikin tambayoyin farko da ake buƙatar magancewa a matakin farko shine yadda za a yi amfani da kayan aikin al'ada a kowane yanayi (CI / CD).

Mun yanke shawarar shigar da sassa na ɓangare na uku a tsari kuma mu sabunta su cikin tsari. Ana iya tura aikace-aikacen al'ada da aka haɓaka a C++ ko Python ta hanyoyi da yawa. Daga cikinsu, misali: ƙirƙirar fakitin tsarin, aika su zuwa ma'ajiyar hotuna da aka gina sannan a sanya su a kan sabobin. Don wani dalili da ba a sani ba, an zaɓi wata hanyar, wato: ta amfani da CI, ana tattara fayilolin aiwatar da aikace-aikacen, an ƙirƙiri yanayin aikin kama-da-wane, ana shigar da py modules daga bukatun.txt, kuma ana aika duk waɗannan kayan tarihi tare da daidaitawa, rubutun da rakiyar yanayin aikace-aikacen zuwa sabobin. Bayan haka, ana ƙaddamar da aikace-aikacen azaman mai amfani mai kama-da-wane ba tare da haƙƙin mai gudanarwa ba.

An zaɓi Gitlab-CI azaman tsarin CI/CD. Sakamakon bututun ya yi kama da haka:

A tsari, gitlab-ci.yml yayi kama da wannan

---
variables:
  # минимальная версия ЦПУ на серверах, где разворачивается кластер
  CMAKE_CPUTYPE: "westmere"

  DEBIAN: "MYREGISTRY:5000/debian:latest"

before_script:
  - eval $(ssh-agent -s)
  - ssh-add <(echo "$SSH_PRIVATE_KEY")
  - mkdir -p ~/.ssh && echo -e "Host *ntStrictHostKeyChecking nonn" > ~/.ssh/config

stages:
  - build
  - testing
  - deploy

debug.debian:
  stage: build
  image: $DEBIAN
  script:
    - cd builds/release && ./build.sh
    paths:
      - bin/
      - builds/release/bin/
    when: always
release.debian:
  stage: build
  image: $DEBIAN
  script:
    - cd builds/release && ./build.sh
    paths:
      - bin/
      - builds/release/bin/
    when: always

## testing stage
tests.codestyle:
  stage: testing
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - /bin/bash run_tests.sh -t codestyle -b "${CI_COMMIT_REF_NAME}_codestyle"
tests.debug.debian:
  stage: testing
  image: $DEBIAN
  dependencies:
    - debug.debian
  script:
    - /bin/bash run_tests.sh -e codestyle/test_pylint.py -b "${CI_COMMIT_REF_NAME}_debian_debug"
  artifacts:
    paths:
      - run_tests/username/
    when: always
    expire_in: 1 week
tests.release.debian:
  stage: testing
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - /bin/bash run_tests.sh -e codestyle/test_pylint.py -b "${CI_COMMIT_REF_NAME}_debian_release"
  artifacts:
    paths:
      - run_tests/username/
    when: always
    expire_in: 1 week

## staging stage
deploy_staging:
  stage: deploy
  environment: staging
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - cd scripts/deploy/ &&
        python3 createconfig.py -s $CI_ENVIRONMENT_NAME &&
        /bin/bash install_venv.sh -d -r ../../requirements.txt &&
        python3 prepare_init.d.py &&
        python3 deploy.py -s $CI_ENVIRONMENT_NAME
  when: manual

Yana da mahimmanci a lura cewa ana gudanar da taro da gwaji akan hoton kansa, inda aka riga an shigar da duk fakitin tsarin da ake buƙata kuma an yi wasu saitunan.

Ko da yake kowane ɗayan waɗannan rubutun a cikin ayyukan yana da ban sha'awa ta hanyarsa, amma ba shakka ba zan yi magana game da su ba. Bayanin kowannensu zai ɗauki lokaci mai yawa kuma wannan ba shine manufar labarin ba. Zan ja hankalin ku kawai ga gaskiyar cewa matakin turawa ya ƙunshi jerin rubutun kira:

ƙirƙirarconfig.py - yana ƙirƙira fayil ɗin settings.ini tare da saitunan kayan aiki a cikin mahalli daban-daban don ƙaddamarwa na gaba (haifar, samarwa, Gwaji, ...)
shigar_venv.sh - yana ƙirƙira yanayin kama-da-wane don abubuwan py a cikin takamaiman kundin adireshi kuma yana kwafe shi zuwa sabobin nesa
shirya_init.d.py - yana shirya rubutun farawa don ɓangaren dangane da samfuri
tura.py - bazuwa kuma yana sake kunna sabbin abubuwa

Lokaci ya wuce. An maye gurbin matakin ƙaddamarwa ta hanyar samarwa da samarwa. Ƙara tallafi don samfurin akan ƙarin rarrabawa (CentOS). An ƙara ƙarin sabbin sabar jiki guda 5 masu ƙarfi da dozin na kama-da-wane. Kuma ya zama da wahala ga masu haɓakawa da masu gwada gwaje-gwaje don gwada ayyukansu a cikin yanayi ko žasa kusa da jihar aiki. A wannan lokacin, ya bayyana a fili cewa ba shi yiwuwa a yi ba tare da shi ba ...

Kashi na II

Don haka, tarin mu wani tsari ne mai ban sha'awa na wasu abubuwa guda goma sha biyu waɗanda Dockerfiles ba su bayyana ba. Kuna iya saita shi kawai don turawa zuwa takamaiman yanayi gabaɗaya. Aikinmu shine mu tura gungu zuwa wurin tsarawa don gwada shi kafin gwajin farko.

A ka'ida, za a iya samun gungu da yawa suna gudana lokaci guda: gwargwadon yawan ayyuka a cikin jihar da aka kammala ko kusa da kammalawa. Ƙarfin sabar da ke hannunmu yana ba mu damar gudanar da gungu da yawa akan kowace sabar. Dole ne a ware kowane gungu na tsari (kada a sami zoba a tashoshin jiragen ruwa, kundayen adireshi, da sauransu).

Mafi kyawun albarkatunmu shine lokacinmu, kuma ba mu da yawa.

Don farawa da sauri, mun zaɓi Docker Swarm saboda sauƙin sa da sassauƙar gine-gine. Abu na farko da muka yi shine ƙirƙirar manaja da nodes da yawa akan sabobin nesa:

$ docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
kilqc94pi2upzvabttikrfr5d     nop-test-1     Ready               Active                                  19.03.2
jilwe56pl2zvabupryuosdj78     nop-test-2     Ready               Active                                  19.03.2
j5a4yz1kr2xke6b1ohoqlnbq5 *   nop-test-3     Ready               Active              Leader              19.03.2

Na gaba, ƙirƙirar hanyar sadarwa:


$ docker network create --driver overlay --subnet 10.10.10.0/24 nw_swarm

Na gaba, mun haɗa Gitlab-CI da Swarm nodes dangane da ramut na nodes daga CI: shigar da takaddun shaida, saita masu canji na sirri, da kafa sabis na Docker akan uwar garken sarrafawa. Wannan labarin ya cece mu lokaci mai yawa.

Bayan haka, mun ƙara ƙirƙira tari da ayyukan lalata zuwa .gitlab-ci .yml.

An ƙara wasu ƙarin ayyuka zuwa .gitlab-ci .yml

## staging stage
deploy_staging:
  stage: testing
  before_script:
    - echo "override global 'before_script'"
  image: "REGISTRY:5000/docker:latest"
  environment: staging
  dependencies: []
  variables:
    DOCKER_CERT_PATH: "/certs"
    DOCKER_HOST: tcp://10.50.173.107:2376
    DOCKER_TLS_VERIFY: 1
    CI_BIN_DEPENDENCIES_JOB: "release.centos.7"
  script:
    - mkdir -p $DOCKER_CERT_PATH
    - echo "$TLSCACERT" > $DOCKER_CERT_PATH/ca.pem
    - echo "$TLSCERT" > $DOCKER_CERT_PATH/cert.pem
    - echo "$TLSKEY" > $DOCKER_CERT_PATH/key.pem
    - docker stack deploy -c docker-compose.yml ${CI_ENVIRONMENT_NAME}_${CI_COMMIT_REF_NAME} --with-registry-auth
    - rm -rf $DOCKER_CERT_PATH
  when: manual

## stop staging stage
stop_staging:
  stage: testing
  before_script:
    - echo "override global 'before_script'"
  image: "REGISTRY:5000/docker:latest"
  environment: staging
  dependencies: []
  variables:
    DOCKER_CERT_PATH: "/certs"
    DOCKER_HOST: tcp://10.50.173.107:2376
    DOCKER_TLS_VERIFY: 1
  script:
    - mkdir -p $DOCKER_CERT_PATH
    - echo "$TLSCACERT" > $DOCKER_CERT_PATH/ca.pem
    - echo "$TLSCERT" > $DOCKER_CERT_PATH/cert.pem
    - echo "$TLSKEY" > $DOCKER_CERT_PATH/key.pem
    - docker stack rm ${CI_ENVIRONMENT_NAME}_${CI_COMMIT_REF_NAME}
    # TODO: need check that stopped
  when: manual

Daga snippet na lambar da ke sama, zaku iya ganin cewa an ƙara maɓallai biyu (deploy_staging, stop_staging) zuwa Pipelines, suna buƙatar aikin hannu.

Sunan tari ya dace da sunan reshe kuma wannan keɓantacce yakamata ya isa. Ayyuka a cikin tarin suna karɓar adiresoshin IP na musamman, da tashar jiragen ruwa, kundayen adireshi, da sauransu. za a ware, amma iri ɗaya daga tari zuwa tari (saboda fayil ɗin daidaitawa iri ɗaya ne ga duk tari) - abin da muke so. Muna tura tari (cluster) ta amfani da docker-compose.yml, wanda ke bayyana tarin mu.

docker-compose.yml

---
version: '3'

services:
  userprop:
    image: redis:alpine
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:
  celery_bcd:
    image: redis:alpine
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  schedulerdb:
    image: mariadb:latest
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
      MYSQL_DATABASE: schedulerdb
      MYSQL_USER: ****
      MYSQL_PASSWORD: ****
    command: ['--character-set-server=utf8mb4', '--collation-server=utf8mb4_unicode_ci', '--explicit_defaults_for_timestamp=1']
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  celerydb:
    image: mariadb:latest
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
      MYSQL_DATABASE: celerydb
      MYSQL_USER: ****
      MYSQL_PASSWORD: ****
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  cluster:
    image: $CENTOS7
    environment:
      - CENTOS
      - CI_ENVIRONMENT_NAME
      - CI_API_V4_URL
      - CI_REPOSITORY_URL
      - CI_PROJECT_ID
      - CI_PROJECT_URL
      - CI_PROJECT_PATH
      - CI_PROJECT_NAME
      - CI_COMMIT_REF_NAME
      - CI_BIN_DEPENDENCIES_JOB
    command: >
      sudo -u myusername -H /bin/bash -c ". /etc/profile &&
        mkdir -p /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME &&
        cd /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME &&
            git clone -b $CI_COMMIT_REF_NAME $CI_REPOSITORY_URL . &&
            curl $CI_API_V4_URL/projects/$CI_PROJECT_ID/jobs/artifacts/$CI_COMMIT_REF_NAME/download?job=$CI_BIN_DEPENDENCIES_JOB -o artifacts.zip &&
            unzip artifacts.zip ;
        cd /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME/scripts/deploy/ &&
            python3 createconfig.py -s $CI_ENVIRONMENT_NAME &&
            /bin/bash install_venv.sh -d -r ../../requirements.txt &&
            python3 prepare_init.d.py &&
            python3 deploy.py -s $CI_ENVIRONMENT_NAME"
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    tty: true
    stdin_open: true
    networks:
      nw_swarm:

networks:
  nw_swarm:
    external: true

Anan za ku ga cewa an haɗa abubuwan haɗin ta hanyar sadarwa ɗaya (nw_swarm) kuma suna samuwa ga juna.

Abubuwan da aka gyara na tsarin (dangane da redis, mysql) an rabu da su daga babban tafkin abubuwan abubuwan al'ada (a cikin tsare-tsare da na al'ada an raba su azaman sabis). Matakin tura tarin mu yayi kama da wucewar CMD zuwa cikin babban hoton da aka tsara kuma, gabaɗaya, a zahiri bai bambanta da aikin da aka bayyana a Sashe na I ba. Zan haskaka bambance-bambance:

git clone... - sami fayilolin da ake buƙata don turawa (createconfig.py, install_venv.sh, da sauransu)
lankwasa... && cire zip... - zazzagewa da buɗe kayan gini (abubuwan da aka haɗa)

Matsala ɗaya ce kawai da har yanzu ba a bayyana ba: abubuwan da ke da haɗin yanar gizo ba sa samun dama daga masu bincike na masu haɓakawa. Muna magance wannan matsalar ta amfani da reverse proxy, ta haka:

A cikin .gitlab-ci.yml, bayan ƙaddamar da tarin gungu, muna ƙara layin ƙaddamar da ma'auni (wanda, lokacin da ya aikata, kawai yana sabunta tsarin sa (yana ƙirƙira sababbin fayilolin nginx bisa ga samfurin: /etc/nginx/conf. d/${CI_COMMIT_REF_NAME}.conf) - duba lambar docker-compose-nginx.yml)

    - docker stack deploy -c docker-compose-nginx.yml ${CI_ENVIRONMENT_NAME} --with-registry-auth

docker-compose-nginx.yml

---
version: '3'

services:
  nginx:
    image: nginx:latest
    environment:
      CI_COMMIT_REF_NAME: ${CI_COMMIT_REF_NAME}
      NGINX_CONFIG: |-
            server {
                listen 8080;
                server_name staging_${CI_COMMIT_REF_NAME}_cluster.dev;

                location / {
                    proxy_pass http://staging_${CI_COMMIT_REF_NAME}_cluster:8080;
                }
            }
            server {
                listen 5555;
                server_name staging_${CI_COMMIT_REF_NAME}_cluster.dev;

                location / {
                    proxy_pass http://staging_${CI_COMMIT_REF_NAME}_cluster:5555;
                }
            }
    volumes:
      - /tmp/staging/nginx:/etc/nginx/conf.d
    command:
      /bin/bash -c "echo -e "$$NGINX_CONFIG" > /etc/nginx/conf.d/${CI_COMMIT_REF_NAME}.conf;
        nginx -g "daemon off;";
        /etc/init.d/nginx reload"
    ports:
      - 8080:8080
      - 5555:5555
      - 3000:3000
      - 443:443
      - 80:80
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

networks:
  nw_swarm:
    external: true

A kan kwamfutocin ci gaba, sabunta /etc/hosts; rubuta url zuwa nginx:

10.50.173.106 staging_BRANCH-1831_cluster.dev
Don haka, an aiwatar da tura gungu na shirye-shiryen keɓe kuma masu haɓakawa yanzu za su iya sarrafa su ta kowace lamba isa don duba ayyukansu.

Shirye-shiryen gaba:

Rarraba abubuwan da muka gyara azaman ayyuka
Yi don kowane Dockerfile
Gane nodes ɗin da ba su da yawa ta atomatik a cikin tari
Ƙayyade nodes ta tsarin suna (maimakon amfani da id kamar a cikin labarin)
Ƙara cak cewa tari ya lalace
...

Godiya ta musamman labarin.

source: www.habr.com

Sanya Aikace-aikace tare da Docker Swarm

Kashi na XNUMX

Kashi na II

Add a comment Отменить ответ