ProHoster > Blog > Administrasyon an > Deplwaye aplikasyon yo lè l sèvi avèk Docker Swarm

Deplwaye aplikasyon yo lè l sèvi avèk Docker Swarm

Sistèm rekòmandasyon kontni videyo sou entènèt n ap travay sou li a se yon devlopman komèsyal fèmen e teknikman se yon gwoup ki gen plizyè konpozan de konpozan propriétaires ak sous louvri. Objektif ekri atik sa a se dekri aplikasyon sistèm gwoupman docker swarm pou yon platfòm staging, san yo pa deranje workflow etabli pwosesis nou yo nan kondisyon tan limite. Se naratif ki prezante a atansyon ou divize an de pati. Premye pati a dekri CI/CD anvan ou sèvi ak docker swarm, ak dezyèm pati a dekri pwosesis pou aplike li. Moun ki pa enterese nan li premye pati a ka san danje ale nan dezyèm lan.

Pati I

Yon fwa sou yon tan, te gen yon bezwen yo mete kanpe yon pwosesis CI/CD pi vit ke posib. Youn nan kondisyon yo te pa sèvi ak Docker pou deplwaman eleman yo ap devlope pou plizyè rezon:

  • pou operasyon plis serye ak ki estab nan konpozan nan Pwodiksyon (sa vle di, nan sans, egzijans pou pa itilize Virtualization)
  • devlopè dirijan yo pa t 'vle travay ak Docker (etranj, men se konsa li te ye)
  • pou rezon ideolojik nan jesyon R&D

Enfrastrikti, pile ak egzijans inisyal apwoksimatif pou MVP a te jan sa a:

  • 4 Intel® X5650 serveurs ak Debian (yon machin ki pi pwisan nèt pou devlopman)
  • Devlopman pwòp konpozan koutim ou yo fèt nan C++, Python3
  • Prensipal zouti twazyèm pati yo itilize: Kafka, Clickhouse, Airflow, Redis, Grafana, Postgresql, Mysql, ...
  • Tiyo pou bati ak tès konpozan separeman pou debogaj ak lage

Youn nan premye kesyon ki bezwen rezoud nan premye etap la se ki jan konpozan koutim yo pral deplwaye nan nenpòt anviwònman (CI/CD).

Nou te deside enstale konpozan twazyèm-pati sistemikman epi mete ajou yo sistemikman. Aplikasyon koutim devlope nan C++ oswa Python ka deplwaye nan plizyè fason. Pami yo, pou egzanp: kreye pakè sistèm, voye yo nan repozitwa a nan imaj kolekte ak enstalasyon ki vin apre yo sou serveurs. Pou yon rezon ki deja enkoni, yo te chwazi yon lòt metòd, sètadi: lè l sèvi avèk CI, dosye ègzèkutabl aplikasyon yo konpile, yo kreye yon anviwònman pwojè vityèl, modil py soti nan requirements.txt yo enstale, ak tout zafè sa yo yo voye ansanm ak konfigirasyon, scripts ak anviwònman aplikasyon ki akonpaye yo nan serveurs yo. Apre sa, aplikasyon yo lanse soti nan yon itilizatè vityèl san dwa administratè.

Gitlab-CI te chwazi kòm sistèm CI/CD. Tiyo a ki te lakòz te sanble yon bagay tankou sa a:

Deplwaye aplikasyon yo lè l sèvi avèk Docker Swarm
Estriktirèl, gitlab-ci.yml te sanble ak sa a:

---
variables:
  # минимальная версия ЦПУ на серверах, где разворачивается кластер
  CMAKE_CPUTYPE: "westmere"

  DEBIAN: "MYREGISTRY:5000/debian:latest"

before_script:
  - eval $(ssh-agent -s)
  - ssh-add <(echo "$SSH_PRIVATE_KEY")
  - mkdir -p ~/.ssh && echo -e "Host *ntStrictHostKeyChecking nonn" > ~/.ssh/config

stages:
  - build
  - testing
  - deploy

debug.debian:
  stage: build
  image: $DEBIAN
  script:
    - cd builds/release && ./build.sh
    paths:
      - bin/
      - builds/release/bin/
    when: always
release.debian:
  stage: build
  image: $DEBIAN
  script:
    - cd builds/release && ./build.sh
    paths:
      - bin/
      - builds/release/bin/
    when: always

## testing stage
tests.codestyle:
  stage: testing
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - /bin/bash run_tests.sh -t codestyle -b "${CI_COMMIT_REF_NAME}_codestyle"
tests.debug.debian:
  stage: testing
  image: $DEBIAN
  dependencies:
    - debug.debian
  script:
    - /bin/bash run_tests.sh -e codestyle/test_pylint.py -b "${CI_COMMIT_REF_NAME}_debian_debug"
  artifacts:
    paths:
      - run_tests/username/
    when: always
    expire_in: 1 week
tests.release.debian:
  stage: testing
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - /bin/bash run_tests.sh -e codestyle/test_pylint.py -b "${CI_COMMIT_REF_NAME}_debian_release"
  artifacts:
    paths:
      - run_tests/username/
    when: always
    expire_in: 1 week

## staging stage
deploy_staging:
  stage: deploy
  environment: staging
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - cd scripts/deploy/ &&
        python3 createconfig.py -s $CI_ENVIRONMENT_NAME &&
        /bin/bash install_venv.sh -d -r ../../requirements.txt &&
        python3 prepare_init.d.py &&
        python3 deploy.py -s $CI_ENVIRONMENT_NAME
  when: manual

Li se vo anyen ke asanble ak tès yo te pote sou pwòp imaj li yo, kote tout pakè sistèm ki nesesè yo deja enstale ak lòt anviwònman yo te fè.

Malgre ke chak nan scripts sa yo nan travay yo enteresan nan pwòp fason pa yo, mwen sètènman pa pral pale sou yo; dekri chak nan yo pral pran anpil tan epi sa a se pa objektif la nan atik la. Kite m 'jis atire atansyon ou sou lefèt ke etap deplwaman an konsiste de yon sekans nan scripts apèl:

  1. createconfig.py — kreye yon fichye settings.ini ak paramèt pou konpozan nan diferan anviwònman pou deplwaman ki vin apre (Prepwodiksyon, Pwodiksyon, Tès, ...)
  2. install_venv.sh — kreye yon anviwònman vityèl pou konpozan py nan yon anyè espesifik epi li kopye li sou sèvè aleka
  3. prepare_init.d.py — prepare scripts pou konpozan start-stop ki baze sou modèl la
  4. deploy.py — deplwaye ak rekòmanse nouvo eleman

Tan pase. Etap sèn nan te ranplase pa prepwodiksyon ak pwodiksyon. Sipò pou pwodwi a te ajoute sou yon lòt distribisyon (CentOS). Yon lòt 5 pwisan serveurs fizik ak yon douzèn moun vityèl yo te ajoute. Epi li te vin de pli zan pli difisil pou devlopè yo ak tèsteur yo teste travay yo nan yon anviwonman plis oswa mwens pre eta k ap travay. Nan moman sa a li te vin klè ke li te enposib fè san li ...

Pati II

Deplwaye aplikasyon yo lè l sèvi avèk Docker Swarm

Se konsa, gwoup nou an se yon sistèm espektakilè nan yon koup la douzèn eleman endividyèl pa dekri pa Dockerfiles. Ou ka configured li pou deplwaman nan yon anviwònman espesifik sèlman an jeneral. Travay nou an se deplwaye gwoup la nan yon anviwònman staging pou teste li anvan tès pre-lage.

Teyorikman, ka gen plizyè gwoup k ap travay ansanm: otan ke gen travay nan yon eta fini oswa tou pre fini. Pouvwa a nan serveurs yo nan jete nou pèmèt nou kouri plizyè grap sou chak sèvè. Chak gwoup staging dwe izole (pa ta dwe gen okenn sipèpoze nan pò, anyè, elatriye).

Resous ki gen plis valè nou an se tan nou, epi nou pa t gen anpil nan li.

Pou yon kòmansman pi vit, nou te chwazi Docker Swarm akòz senplisite li yo ak achitekti fleksib. Premye bagay nou te fè se te kreye yon manadjè ak plizyè nœuds sou sèvè aleka:

$ docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
kilqc94pi2upzvabttikrfr5d     nop-test-1     Ready               Active                                  19.03.2
jilwe56pl2zvabupryuosdj78     nop-test-2     Ready               Active                                  19.03.2
j5a4yz1kr2xke6b1ohoqlnbq5 *   nop-test-3     Ready               Active              Leader              19.03.2

Apre sa, nou te kreye yon rezo:


$ docker network create --driver overlay --subnet 10.10.10.0/24 nw_swarm

Apre sa, nou konekte Gitlab-CI ak nœuds Swarm an tèm de jesyon aleka nan nœuds soti nan CI: enstale sètifika, mete varyab sekrè, epi tou li mete kanpe sèvis Docker sou sèvè kontwòl la. Yon sèl sa a atik te sove nou anpil tan.

Apre sa, nou te ajoute travay pou kreye ak detwi pil la nan .gitlab-ci .yml.

Plizyè plis travay yo te ajoute nan .gitlab-ci .yml

## staging stage
deploy_staging:
  stage: testing
  before_script:
    - echo "override global 'before_script'"
  image: "REGISTRY:5000/docker:latest"
  environment: staging
  dependencies: []
  variables:
    DOCKER_CERT_PATH: "/certs"
    DOCKER_HOST: tcp://10.50.173.107:2376
    DOCKER_TLS_VERIFY: 1
    CI_BIN_DEPENDENCIES_JOB: "release.centos.7"
  script:
    - mkdir -p $DOCKER_CERT_PATH
    - echo "$TLSCACERT" > $DOCKER_CERT_PATH/ca.pem
    - echo "$TLSCERT" > $DOCKER_CERT_PATH/cert.pem
    - echo "$TLSKEY" > $DOCKER_CERT_PATH/key.pem
    - docker stack deploy -c docker-compose.yml ${CI_ENVIRONMENT_NAME}_${CI_COMMIT_REF_NAME} --with-registry-auth
    - rm -rf $DOCKER_CERT_PATH
  when: manual

## stop staging stage
stop_staging:
  stage: testing
  before_script:
    - echo "override global 'before_script'"
  image: "REGISTRY:5000/docker:latest"
  environment: staging
  dependencies: []
  variables:
    DOCKER_CERT_PATH: "/certs"
    DOCKER_HOST: tcp://10.50.173.107:2376
    DOCKER_TLS_VERIFY: 1
  script:
    - mkdir -p $DOCKER_CERT_PATH
    - echo "$TLSCACERT" > $DOCKER_CERT_PATH/ca.pem
    - echo "$TLSCERT" > $DOCKER_CERT_PATH/cert.pem
    - echo "$TLSKEY" > $DOCKER_CERT_PATH/key.pem
    - docker stack rm ${CI_ENVIRONMENT_NAME}_${CI_COMMIT_REF_NAME}
    # TODO: need check that stopped
  when: manual

Soti nan fragman kòd ki anwo a li klè ke de bouton yo te ajoute nan Pipelines (deploy_staging, stop_staging) ki mande pou aksyon manyèl.

Deplwaye aplikasyon yo lè l sèvi avèk Docker Swarm
Non pile a matche ak non branch lan ak singularité sa a ta dwe ase. Sèvis nan pil la resevwa adrès IP inik, ak pò, anyè, elatriye. yo pral izole, men menm bagay la tou de chemine an chemine (depi dosye a konfigirasyon se menm bagay la pou tout pil) - se sa nou te vle. Nou deplwaye pil la (grap) lè l sèvi avèk docker-compose.yml, ki dekri gwoup nou an.

docker-compose.yml

---
version: '3'

services:
  userprop:
    image: redis:alpine
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:
  celery_bcd:
    image: redis:alpine
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  schedulerdb:
    image: mariadb:latest
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
      MYSQL_DATABASE: schedulerdb
      MYSQL_USER: ****
      MYSQL_PASSWORD: ****
    command: ['--character-set-server=utf8mb4', '--collation-server=utf8mb4_unicode_ci', '--explicit_defaults_for_timestamp=1']
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  celerydb:
    image: mariadb:latest
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
      MYSQL_DATABASE: celerydb
      MYSQL_USER: ****
      MYSQL_PASSWORD: ****
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  cluster:
    image: $CENTOS7
    environment:
      - CENTOS
      - CI_ENVIRONMENT_NAME
      - CI_API_V4_URL
      - CI_REPOSITORY_URL
      - CI_PROJECT_ID
      - CI_PROJECT_URL
      - CI_PROJECT_PATH
      - CI_PROJECT_NAME
      - CI_COMMIT_REF_NAME
      - CI_BIN_DEPENDENCIES_JOB
    command: >
      sudo -u myusername -H /bin/bash -c ". /etc/profile &&
        mkdir -p /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME &&
        cd /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME &&
            git clone -b $CI_COMMIT_REF_NAME $CI_REPOSITORY_URL . &&
            curl $CI_API_V4_URL/projects/$CI_PROJECT_ID/jobs/artifacts/$CI_COMMIT_REF_NAME/download?job=$CI_BIN_DEPENDENCIES_JOB -o artifacts.zip &&
            unzip artifacts.zip ;
        cd /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME/scripts/deploy/ &&
            python3 createconfig.py -s $CI_ENVIRONMENT_NAME &&
            /bin/bash install_venv.sh -d -r ../../requirements.txt &&
            python3 prepare_init.d.py &&
            python3 deploy.py -s $CI_ENVIRONMENT_NAME"
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    tty: true
    stdin_open: true
    networks:
      nw_swarm:

networks:
  nw_swarm:
    external: true

Isit la ou ka wè ke eleman yo konekte pa yon sèl rezo (nw_swarm) epi yo aksesib youn ak lòt.

Konpozan sistèm (ki baze sou redis, mysql) yo separe ak pisin jeneral nan eleman koutim (nan plan, konpozan koutim yo divize tou kòm sèvis). Etap deplwaman gwoup nou an sanble transfere CMD nan yon sèl gwo imaj configuré nou an epi, an jeneral, pratikman pa diferan de deplwaman ki dekri nan Pati I a. Mwen pral mete aksan sou diferans ki genyen:

  • git klon... — nou jwenn dosye ki nesesè pou fè deplwaman an (createconfig.py, install_venv.sh, elatriye)
  • pli... && dézip... - telechaje ak dekonprime zafè bati yo (utilite konpile)

Gen yon sèl pwoblèm ki poko dekri: eleman ki gen yon koòdone entènèt pa aksesib nan navigatè devlopè yo. Nou rezoud pwoblèm sa a lè l sèvi avèk reverse proxy, konsa:

Nan .gitlab-ci.yml, apre yo fin deplwaye pile gwoup la, ajoute yon liy pou deplwaye balanse a (ki, lè yo pran angajman, sèlman mete ajou konfigirasyon li yo (kreye nouvo fichye konfigirasyon nginx dapre modèl la: /etc/nginx/conf.d). /${CI_COMMIT_REF_NAME}.conf) - gade kòd docker-compose-nginx.yml)

    - docker stack deploy -c docker-compose-nginx.yml ${CI_ENVIRONMENT_NAME} --with-registry-auth

docker-compose-nginx.yml

---
version: '3'

services:
  nginx:
    image: nginx:latest
    environment:
      CI_COMMIT_REF_NAME: ${CI_COMMIT_REF_NAME}
      NGINX_CONFIG: |-
            server {
                listen 8080;
                server_name staging_${CI_COMMIT_REF_NAME}_cluster.dev;

                location / {
                    proxy_pass http://staging_${CI_COMMIT_REF_NAME}_cluster:8080;
                }
            }
            server {
                listen 5555;
                server_name staging_${CI_COMMIT_REF_NAME}_cluster.dev;

                location / {
                    proxy_pass http://staging_${CI_COMMIT_REF_NAME}_cluster:5555;
                }
            }
    volumes:
      - /tmp/staging/nginx:/etc/nginx/conf.d
    command:
      /bin/bash -c "echo -e "$$NGINX_CONFIG" > /etc/nginx/conf.d/${CI_COMMIT_REF_NAME}.conf;
        nginx -g "daemon off;";
        /etc/init.d/nginx reload"
    ports:
      - 8080:8080
      - 5555:5555
      - 3000:3000
      - 443:443
      - 80:80
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

networks:
  nw_swarm:
    external: true

Sou òdinatè devlopè yo, mete ajou /etc/hosts; mete url la nan nginx:

10.50.173.106 staging_BRANCH-1831_cluster.dev

Se konsa, deplwaman gwoup izole yo te aplike epi devlopè yo kapab kounye a lanse yo nan nenpòt ki kantite ase pou teste travay yo.

Plan pou lavni:

  • Separe konpozan nou yo kòm sèvis
  • Kreye yon Dockerfile pou chak
  • Detekte otomatikman mwens nœuds chaje nan chemine a
  • Espesifye nœuds lè l sèvi avèk yon modèl non (olye ke w itilize id tankou nan atik la)
  • Ajoute yon chèk ke chemine a te detwi
  • ...

Mèsi espesyal pou yon atik.

Sous: www.habr.com

Add nouvo kòmantè