ProHoster > Блог > Maamulka > Dhig Codsiyada Docker Swarm

Dhig Codsiyada Docker Swarm

Habka tallo bixinta fiidyowga online-ka ah ee aanu ku shaqaynayno waa horumar ganacsi oo xidhan oo farsamo ahaan waa koox ka kooban qaybo badan oo leh qaybo il furan. Ujeedada qorista maqaalkan waa in lagu qeexo hirgelinta nidaamka kooxaynta raxan ee docker-ka ee goobta diyaarinta iyada oo aan carqaladayn habsocodka shaqo ee hab-socodkayaga waqti xaddidan. Sheekada lagu soo bandhigay dareenkaaga waxay u qaybsantaa laba qaybood. Qaybta hore waxay qeexaysaa CI / CD ka hor inta aan la isticmaalin docker swarm, iyo tan labaad waxay qeexaysaa habka loo hirgelin karo. Kuwa aan xiisaynaynin inay akhriyaan qaybta hore waxay si badbaado leh ugu gudbi karaan qaybta labaad.

I

Dib ugu noqoshada fog, sanadka fog, waxay ahayd lagama maarmaan in la dejiyo habka CI / CD sida ugu dhakhsaha badan ee suurtogalka ah. Mid ka mid ah shuruudaha ayaa ahaa inaan la isticmaalin Docker si loo geeyo qaybo loo sameeyay dhowr sababood dartood:

  • si loogu kalsoonaan karo oo xasiloon oo ka kooban qaybaha wax soo saarka (taasi waa, dhab ahaantii, shuruuda in aan la isticmaalin khayaali)
  • Hormuudka horumariyayaashu ma rabin inay la shaqeeyaan Docker (yaab, laakiin taasi waa sida ay ahayd)
  • iyadoo la raacayo tixgalinta fikirka ee maamulka R&D

Kaabayaasha, raxan iyo qiyaasaha shuruudaha bilowga ah ee MVP ayaa loo soo bandhigay sida soo socota:

  • 4 Intel® X5650 oo leh Debian (hal mishiin ka xoog badan ayaa si buuxda loo horumariyay)
  • Horumarinta qaybaha gaarka ah waxaa lagu fuliyaa C ++, Python3
  • Aaladaha ugu muhiimsan ee xisbiga 3aad ee la isticmaalo: Kafka, Clickhouse, Airflow, Redis, Grafana, Postgresql, Mysql,…
  • Dhuumaha dhismaha iyo tijaabinta qaybaha si gaar ah loogu talagalay cilladaha iyo sii deynta

Mid ka mid ah su'aalaha ugu horreeya ee u baahan in wax laga qabto marxaladda hore waa sida qaybaha gaarka ah loo geyn doono deegaan kasta (CI / CD).

Waxaan go'aansanay inaan si nidaamsan u rakibno qaybaha dhinac saddexaad oo aan u cusbooneysiinno si nidaamsan. Codsiyada gaarka ah ee lagu sameeyay C++ ama Python waxaa loo diri karaa siyaabo dhowr ah. Waxaa ka mid ah, tusaale ahaan: abuurista xirmooyinka nidaamka, u dirida bakhaarka sawirada la dhisay ka dibna ku rakibaya server-yada. Sababta aan la garanayn, hab kale ayaa loo doortay, kaas oo ah: iyadoo la adeegsanayo CI, codsiyada la fulin karo ayaa la soo ururiyey, jawi mashruuc macmal ah ayaa la abuuray, modules py ayaa lagu rakibay shuruudaha.txt, dhammaan artifacts waxaa loo soo diray iyada oo ay la socdaan qaab-dhismeedka, qoraallada iyo deegaanka codsiga ee server-yada la socda. Marka xigta, codsiyada waxaa loo bilaabaa sidii isticmaale muuqaal ah oo aan lahayn xuquuq maamul.

Gitlab-CI waxaa loo doortay nidaamka CI/CD. Dhuunta ka dhalatay waxay u ekayd sidan:

Dhig Codsiyada Docker Swarm
Qaab dhismeed ahaan, gitlab-ci.yml sidan ayuu u ekaa

---
variables:
  # минимальная версия ЦПУ на серверах, где разворачивается кластер
  CMAKE_CPUTYPE: "westmere"

  DEBIAN: "MYREGISTRY:5000/debian:latest"

before_script:
  - eval $(ssh-agent -s)
  - ssh-add <(echo "$SSH_PRIVATE_KEY")
  - mkdir -p ~/.ssh && echo -e "Host *ntStrictHostKeyChecking nonn" > ~/.ssh/config

stages:
  - build
  - testing
  - deploy

debug.debian:
  stage: build
  image: $DEBIAN
  script:
    - cd builds/release && ./build.sh
    paths:
      - bin/
      - builds/release/bin/
    when: always
release.debian:
  stage: build
  image: $DEBIAN
  script:
    - cd builds/release && ./build.sh
    paths:
      - bin/
      - builds/release/bin/
    when: always

## testing stage
tests.codestyle:
  stage: testing
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - /bin/bash run_tests.sh -t codestyle -b "${CI_COMMIT_REF_NAME}_codestyle"
tests.debug.debian:
  stage: testing
  image: $DEBIAN
  dependencies:
    - debug.debian
  script:
    - /bin/bash run_tests.sh -e codestyle/test_pylint.py -b "${CI_COMMIT_REF_NAME}_debian_debug"
  artifacts:
    paths:
      - run_tests/username/
    when: always
    expire_in: 1 week
tests.release.debian:
  stage: testing
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - /bin/bash run_tests.sh -e codestyle/test_pylint.py -b "${CI_COMMIT_REF_NAME}_debian_release"
  artifacts:
    paths:
      - run_tests/username/
    when: always
    expire_in: 1 week

## staging stage
deploy_staging:
  stage: deploy
  environment: staging
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - cd scripts/deploy/ &&
        python3 createconfig.py -s $CI_ENVIRONMENT_NAME &&
        /bin/bash install_venv.sh -d -r ../../requirements.txt &&
        python3 prepare_init.d.py &&
        python3 deploy.py -s $CI_ENVIRONMENT_NAME
  when: manual

Waxaa xusid mudan in shirka iyo tijaabinta lagu sameeyay sawirkeeda, halkaas oo dhammaan xirmooyinka nidaamka lagama maarmaanka ah horay loo rakibay iyo goobo kale oo la sameeyay.

Inkasta oo mid kasta oo ka mid ah qoraalladan shaqooyinka ay si gaar ah u xiiseeyaan, laakiin dabcan kama hadli doono iyaga. Sharaxaadda mid kasta oo iyaga ka mid ah waxay qaadan doontaa wakhti badan oo tani maaha ujeedada maqaalka. Waxaan kaliya ku soo jeedin doonaa dareenkaaga xaqiiqda ah in marxaladda geyntu ay ka kooban tahay taxane qoraallo wicitaan ah:

  1. abuuroconfig.py - waxay abuurtaa faylka settings.ini oo leh jaangooyooyin ka kooban goobo kala duwan oo loogu talagalay sii daynta xigta (soo saarista, soo saarista, tijaabinta, ...)
  2. install_venv.sh - waxay abuurtaa jawi muuqaal ah oo loogu talagalay qaybaha py ee buug gaar ah oo ku koobiyaya server-yada fog
  3. diyaarin_init.d.py - waxay u diyaarisaa qoraallada bilowga-joojinta ee qaybta ku salaysan qaabka
  4. hawlgelin.py - Burburiyaa oo dib u bilaabo qaybo cusub

Waqti ayaa dhaafey. Marxaladda diyaarinta waxa beddelay wax-soo-saar iyo wax-soo-saar. Taageero lagu daray badeecada hal qaybin dheeri ah (CentOS). Waxaa lagu daray 5 adeegayaal jireed oo awood badan iyo daraasiin kuwa casriga ah. Waxayna aad iyo aad ugu adkeyd kuwa horumariya iyo tijaabiyeyaashu in ay tijaabiyaan hawlahooda deegaanka in ka badan ama ka yar oo u dhow gobolka shaqada. Waqtigaan, waxaa caddaatay in aysan suurtagal ahayn in la sameeyo isaga la'aanteed ...

Qaybta II

Dhig Codsiyada Docker Swarm

Marka, kooxdeenu waa nidaam cajiib ah oo ka kooban dhowr iyo toban qaybood oo kala duwan oo aan lagu sifeynin Dockerfiles. Waxa kaliya oo aad u habeyn kartaa in la geeyo deegaan gaar ah guud ahaan. Hawsheenu waa in aan kooxda geyno meel lagu diyaariyo si loo tijaabiyo ka hor inta aan la sii dayn tijaabada ka hor.

Aragti ahaan, waxa jiri kara kooxo dhawr ah oo isku mar wada socda: inta ay jiraan hawlo gobolka la dhamaystiray ama ku dhaw dhamaystirka. Awoodaha server-yada aan gacanta ku hayno ayaa noo ogolaanaya inaan ku socodsiino dhowr rucubood oo ka mid ah server kasta. Kutlad kasta oo diyaarinta waa in ay ahaato mid go'doon ah (waa in aysan jirin isgoysyada dekedaha, hagaha, iwm.).

Khayraadkayaga ugu qiimaha badan waa wakhtigeena, wax badanna kamaanu haysan.

Bilawga degdega ah, waxaan dooranay Docker Swarm sababtoo ah fududaantiisa iyo dabacsanaantiisa qaab dhismeedka. Waxa ugu horreeya ee aan sameynay waxay ahayd abuurista maamule iyo dhowr nood oo ku yaal server-yada fog:

$ docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
kilqc94pi2upzvabttikrfr5d     nop-test-1     Ready               Active                                  19.03.2
jilwe56pl2zvabupryuosdj78     nop-test-2     Ready               Active                                  19.03.2
j5a4yz1kr2xke6b1ohoqlnbq5 *   nop-test-3     Ready               Active              Leader              19.03.2

Marka xigta, samee shabakad:


$ docker network create --driver overlay --subnet 10.10.10.0/24 nw_swarm

Marka xigta, waxaan ku xirnay Gitlab-CI iyo Swarm nodes marka la eego kontoroolka fog ee noodhka CI: rakibidda shahaadooyinka, dejinta doorsoomayaasha sirta ah, iyo dejinta adeegga Docker ee server-ka kantaroolka. Midkan maqaal waqti badan ayaa naga badbaadiyay.

Marka xigta, waxaan ku darnay shaqo abuurista iyo burburinta .gitlab-ci .yml.

Dhowr shaqo oo kale ayaa lagu daray .gitlab-ci .yml

## staging stage
deploy_staging:
  stage: testing
  before_script:
    - echo "override global 'before_script'"
  image: "REGISTRY:5000/docker:latest"
  environment: staging
  dependencies: []
  variables:
    DOCKER_CERT_PATH: "/certs"
    DOCKER_HOST: tcp://10.50.173.107:2376
    DOCKER_TLS_VERIFY: 1
    CI_BIN_DEPENDENCIES_JOB: "release.centos.7"
  script:
    - mkdir -p $DOCKER_CERT_PATH
    - echo "$TLSCACERT" > $DOCKER_CERT_PATH/ca.pem
    - echo "$TLSCERT" > $DOCKER_CERT_PATH/cert.pem
    - echo "$TLSKEY" > $DOCKER_CERT_PATH/key.pem
    - docker stack deploy -c docker-compose.yml ${CI_ENVIRONMENT_NAME}_${CI_COMMIT_REF_NAME} --with-registry-auth
    - rm -rf $DOCKER_CERT_PATH
  when: manual

## stop staging stage
stop_staging:
  stage: testing
  before_script:
    - echo "override global 'before_script'"
  image: "REGISTRY:5000/docker:latest"
  environment: staging
  dependencies: []
  variables:
    DOCKER_CERT_PATH: "/certs"
    DOCKER_HOST: tcp://10.50.173.107:2376
    DOCKER_TLS_VERIFY: 1
  script:
    - mkdir -p $DOCKER_CERT_PATH
    - echo "$TLSCACERT" > $DOCKER_CERT_PATH/ca.pem
    - echo "$TLSCERT" > $DOCKER_CERT_PATH/cert.pem
    - echo "$TLSKEY" > $DOCKER_CERT_PATH/key.pem
    - docker stack rm ${CI_ENVIRONMENT_NAME}_${CI_COMMIT_REF_NAME}
    # TODO: need check that stopped
  when: manual

Koodhka kore ee jajaban, waxaad arki kartaa in laba badhan (deploy_staging, stop_staging) lagu daray Pipelines, una baahan ficil gacanta ah.

Dhig Codsiyada Docker Swarm
Magaca raasamaalku waxa uu la mid yahay magaca laanta oo middani waa inay ku filnaataa. Adeegyada ku jira xidhmooyinku waxay helayaan ciwaanno ip gaar ah, iyo dekedo, hagayaal, iwm. waa la go'doomi doonaa, laakiin isku mid laga bilaabo raso ilaa raso (maxaa yeelay faylka qaabeynta waa isku mid dhammaan xirmooyinka) - waxa aan rabnay. Waxaan geynnaa xirmada (cluster) annagoo isticmaalna docker-compose.yml, kaas oo qeexaya kooxdeena.

docker-compose.yml

---
version: '3'

services:
  userprop:
    image: redis:alpine
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:
  celery_bcd:
    image: redis:alpine
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  schedulerdb:
    image: mariadb:latest
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
      MYSQL_DATABASE: schedulerdb
      MYSQL_USER: ****
      MYSQL_PASSWORD: ****
    command: ['--character-set-server=utf8mb4', '--collation-server=utf8mb4_unicode_ci', '--explicit_defaults_for_timestamp=1']
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  celerydb:
    image: mariadb:latest
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
      MYSQL_DATABASE: celerydb
      MYSQL_USER: ****
      MYSQL_PASSWORD: ****
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  cluster:
    image: $CENTOS7
    environment:
      - CENTOS
      - CI_ENVIRONMENT_NAME
      - CI_API_V4_URL
      - CI_REPOSITORY_URL
      - CI_PROJECT_ID
      - CI_PROJECT_URL
      - CI_PROJECT_PATH
      - CI_PROJECT_NAME
      - CI_COMMIT_REF_NAME
      - CI_BIN_DEPENDENCIES_JOB
    command: >
      sudo -u myusername -H /bin/bash -c ". /etc/profile &&
        mkdir -p /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME &&
        cd /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME &&
            git clone -b $CI_COMMIT_REF_NAME $CI_REPOSITORY_URL . &&
            curl $CI_API_V4_URL/projects/$CI_PROJECT_ID/jobs/artifacts/$CI_COMMIT_REF_NAME/download?job=$CI_BIN_DEPENDENCIES_JOB -o artifacts.zip &&
            unzip artifacts.zip ;
        cd /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME/scripts/deploy/ &&
            python3 createconfig.py -s $CI_ENVIRONMENT_NAME &&
            /bin/bash install_venv.sh -d -r ../../requirements.txt &&
            python3 prepare_init.d.py &&
            python3 deploy.py -s $CI_ENVIRONMENT_NAME"
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    tty: true
    stdin_open: true
    networks:
      nw_swarm:

networks:
  nw_swarm:
    external: true

Halkan waxaad ku arki kartaa in qaybaha ay ku xiran yihiin hal shabakad (nw_swarm) oo ay diyaar u yihiin midba midka kale.

Qaybaha nidaamka (ku salaysan redis, mysql) ayaa laga soocay barkada guud ee qaybaha gaarka ah (qorshayaasha iyo kuwa caadiga ah ayaa loo qaybiyaa adeegyo ahaan). Marxaladda geynta kooxdeenu waxay u eegtahay u gudbinta CMD ee hal sawir oo la habeeyay, guud ahaan, kama duwana hawlgelinta lagu sheegay Qaybta I. Waxaan iftiimin doonaa kala duwanaanshaha:

  • git clone... - Hel faylasha loo baahan yahay si loo geeyo (createconfig.py, install_venv.sh, iwm.)
  • curl... && fur... - soo deji oo fur qalabka dhismaha (utiliyada la soo uruuriyay)

Waxaa jirta hal dhibaato oo kaliya oo aan weli la qeexin: qaybaha leh interface interneedka lagama heli karo daalacashada horumariyeyaasha. Waxaan ku xallinaa dhibaatadan annagoo adeegsanayna wakiil-ka-noqoshada, sida:

Gudaha .gitlab-ci.yml, ka dib markii la geeyo xirmooyinka kooxda, waxaan ku darnaa xariiqda geynta dheelitirka (kaas oo, marka uu sameeyo, kaliya wuxuu cusbooneysiiyaa qaabeyntiisa (wuxuu abuuraa faylal cusub oo qaabeynta nginx iyadoo loo eegayo qaabka: /etc/nginx/conf. d/${CI_COMMIT_REF_NAME}.conf) - eeg docker-compose-nginx.yml code)

    - docker stack deploy -c docker-compose-nginx.yml ${CI_ENVIRONMENT_NAME} --with-registry-auth

docker-ka kooban-nginx.yml

---
version: '3'

services:
  nginx:
    image: nginx:latest
    environment:
      CI_COMMIT_REF_NAME: ${CI_COMMIT_REF_NAME}
      NGINX_CONFIG: |-
            server {
                listen 8080;
                server_name staging_${CI_COMMIT_REF_NAME}_cluster.dev;

                location / {
                    proxy_pass http://staging_${CI_COMMIT_REF_NAME}_cluster:8080;
                }
            }
            server {
                listen 5555;
                server_name staging_${CI_COMMIT_REF_NAME}_cluster.dev;

                location / {
                    proxy_pass http://staging_${CI_COMMIT_REF_NAME}_cluster:5555;
                }
            }
    volumes:
      - /tmp/staging/nginx:/etc/nginx/conf.d
    command:
      /bin/bash -c "echo -e "$$NGINX_CONFIG" > /etc/nginx/conf.d/${CI_COMMIT_REF_NAME}.conf;
        nginx -g "daemon off;";
        /etc/init.d/nginx reload"
    ports:
      - 8080:8080
      - 5555:5555
      - 3000:3000
      - 443:443
      - 80:80
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

networks:
  nw_swarm:
    external: true

Kumbiyuutarrada horumarinta, cusbooneysii /etc/hosts; Url u qor nginx:

10.50.173.106 staging_BRANCH-1831_cluster.dev

Markaa, daabulidda kooxaha diyaarinta ee go'doonsan waa la hirgeliyey, horumariyayaashuna hadda waxay ku wadi karaan tiro kasta oo ku filan si loo hubiyo hawlahooda.

Qorshayaasha mustaqbalka:

  • Kala saar qaybahayaga adeeg ahaan
  • U hayso Dockerfile kasta
  • Si toos ah u ogow noodhadhka raran ee ku jira xidhmada
  • Ku qeex noodhka qaabka magaca (halkii aad isticmaali lahayd id sida ku qoran maqaalka)
  • Ku dar jeeg in raasasku burburay
  • ...

mahad gaar ah maqaal.

Source: www.habr.com

Add a comment