I-ProHoster > Блог > Ukuphatha > Hambisa izinhlelo zokusebenza usebenzisa i-Docker Swarm

Hambisa izinhlelo zokusebenza usebenzisa i-Docker Swarm

Uhlelo lokuncoma okuqukethwe kwamavidiyo aku-inthanethi esisebenza kulo luwukuthuthukiswa kokuhweba okuvaliwe futhi ngokobuchwepheshe luyiqoqo lezingxenye eziningi zezingxenye zobunikazi nemithombo evulekile. Inhloso yokubhala lesi sihloko ukuchaza ukuqaliswa kwesistimu yeqoqo le-docker swarm yesikhulumi sesiteji, ngaphandle kokuphazamisa ukuhamba komsebenzi okumisiwe kwezinqubo zethu ngaphansi kwezimo zesikhathi esilinganiselwe. Ukulandisa okwethulwe ekunakeni kwakho kuhlukaniswe izingxenye ezimbili. Ingxenye yokuqala ichaza i-CI/CD ngaphambi kokusebenzisa i-docker swarm, futhi ingxenye yesibili ichaza inqubo yokuyisebenzisa. Labo abangenaso isithakazelo sokufunda ingxenye yokuqala bangadlulela kweyesibili ngokuphepha.

Iасть Mina

Kwake kwaba nesidingo sokumisa inqubo ye-CI/CD ngokushesha okukhulu. Omunye wemibandela bekuwukungasebenzisi i-Docker ukuthunyelwa izingxenye ezakhiwa ngenxa yezizathu eziningana:

  • ukuze uthole ukusebenza okuthembekile nokuzinzile kwezingxenye Zokukhiqiza (okungukuthi, empeleni, imfuneko yokungasebenzisi i-virtualization)
  • abathuthukisi abahamba phambili bebengafuni ukusebenzisana ne-Docker (okumangazayo, kodwa kwaba kanjalo)
  • ngezizathu ezingokwengqondo zokuphathwa kwe-R&D

Ingqalasizinda, isitaki kanye nezimfuneko zokuqala ezilinganiselwe ze-MVP bezimi kanje:

  • Amaseva angu-4 e-Intel® X5650 ane-Debian (umshini owodwa onamandla ngokuphelele wokuthuthukiswa)
  • Ukuthuthukiswa kwezingxenye zakho zangokwezifiso kwenziwa ku-C++, Python3
  • Amathuluzi amakhulu enkampani yangaphandle asetshenzisiwe: Kafka, Clickhouse, Airflow, Redis, Grafana, Postgresql, Mysql, ...
  • Amapayipi okwakha nokuhlola izingxenye ngokwehlukana ukuze kulungiswe iphutha nokukhululwa

Omunye wemibuzo yokuqala okufanele ixazululwe ekuqaleni ukuthi izingxenye zangokwezifiso zizosetshenziswa kanjani kunoma iyiphi indawo (CI/CD).

Sinqume ukufaka izingxenye zenkampani yangaphandle ngohlelo futhi sizibuyekeze ngohlelo. Izinhlelo zokusebenza ezenziwe ngokwezifiso ezakhiwe ku-C++ noma i-Python zingasatshalaliswa ngezindlela eziningi. Phakathi kwazo, isibonelo: ukudala amaphakheji wesistimu, ukuwathumela endaweni yokugcina izithombe eziqoqiwe kanye nokufakwa kwazo okulandelayo kumaseva. Ngesizathu esingaziwa kakade, kwakhethwa enye indlela, okungukuthi: kusetshenziswa i-CI, kuhlanganiswa amafayela angasebenziseka kalula, kwakhiwa indawo yephrojekthi ebonakalayo, kufakwa amamojula we-py asuka ku-requirements.txt, futhi zonke lezi zinto zobuciko zithunyelwa kanye ne-configs, imibhalo kanye indawo yohlelo lokusebenza ehambisana namaseva. Okulandelayo, izinhlelo zokusebenza ziqaliswa kumsebenzisi ongekho emthethweni ngaphandle kwamalungelo omlawuli.

I-Gitlab-CI ikhethwe njengohlelo lwe-CI/CD. Umphumela wepayipi wawubukeka kanjena:

Hambisa izinhlelo zokusebenza usebenzisa i-Docker Swarm
Ngokwesakhiwo, i-gitlab-ci.yml ibukeke kanje:

---
variables:
  # минимальная версия ЦПУ на серверах, где разворачивается кластер
  CMAKE_CPUTYPE: "westmere"

  DEBIAN: "MYREGISTRY:5000/debian:latest"

before_script:
  - eval $(ssh-agent -s)
  - ssh-add <(echo "$SSH_PRIVATE_KEY")
  - mkdir -p ~/.ssh && echo -e "Host *ntStrictHostKeyChecking nonn" > ~/.ssh/config

stages:
  - build
  - testing
  - deploy

debug.debian:
  stage: build
  image: $DEBIAN
  script:
    - cd builds/release && ./build.sh
    paths:
      - bin/
      - builds/release/bin/
    when: always
release.debian:
  stage: build
  image: $DEBIAN
  script:
    - cd builds/release && ./build.sh
    paths:
      - bin/
      - builds/release/bin/
    when: always

## testing stage
tests.codestyle:
  stage: testing
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - /bin/bash run_tests.sh -t codestyle -b "${CI_COMMIT_REF_NAME}_codestyle"
tests.debug.debian:
  stage: testing
  image: $DEBIAN
  dependencies:
    - debug.debian
  script:
    - /bin/bash run_tests.sh -e codestyle/test_pylint.py -b "${CI_COMMIT_REF_NAME}_debian_debug"
  artifacts:
    paths:
      - run_tests/username/
    when: always
    expire_in: 1 week
tests.release.debian:
  stage: testing
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - /bin/bash run_tests.sh -e codestyle/test_pylint.py -b "${CI_COMMIT_REF_NAME}_debian_release"
  artifacts:
    paths:
      - run_tests/username/
    when: always
    expire_in: 1 week

## staging stage
deploy_staging:
  stage: deploy
  environment: staging
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - cd scripts/deploy/ &&
        python3 createconfig.py -s $CI_ENVIRONMENT_NAME &&
        /bin/bash install_venv.sh -d -r ../../requirements.txt &&
        python3 prepare_init.d.py &&
        python3 deploy.py -s $CI_ENVIRONMENT_NAME
  when: manual

Kuyaqapheleka ukuthi ukuhlangana nokuhlola kwenziwa ngomfanekiso wayo, lapho wonke amaphakheji wesistimu adingekayo esevele efakiwe nezinye izilungiselelo zenziwa.

Nakuba ngayinye yalezi scripts emisebenzini ithakazelisa ngendlela yayo, ngokuqinisekile ngeke ngikhulume ngayo; ukuchaza ngayinye yazo kuzothatha isikhathi eside futhi lokhu akuyona inhloso ye-athikili. Ake ngivele ngidonse ukunaka kwakho eqinisweni lokuthi isigaba sokuphakelwa siqukethe ukulandelana kwemibhalo yokubiza:

  1. createconfig.py - idala ifayela le-setting.ini elinezilungiselelo zezingxenye ezisendaweni ehlukene ukuze zisetshenziswe ngokulandelayo (Ukukhiqiza, Ukukhiqiza, Ukuhlola, ...)
  2. install_venv.sh - idala indawo ebonakalayo yezingxenye ze-py kuhla lwemibhalo ethile futhi ikopishelwe kumaseva akude
  3. lungiselela_init.d.py — ilungiselela imibhalo yezingxenye zokuqalisa ezisekelwe kusifanekiso
  4. deploy.py - sebenzisa futhi iqale kabusha izingxenye ezintsha

Sahamba isikhathi. Isiteji sesiteji sathathelwa indawo ukukhiqizwa nokukhiqizwa. Ukusekelwa komkhiqizo kwengezwe ekusatshalalisweni okukodwa okwengeziwe (i-CentOS). Amanye amaseva anamandla angu-5 kanye nama-virtual ayishumi nambili angeziwe. Futhi kuye kwaba nzima kakhulu konjiniyela nabahloli ukuhlola imisebenzi yabo endaweni eseduze noma kancane nesimo sokusebenza. Ngalesi sikhathi kwacaca ukuthi akunakwenzeka ukwenza ngaphandle kwakhe ...

Ingxenye II

Hambisa izinhlelo zokusebenza usebenzisa i-Docker Swarm

Ngakho-ke, iqoqo lethu liwuhlelo olumangalisayo lwezingxenye ezimbalwa ezingachazwanga yi-Dockerfiles. Ungayilungiselela ukuthi isetshenziswe endaweni ethile kuphela ngokujwayelekile. Umsebenzi wethu uwukusebenzisa iqoqo endaweni yesiteji ukuze silihlole ngaphambi kokuhlolwa kokukhishwa kwangaphambilini.

Ngokwetiyori, kungase kube namaqoqo amaningana asebenza ngesikhathi esisodwa: miningi kangangoba kunemisebenzi esesimweni esiqediwe noma esiseduze nokuqedwa. Amandla amaseva esinawo asivumela ukuthi sisebenzise amaqoqo amaningana kuseva ngayinye. Iqoqo ngalinye lesiteji kufanele lihlukaniswe (akumele kube nokugqagqana ezimbobeni, kuhlu lwemibhalo, njll.).

Insiza yethu eyigugu kakhulu yisikhathi sethu, futhi asizange sibe nakho okuningi.

Ukuze siqale ngokushesha, sikhethe i-Docker Swarm ngenxa yobulula bayo nesakhiwo esivumelana nezimo. Into yokuqala esiyenzile kwakha umphathi namanodi amaningana kumaseva akude:

$ docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
kilqc94pi2upzvabttikrfr5d     nop-test-1     Ready               Active                                  19.03.2
jilwe56pl2zvabupryuosdj78     nop-test-2     Ready               Active                                  19.03.2
j5a4yz1kr2xke6b1ohoqlnbq5 *   nop-test-3     Ready               Active              Leader              19.03.2

Okulandelayo, sidale inethiwekhi:


$ docker network create --driver overlay --subnet 10.10.10.0/24 nw_swarm

Okulandelayo, sixhume ama-node e-Gitlab-CI kanye ne-Swarm mayelana nokuphathwa okukude kwama-node kusuka ku-CI: ukufaka izitifiketi, ukusetha okuguquguqukayo okuyimfihlo, futhi nokumisa isevisi ye-Docker kuseva yokulawula. Lena indatshana usisindise isikhathi esiningi.

Okulandelayo, sengeze imisebenzi yokudala nokucekela phansi isitaki kokuthi .gitlab-ci .yml.

Eminye imisebenzi eminingana yengezwe ku-.gitlab-ci .yml

## staging stage
deploy_staging:
  stage: testing
  before_script:
    - echo "override global 'before_script'"
  image: "REGISTRY:5000/docker:latest"
  environment: staging
  dependencies: []
  variables:
    DOCKER_CERT_PATH: "/certs"
    DOCKER_HOST: tcp://10.50.173.107:2376
    DOCKER_TLS_VERIFY: 1
    CI_BIN_DEPENDENCIES_JOB: "release.centos.7"
  script:
    - mkdir -p $DOCKER_CERT_PATH
    - echo "$TLSCACERT" > $DOCKER_CERT_PATH/ca.pem
    - echo "$TLSCERT" > $DOCKER_CERT_PATH/cert.pem
    - echo "$TLSKEY" > $DOCKER_CERT_PATH/key.pem
    - docker stack deploy -c docker-compose.yml ${CI_ENVIRONMENT_NAME}_${CI_COMMIT_REF_NAME} --with-registry-auth
    - rm -rf $DOCKER_CERT_PATH
  when: manual

## stop staging stage
stop_staging:
  stage: testing
  before_script:
    - echo "override global 'before_script'"
  image: "REGISTRY:5000/docker:latest"
  environment: staging
  dependencies: []
  variables:
    DOCKER_CERT_PATH: "/certs"
    DOCKER_HOST: tcp://10.50.173.107:2376
    DOCKER_TLS_VERIFY: 1
  script:
    - mkdir -p $DOCKER_CERT_PATH
    - echo "$TLSCACERT" > $DOCKER_CERT_PATH/ca.pem
    - echo "$TLSCERT" > $DOCKER_CERT_PATH/cert.pem
    - echo "$TLSKEY" > $DOCKER_CERT_PATH/key.pem
    - docker stack rm ${CI_ENVIRONMENT_NAME}_${CI_COMMIT_REF_NAME}
    # TODO: need check that stopped
  when: manual

Kusukela esiqeshini sekhodi esingenhla kuyacaca ukuthi izinkinobho ezimbili zengezwe ku-Pipelines (deploy_staging, stop_staging) ezidinga isenzo sesandla.

Hambisa izinhlelo zokusebenza usebenzisa i-Docker Swarm
Igama lesitaki lihambisana negama legatsha futhi lokhu kuhluka kufanele kube ngokwanele. Amasevisi esitaki athola amakheli e-IP ahlukile, nezimbobo, izinkomba, njll. zizohlukaniswa, kodwa okufanayo ukusuka kusitaki kuye kusitaki (njengoba ifayela lokumisa liyafana kuzo zonke izitaki) - yilokho ebesikufuna. Siphakela isitaki (iqoqo) sisebenzisa docker-compose.yml, echaza iqoqo lethu.

docker-compose.yml

---
version: '3'

services:
  userprop:
    image: redis:alpine
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:
  celery_bcd:
    image: redis:alpine
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  schedulerdb:
    image: mariadb:latest
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
      MYSQL_DATABASE: schedulerdb
      MYSQL_USER: ****
      MYSQL_PASSWORD: ****
    command: ['--character-set-server=utf8mb4', '--collation-server=utf8mb4_unicode_ci', '--explicit_defaults_for_timestamp=1']
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  celerydb:
    image: mariadb:latest
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
      MYSQL_DATABASE: celerydb
      MYSQL_USER: ****
      MYSQL_PASSWORD: ****
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  cluster:
    image: $CENTOS7
    environment:
      - CENTOS
      - CI_ENVIRONMENT_NAME
      - CI_API_V4_URL
      - CI_REPOSITORY_URL
      - CI_PROJECT_ID
      - CI_PROJECT_URL
      - CI_PROJECT_PATH
      - CI_PROJECT_NAME
      - CI_COMMIT_REF_NAME
      - CI_BIN_DEPENDENCIES_JOB
    command: >
      sudo -u myusername -H /bin/bash -c ". /etc/profile &&
        mkdir -p /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME &&
        cd /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME &&
            git clone -b $CI_COMMIT_REF_NAME $CI_REPOSITORY_URL . &&
            curl $CI_API_V4_URL/projects/$CI_PROJECT_ID/jobs/artifacts/$CI_COMMIT_REF_NAME/download?job=$CI_BIN_DEPENDENCIES_JOB -o artifacts.zip &&
            unzip artifacts.zip ;
        cd /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME/scripts/deploy/ &&
            python3 createconfig.py -s $CI_ENVIRONMENT_NAME &&
            /bin/bash install_venv.sh -d -r ../../requirements.txt &&
            python3 prepare_init.d.py &&
            python3 deploy.py -s $CI_ENVIRONMENT_NAME"
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    tty: true
    stdin_open: true
    networks:
      nw_swarm:

networks:
  nw_swarm:
    external: true

Lapha ungabona ukuthi izingxenye zixhunywe inethiwekhi eyodwa (nw_swarm) futhi zifinyeleleka komunye nomunye.

Izingxenye zesistimu (ngokusekelwe ku-redis, mysql) zihlukaniswa neqembu elijwayelekile lezingxenye zangokwezifiso (ezinhlelweni, izingxenye zangokwezifiso nazo zihlukaniswa njengezinsizakalo). Isigaba sokuphakelwa seqoqo lethu sibukeka njengokudlulisela i-CMD esithombeni sethu esisodwa esikhulu esimisiwe futhi, ngokuvamile, asihlukile neze ekusetshenzisweni okuchazwe Engxenyeni I. Ngizogcizelela umehluko:

  • git clone... - sithola amafayela adingekayo ukuze senze ukuthunyelwa (createconfig.py, install_venv.sh, njll.)
  • curl... && unzip... - landa futhi unzip izinto zokwakha (izinsiza ezihlanganisiwe)

Kunenkinga eyodwa kuphela okwamanje engachazwanga: izingxenye ezinokusebenzelana kwewebhu azifinyeleleki kuziphequluli zonjiniyela. Sixazulula le nkinga sisebenzisa ummeleli ohlanekezelwe, ngale ndlela:

Ku-.gitlab-ci.yml, ngemva kokukhipha isitaki se-cluster, engeza umugqa wokukhipha isilinganisi (okuthi, uma sizibophezele, sibuyekeze ukucushwa kwayo kuphela (kudala amafayela amasha wokucushwa kwe-nginx ngokuvumelana nesifanekiso: /etc/nginx/conf.d /${CI_COMMIT_REF_NAME}.conf) - bona ikhodi docker-compose-nginx.yml)

    - docker stack deploy -c docker-compose-nginx.yml ${CI_ENVIRONMENT_NAME} --with-registry-auth

i-docker-compose-nginx.yml

---
version: '3'

services:
  nginx:
    image: nginx:latest
    environment:
      CI_COMMIT_REF_NAME: ${CI_COMMIT_REF_NAME}
      NGINX_CONFIG: |-
            server {
                listen 8080;
                server_name staging_${CI_COMMIT_REF_NAME}_cluster.dev;

                location / {
                    proxy_pass http://staging_${CI_COMMIT_REF_NAME}_cluster:8080;
                }
            }
            server {
                listen 5555;
                server_name staging_${CI_COMMIT_REF_NAME}_cluster.dev;

                location / {
                    proxy_pass http://staging_${CI_COMMIT_REF_NAME}_cluster:5555;
                }
            }
    volumes:
      - /tmp/staging/nginx:/etc/nginx/conf.d
    command:
      /bin/bash -c "echo -e "$$NGINX_CONFIG" > /etc/nginx/conf.d/${CI_COMMIT_REF_NAME}.conf;
        nginx -g "daemon off;";
        /etc/init.d/nginx reload"
    ports:
      - 8080:8080
      - 5555:5555
      - 3000:3000
      - 443:443
      - 80:80
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

networks:
  nw_swarm:
    external: true

Kumakhompyutha kanjiniyela, buyekeza /etc/hosts; setha i-url ku-nginx:

10.50.173.106 staging_BRANCH-1831_cluster.dev

Ngakho-ke, ukuthunyelwa kwamaqoqo esiteji ahlukanisiwe sekuqalisiwe futhi onjiniyela manje bangawaqalisa nganoma yiliphi inani elanele ukuhlola imisebenzi yabo.

Izinhlelo zesikhathi esizayo:

  • Hlukanisa izingxenye zethu njengamasevisi
  • Dala i-Dockerfile ngayinye
  • Thola ngokuzenzakalelayo ama-node angalayishiwe kancane kusitaki
  • Cacisa ama-node usebenzisa isifanekiso segama (kunokuba usebenzise i-id njengaku-athikili)
  • Engeza isheke lokuthi isitaki sicekeleke phansi
  • ...

Siyabonga ngokukhethekile indatshana.

Source: www.habr.com

Engeza amazwana