ProHoster > Ibhulogi > Ulawulo > Faka izicelo usebenzisa i-Docker Swarm

Faka izicelo usebenzisa i-Docker Swarm

Inkqubo yesindululo somxholo wevidiyo ekwi-intanethi sisebenza kuyo luphuhliso lorhwebo oluvaliweyo kwaye ngokobuchwepheshe liqela lamacandelo amaninzi amacandelo obunikazi kunye nomthombo ovulekileyo. Injongo yokubhala eli nqaku kukuchaza ukuphunyezwa kwenkqubo ye-docker swarm clustering yeqonga leqonga, ngaphandle kokuphazamisa ukuhamba komsebenzi okusekiweyo kweenkqubo zethu phantsi kweemeko zexesha elilinganiselweyo. Ingxelo enikelwe ingqalelo yakho yahlulwe yangamacandelo amabini. Inxalenye yokuqala ichaza i-CI / CD ngaphambi kokusebenzisa i-docker swarm, kwaye inxalenye yesibini ichaza inkqubo yokuyiphumeza. Abo bangenamdla wokufunda inxalenye yokuqala banokudlulela ngokukhuselekileyo ukuya kweyesibini.

Icandelo I

Kanye ngelo xesha, kwakukho imfuneko yokuseka inkqubo yeCI / CD ngokukhawuleza. Enye yeemeko yayikukusebenzisa iDocker ukusasazwa amacandelo aphuhliswa ngenxa yezizathu ezininzi:

  • ukusebenza okuthembekileyo ngakumbi kunye nokuzinza kwamacandelo kwiMveliso (oko kukuthi, ngokwenyani, imfuno yokungasebenzisi i-virtualization)
  • abaphuhlisi abaphambili abakhange bafune ukusebenza noDocker (okungaqhelekanga, kodwa kwakunjalo)
  • ngenxa yezizathu zengqondo zolawulo lweR&D

Iziseko zophuhliso, imfumba kunye neemfuneko eziqikelelweyo zokuqala zeMVP bezimi ngolu hlobo lulandelayo:

  • Iiseva ezi-4 ze-Intel® X5650 ezineDebian (omnye umatshini onamandla ngakumbi wophuhliso)
  • Uphuhliso lwamacandelo akho esiko lwenziwa kwiC ++, Python3
  • Izixhobo eziphambili zeqela lesithathu ezisetyenzisiweyo: Kafka, Clickhouse, Airflow, Redis, Grafana, Postgresql, Mysql, ...
  • Imibhobho yokwakha kunye nokuvavanya amacandelo ngokwahlukeneyo okulungisa kunye nokukhululwa

Omnye wemibuzo yokuqala ekufuneka isonjululwe kwinqanaba lokuqala yindlela amacandelo esiqhelo aya kuhanjiswa ngayo kuyo nayiphi na imeko (CI/CD).

Sigqibe ekubeni sifakele izinto zomntu wesithathu ngokwenkqubo kwaye sizihlaziye ngokwenkqubo. Usetyenziso oluqhelekileyo oluphuhliswe kwiC ++ okanye kwiPython lunokuthunyelwa ngeendlela ezininzi. Phakathi kwabo, umzekelo: ukudala iipakethi zenkqubo, ukuzithumela kwindawo yokugcina imifanekiso eqokelelweyo kunye nokufakwa kwazo okulandelayo kwiiseva. Ngesizathu esele singaziwayo, kwakhethwa enye indlela, eyile: usebenzisa iCI, iifayile eziphunyeziweyo zesicelo ziyaqokelelwa, indawo yeprojekti enenyani yenziwa, iimodyuli zepy ezivela kwizidingo.txt zifakiwe, kwaye zonke ezi zixhobo zithunyelwa kunye configs, imibhalo kunye indawo ekhaphayo yesicelo ukuya kubancedisi. Okulandelayo, izicelo ziqaliswa ukusuka kumsebenzisi wenyani ngaphandle kwamalungelo omlawuli.

I-Gitlab-CI yakhethwa njengenkqubo ye-CI / CD. Isiphumo sombhobho sijonge ngolu hlobo:

Faka izicelo usebenzisa i-Docker Swarm
Ngokwesakhiwo, i-gitlab-ci.yml yayijongeka ngolu hlobo:

---
variables:
  # минимальная версия ЦПУ на серверах, где разворачивается кластер
  CMAKE_CPUTYPE: "westmere"

  DEBIAN: "MYREGISTRY:5000/debian:latest"

before_script:
  - eval $(ssh-agent -s)
  - ssh-add <(echo "$SSH_PRIVATE_KEY")
  - mkdir -p ~/.ssh && echo -e "Host *ntStrictHostKeyChecking nonn" > ~/.ssh/config

stages:
  - build
  - testing
  - deploy

debug.debian:
  stage: build
  image: $DEBIAN
  script:
    - cd builds/release && ./build.sh
    paths:
      - bin/
      - builds/release/bin/
    when: always
release.debian:
  stage: build
  image: $DEBIAN
  script:
    - cd builds/release && ./build.sh
    paths:
      - bin/
      - builds/release/bin/
    when: always

## testing stage
tests.codestyle:
  stage: testing
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - /bin/bash run_tests.sh -t codestyle -b "${CI_COMMIT_REF_NAME}_codestyle"
tests.debug.debian:
  stage: testing
  image: $DEBIAN
  dependencies:
    - debug.debian
  script:
    - /bin/bash run_tests.sh -e codestyle/test_pylint.py -b "${CI_COMMIT_REF_NAME}_debian_debug"
  artifacts:
    paths:
      - run_tests/username/
    when: always
    expire_in: 1 week
tests.release.debian:
  stage: testing
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - /bin/bash run_tests.sh -e codestyle/test_pylint.py -b "${CI_COMMIT_REF_NAME}_debian_release"
  artifacts:
    paths:
      - run_tests/username/
    when: always
    expire_in: 1 week

## staging stage
deploy_staging:
  stage: deploy
  environment: staging
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - cd scripts/deploy/ &&
        python3 createconfig.py -s $CI_ENVIRONMENT_NAME &&
        /bin/bash install_venv.sh -d -r ../../requirements.txt &&
        python3 prepare_init.d.py &&
        python3 deploy.py -s $CI_ENVIRONMENT_NAME
  when: manual

Kuyafaneleka ukuba uqaphele ukuba indibano kunye novavanyo lwenziwa kumfanekiso walo, apho zonke iipakethi zenkqubo eziyimfuneko sele zifakwe kwaye ezinye izicwangciso zenziwe.

Nangona nganye yezi scripts kwimisebenzi inomdla ngendlela yayo, ngokuqinisekileyo andizukuthetha ngayo; Ukuchaza nganye kuzo kuya kuthatha ixesha elininzi kwaye ayisiyonjongo yenqaku le. Makhe nditsalele ingqalelo yakho kwinto yokuba inqanaba lokusasazwa libandakanya ulandelelwano lwemibhalo yokufowuna:

  1. dalaconfig.py - yenza ifayile yesethingi.ini kunye nezicwangciso zamacandelo kwiindawo ezahlukeneyo zokusasazwa okulandelayo (Ukuveliswa kwangaphambili, ukuveliswa, ukuvavanywa, ...)
  2. install_venv.sh — yenza imeko-bume yenyani yamacandelo epy kulawulo oluthile kwaye ikhuphele kwiiseva ezikude
  3. lungiselela_init.d.py — ulungisa izikripthi zamalungu okuqalisa-yokuyeka ngokusekwe kwithempleyithi
  4. deploy.py - sebenzisa kwaye uqalise kwakhona izixhobo ezintsha

Lahamba ixesha. Inqanaba leqonga lithatyathelw' indawo yimveliso yangaphambili kunye nemveliso. Inkxaso yemveliso yongezwe kwi-distribution enye ngaphezulu (CentOS). Ezinye iiseva ezi-5 ezinamandla kunye neshumi elinesibini lenyani zongezwa. Kwaye kuye kwaba nzima ngakumbi kubaphuhlisi kunye nabavavanyi ukuvavanya imisebenzi yabo kwindawo ekufutshane okanye ekufutshane nemeko yokusebenza. Ngeli xesha kwacaca ukuba akunakwenzeka ukwenza ngaphandle kwakhe ...

Icandelo II

Faka izicelo usebenzisa i-Docker Swarm

Ke, iqela lethu yinkqubo ebukekayo yezinto ezimbalwa ezingachazwanga yiDockerfiles. Ungayiqwalasela ukuba isetyenziswe kwindawo ethile kuphela ngokubanzi. Umsebenzi wethu kukubeka iqela kwindawo yeqonga ukuyivavanya phambi kovavanyo lokukhutshwa kwangaphambili.

Ngokwethiyori, kunokubakho amaqela aliqela asebenza ngaxeshanye: kangangoko kukho imisebenzi ekwimeko egqityiweyo okanye ekufutshane ekugqityweni. Amandla amaseva esinawo avumela ukuba siqhube amaqela amaninzi kwiseva nganye. Nganye iklasta yeqonga mayibe yodwa (kufuneka kungabikho kuthungelana kumazibuko, kuluhlu, njl. njl.).

Owona mthombo wethu uxabisekileyo lixesha lethu, kwaye besingenalo kakhulu.

Ukuqala ngokukhawuleza, sikhethe iDocker Swarm ngenxa yokulula kwayo kunye noyilo oluguqukayo. Into yokuqala esiyenzileyo kukudala umphathi kunye neendawo ezininzi kwiiseva ezikude:

$ docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
kilqc94pi2upzvabttikrfr5d     nop-test-1     Ready               Active                                  19.03.2
jilwe56pl2zvabupryuosdj78     nop-test-2     Ready               Active                                  19.03.2
j5a4yz1kr2xke6b1ohoqlnbq5 *   nop-test-3     Ready               Active              Leader              19.03.2

Okulandelayo, senze inethiwekhi:


$ docker network create --driver overlay --subnet 10.10.10.0/24 nw_swarm

Emva koko, siqhagamshele i-Gitlab-CI kunye ne-Swarm nodes ngokubhekiselele kulawulo olukude lwee-nodes ukusuka kwi-CI: ukufaka izatifikethi, ukuseta iinguqu eziyimfihlo, kunye nokuseta inkonzo ye-Docker kwi-server yokulawula. Le inqaku sisisindise ixesha elininzi.

Emva koko, songeze imisebenzi yokudala kunye nokutshabalalisa isitaki kwi-.gitlab-ci .yml.

Imisebenzi emininzi eyongeziweyo yongezwe kwi-.gitlab-ci .yml

## staging stage
deploy_staging:
  stage: testing
  before_script:
    - echo "override global 'before_script'"
  image: "REGISTRY:5000/docker:latest"
  environment: staging
  dependencies: []
  variables:
    DOCKER_CERT_PATH: "/certs"
    DOCKER_HOST: tcp://10.50.173.107:2376
    DOCKER_TLS_VERIFY: 1
    CI_BIN_DEPENDENCIES_JOB: "release.centos.7"
  script:
    - mkdir -p $DOCKER_CERT_PATH
    - echo "$TLSCACERT" > $DOCKER_CERT_PATH/ca.pem
    - echo "$TLSCERT" > $DOCKER_CERT_PATH/cert.pem
    - echo "$TLSKEY" > $DOCKER_CERT_PATH/key.pem
    - docker stack deploy -c docker-compose.yml ${CI_ENVIRONMENT_NAME}_${CI_COMMIT_REF_NAME} --with-registry-auth
    - rm -rf $DOCKER_CERT_PATH
  when: manual

## stop staging stage
stop_staging:
  stage: testing
  before_script:
    - echo "override global 'before_script'"
  image: "REGISTRY:5000/docker:latest"
  environment: staging
  dependencies: []
  variables:
    DOCKER_CERT_PATH: "/certs"
    DOCKER_HOST: tcp://10.50.173.107:2376
    DOCKER_TLS_VERIFY: 1
  script:
    - mkdir -p $DOCKER_CERT_PATH
    - echo "$TLSCACERT" > $DOCKER_CERT_PATH/ca.pem
    - echo "$TLSCERT" > $DOCKER_CERT_PATH/cert.pem
    - echo "$TLSKEY" > $DOCKER_CERT_PATH/key.pem
    - docker stack rm ${CI_ENVIRONMENT_NAME}_${CI_COMMIT_REF_NAME}
    # TODO: need check that stopped
  when: manual

Ukususela kwisiqwenga sekhowudi engentla kuyacaca ukuba amaqhosha amabini afakwe kwiiPipelines (deploy_staging, stop_staging) ezifuna isenzo sesandla.

Faka izicelo usebenzisa i-Docker Swarm
Igama lestakhi lihambelana negama lesebe kwaye oku kungafaniyo kufuneka kube ngokwaneleyo. Iinkonzo kwi-stack zifumana iidilesi ze-IP ezizodwa, kunye namachweba, abalawuli, njl. iyakwahlulwa, kodwa iyafana ukusuka kwisitaki ukuya kwisitaki (ekubeni ifayile yoqwalaselo iyafana kuzo zonke izitaki) - yile nto besiyifuna. Sibeka isitaki (iqela) sisebenzisa docker-compose.yml, echaza iqela lethu.

docker-compose.yml

---
version: '3'

services:
  userprop:
    image: redis:alpine
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:
  celery_bcd:
    image: redis:alpine
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  schedulerdb:
    image: mariadb:latest
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
      MYSQL_DATABASE: schedulerdb
      MYSQL_USER: ****
      MYSQL_PASSWORD: ****
    command: ['--character-set-server=utf8mb4', '--collation-server=utf8mb4_unicode_ci', '--explicit_defaults_for_timestamp=1']
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  celerydb:
    image: mariadb:latest
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
      MYSQL_DATABASE: celerydb
      MYSQL_USER: ****
      MYSQL_PASSWORD: ****
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  cluster:
    image: $CENTOS7
    environment:
      - CENTOS
      - CI_ENVIRONMENT_NAME
      - CI_API_V4_URL
      - CI_REPOSITORY_URL
      - CI_PROJECT_ID
      - CI_PROJECT_URL
      - CI_PROJECT_PATH
      - CI_PROJECT_NAME
      - CI_COMMIT_REF_NAME
      - CI_BIN_DEPENDENCIES_JOB
    command: >
      sudo -u myusername -H /bin/bash -c ". /etc/profile &&
        mkdir -p /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME &&
        cd /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME &&
            git clone -b $CI_COMMIT_REF_NAME $CI_REPOSITORY_URL . &&
            curl $CI_API_V4_URL/projects/$CI_PROJECT_ID/jobs/artifacts/$CI_COMMIT_REF_NAME/download?job=$CI_BIN_DEPENDENCIES_JOB -o artifacts.zip &&
            unzip artifacts.zip ;
        cd /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME/scripts/deploy/ &&
            python3 createconfig.py -s $CI_ENVIRONMENT_NAME &&
            /bin/bash install_venv.sh -d -r ../../requirements.txt &&
            python3 prepare_init.d.py &&
            python3 deploy.py -s $CI_ENVIRONMENT_NAME"
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    tty: true
    stdin_open: true
    networks:
      nw_swarm:

networks:
  nw_swarm:
    external: true

Apha ungabona ukuba amacandelo adityaniswe yinethiwekhi enye (nw_swarm) kwaye iyafikeleleka enye kwenye.

Amacandelo enkqubo (esekelwe kwi-redis, i-mysql) ihlukaniswe kwi-pool jikelele yamacandelo amasiko (kwizicwangciso, amacandelo amasiko ahlukaniswe njengeenkonzo). Inqanaba lokusasazwa kweqela lethu lijongeka njengokudlulisa i-CMD kumfanekiso wethu omkhulu omiselweyo kwaye, ngokubanzi, ahluke ngokupheleleyo kulungiselelo oluchazwe kwiCandelo I. Ndiza kugxininisa umahluko:

  • git clone... - sifumana iifayile eziyimfuneko ukwenza ukuthunyelwa (createconfig.py, install_venv.sh, njl.)
  • curl... && unzip... - Khuphela kwaye unzip izinto zokwakha (ezihlanganisiweyo eziluncedo)

Inye kuphela ingxaki engekachazwa: amacandelo anojongano lwewebhu akafikeleli kwibhrawuza zabaphuhlisi. Sisombulula le ngxaki sisebenzisa iproxy ebuyela umva, ngolu hlobo:

Kwi-.gitlab-ci.yml, emva kokuhambisa i-stack ye-cluster, yongeza umgca wokuhambisa i-balancer (ethi, xa izinikele, ihlaziya uqwalaselo lwayo kuphela (yenza iifayile zoqwalaselo ezintsha ze-nginx ngokwe template: /etc/nginx/conf.d /${CI_COMMIT_REF_NAME}.conf) - bona ikhowudi docker-compose-nginx.yml)

    - docker stack deploy -c docker-compose-nginx.yml ${CI_ENVIRONMENT_NAME} --with-registry-auth

docker-compose-nginx.yml

---
version: '3'

services:
  nginx:
    image: nginx:latest
    environment:
      CI_COMMIT_REF_NAME: ${CI_COMMIT_REF_NAME}
      NGINX_CONFIG: |-
            server {
                listen 8080;
                server_name staging_${CI_COMMIT_REF_NAME}_cluster.dev;

                location / {
                    proxy_pass http://staging_${CI_COMMIT_REF_NAME}_cluster:8080;
                }
            }
            server {
                listen 5555;
                server_name staging_${CI_COMMIT_REF_NAME}_cluster.dev;

                location / {
                    proxy_pass http://staging_${CI_COMMIT_REF_NAME}_cluster:5555;
                }
            }
    volumes:
      - /tmp/staging/nginx:/etc/nginx/conf.d
    command:
      /bin/bash -c "echo -e "$$NGINX_CONFIG" > /etc/nginx/conf.d/${CI_COMMIT_REF_NAME}.conf;
        nginx -g "daemon off;";
        /etc/init.d/nginx reload"
    ports:
      - 8080:8080
      - 5555:5555
      - 3000:3000
      - 443:443
      - 80:80
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

networks:
  nw_swarm:
    external: true

Kwiikhompyuter zomphuhlisi, hlaziya /etc/hosts; seta i-url kwi nginx:

10.50.173.106 staging_BRANCH-1831_cluster.dev

Ke, ukusasazwa kwamaqela eqonga azimeleyo aphunyeziwe kwaye abaphuhlisi ngoku banokuwasungula nangaliphi na inani elaneleyo lokuvavanya imisebenzi yabo.

Izicwangciso zexesha elizayo:

  • Yahlula amacandelo ethu njengeenkonzo
  • Yenza iDockerfile nganye
  • Zibonele ngokuzenzekelayo iindawo ezingalayishwanga kakhulu kwisitaki
  • Cacisa iindawo usebenzisa itemplate yegama (kunokusebenzisa id njengakwinqaku)
  • Yongeza itshekhi ukuba istaki sitshabalalisiwe
  • ...

Umbulelo okhethekileyo inqaku.

umthombo: www.habr.com

Yongeza izimvo