ProHoster > Blog > Tsamaiso > Kenya lits'ebetso u sebelisa Docker Swarm

Kenya lits'ebetso u sebelisa Docker Swarm

Sistimi ea likhothaletso ea litaba tsa video tse fumanehang marang-rang eo re sebetsang ho eona ke nts'etsopele ea khoebo e koetsoeng 'me ka botekgeniki ke sehlopha sa likarolo tse ngata tsa likarolo tsa botho le tse bulehileng. Morero oa ho ngola sengoloa sena ke ho hlalosa ts'ebetsong ea sistimi e kopanyang li-docker swarm bakeng sa sebaka sa marang-rang ntle le ho senya ts'ebetso e hlophisitsoeng ea lits'ebetso tsa rona ka nako e lekanyelitsoeng. Tlaleho e hlahisitsoeng tlhokomelong ea hau e arotsoe likarolo tse peli. Karolo ea pele e hlalosa CI / CD pele u sebelisa sehlopha sa docker, 'me ea bobeli e hlalosa ts'ebetso ea ts'ebetsong ea eona. Ba sa thahaselleng ho bala karolo ea pele ba ka fetela ea bobeli ka mokhoa o sireletsehileng.

Karolo ea XNUMX

Ka nako e 'ngoe, ho ne ho e-na le tlhokahalo ea ho theha ts'ebetso ea CI / CD kapele kamoo ho ka khonehang. E 'ngoe ea maemo e ne e le ho se sebelise Docker bakeng sa ho tsamaisoa likarolo li etsoa ka mabaka a 'maloa:

  • bakeng sa ts'ebetso e tšepahalang le e tsitsitseng ea likarolo ho Tlhahiso (ke hore, ha e le hantle, tlhokahalo ea ho se sebelise virtualization)
  • bahlahisi ba etelletseng pele ba ne ba sa batle ho sebetsa le Docker (e makatsang, empa ho ne ho le joalo)
  • ho ea ka maikutlo a maikutlo a tsamaiso ea R&D

Mehaho ea motheo, mekotla le litlhoko tsa pele tse hakanyetsoang bakeng sa MVP e ne e le tse latelang:

  • Li-server tse 4 tsa Intel® X5650 tse nang le Debian (mochini o le mong o matla haholoanyane bakeng sa nts'etsopele)
  • Nts'etsopele ea likarolo tsa hau tsa moetlo e etsoa ho C ++, Python3
  • Lisebelisoa tsa mantlha tsa mokha oa boraro tse sebelisitsoeng: Kafka, Clickhouse, Airflow, Redis, Grafana, Postgresql, Mysql, ...
  • Liphaephe tsa ho aha le ho etsa liteko ka thoko bakeng sa ho lokisa le ho lokolla

E 'ngoe ea lipotso tsa pele tse lokelang ho rarolloa sethaleng sa pele ke hore na likarolo tsa moetlo li tla sebelisoa joang tikolohong efe kapa efe (CI / CD).

Re nkile qeto ea ho kenya likarolo tsa mokha oa boraro ka mokhoa o hlophisehileng le ho li nchafatsa ka mokhoa o hlophisitsoeng. Lisebelisoa tsa tloaelo tse ntlafalitsoeng ho C ++ kapa Python li ka sebelisoa ka mekhoa e mengata. Har'a bona, mohlala: ho theha liphutheloana tsa tsamaiso, ho li romela sebakeng sa polokelo ea litšoantšo tse bokelitsoeng le ho kenngoa ha tsona ka morao ho li-server. Ka lebaka le neng le se le ntse le sa tsejoe, ho ile ha khethoa mokhoa o mong, e leng: ho sebelisa CI, lifaele tsa ts'ebetsong li hlophisitsoe, ho thehoa tikoloho ea morero, py modules ho tloha litlhoko.txt e kenngoa, 'me lintho tsena tsohle tsa khale li romelloa hammoho le configs, scripts le. tikoloho ea ts'ebeliso e tsamaeang le li-server. Ka mor'a moo, lits'ebetso li hlahisoa ho tsoa ho mosebelisi ea se nang litokelo tsa motsamaisi.

Gitlab-CI e khethiloe e le sistimi ea CI/CD. Phaello e hlahisitsoeng e ne e shebahala tjena:

Kenya lits'ebetso u sebelisa Docker Swarm
Ka sebopeho, gitlab-ci.yml e ne e shebahala tjena:

---
variables:
  # минимальная версия ЦПУ на серверах, где разворачивается кластер
  CMAKE_CPUTYPE: "westmere"

  DEBIAN: "MYREGISTRY:5000/debian:latest"

before_script:
  - eval $(ssh-agent -s)
  - ssh-add <(echo "$SSH_PRIVATE_KEY")
  - mkdir -p ~/.ssh && echo -e "Host *ntStrictHostKeyChecking nonn" > ~/.ssh/config

stages:
  - build
  - testing
  - deploy

debug.debian:
  stage: build
  image: $DEBIAN
  script:
    - cd builds/release && ./build.sh
    paths:
      - bin/
      - builds/release/bin/
    when: always
release.debian:
  stage: build
  image: $DEBIAN
  script:
    - cd builds/release && ./build.sh
    paths:
      - bin/
      - builds/release/bin/
    when: always

## testing stage
tests.codestyle:
  stage: testing
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - /bin/bash run_tests.sh -t codestyle -b "${CI_COMMIT_REF_NAME}_codestyle"
tests.debug.debian:
  stage: testing
  image: $DEBIAN
  dependencies:
    - debug.debian
  script:
    - /bin/bash run_tests.sh -e codestyle/test_pylint.py -b "${CI_COMMIT_REF_NAME}_debian_debug"
  artifacts:
    paths:
      - run_tests/username/
    when: always
    expire_in: 1 week
tests.release.debian:
  stage: testing
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - /bin/bash run_tests.sh -e codestyle/test_pylint.py -b "${CI_COMMIT_REF_NAME}_debian_release"
  artifacts:
    paths:
      - run_tests/username/
    when: always
    expire_in: 1 week

## staging stage
deploy_staging:
  stage: deploy
  environment: staging
  image: $DEBIAN
  dependencies:
    - release.debian
  script:
    - cd scripts/deploy/ &&
        python3 createconfig.py -s $CI_ENVIRONMENT_NAME &&
        /bin/bash install_venv.sh -d -r ../../requirements.txt &&
        python3 prepare_init.d.py &&
        python3 deploy.py -s $CI_ENVIRONMENT_NAME
  when: manual

Ke habohlokoa ho hlokomela hore kopano le tlhahlobo li etsoa ka setšoantšo sa eona, moo liphutheloana tsohle tse hlokahalang tsa tsamaiso li se li kentsoe 'me litlhophiso tse ling li entsoe.

Le hoja e 'ngoe le e' ngoe ea mangolo ana a mesebetsi e thahasellisa ka tsela ea eona, empa ha e le hantle nke ke ka bua ka eona Tlhaloso ea e mong le e mong oa bona e tla nka nako e ngata 'me sena hase morero oa sehlooho. Ke tla lebisa tlhokomelo ea hau ntlheng ea hore sethala sa phepelo se na le tatellano ea mangolo a ho letsetsa:

  1. createconfig.py - e theha faele ea li-setting.ini e nang le litlhophiso tsa likarolo libakeng tse fapaneng bakeng sa ho romelloa ka mor'a moo (Preproduction, Production, Testing, ...)
  2. kenya_venv.sh - e theha tikoloho e fumanehang bakeng sa likarolo tsa py bukeng e itseng ebe e e kopitsa ho li-server tse hole
  3. lokisetsa_init.d.py — e lokisa mengolo bakeng sa likarolo tsa ho emisa ho qala ho ipapisitse le thempleite
  4. deploy.py - e kenya le ho qala likarolo tse ncha hape

Nako e ile ea feta. Sethala sa sethala se ile sa nkeloa sebaka ke tlhahiso ea pele le tlhahiso. Ts'ehetso ea sehlahisoa e kenyellelitsoe kabong e 'ngoe hape (CentOS). E kenyellelitse li-server tse 5 tse matla haholoanyane le tse ling tse leshome le metso e 'meli. 'Me ho ile ha e-ba thata le ho feta hore bahlahisi le bahlahlobi ba leke mesebetsi ea bona tikolohong e haufi kapa e haufi le boemo ba ho sebetsa. Ka nako ena, ho ile ha hlaka hore ho ne ho ke ke ha khoneha ho etsa ntle le eena ...

Karolo ea II

Kenya lits'ebetso u sebelisa Docker Swarm

Kahoo, sehlopha sa rona ke sistimi e makatsang ea likarolo tse 'maloa tse arohaneng tse sa hlalosoang ke Dockerfiles. U ka e hlophisa feela bakeng sa ho romelloa tikolohong e itseng ka kakaretso. Mosebetsi oa rona ke ho beha sehlopha sebakeng sa sethala ho se leka pele ho tlhahlobo ea pele ho tokollo.

Ka khopolo, ho ka ba le lihlopha tse 'maloa tse sebetsang ka nako e le' ngoe: tse ngata kamoo ho nang le mesebetsi e boemong bo phethiloeng kapa e haufi le ho phethoa. Matla a li-server tseo re nang le tsona a re lumella ho tsamaisa lihlopha tse 'maloa ho seva ka seng. Sehlopha se seng le se seng sa sethala se tlameha ho qheleloa ka thoko (ha hoa lokela ho ba le tšubuhlellano ea likou, li-directory, joalo-joalo).

Mohloli oa rona oa bohlokoahali ke nako ea rona, 'me ha rea ​​ka ra ba le e ngata ea eona.

Bakeng sa qalo e potlakileng, re khethile Docker Swarm ka lebaka la bonolo le boqapi ba eona bo bonolo. Ntho ea pele eo re e entseng ke ho theha mookameli le li-node tse 'maloa ho li-server tse hole:

$ docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
kilqc94pi2upzvabttikrfr5d     nop-test-1     Ready               Active                                  19.03.2
jilwe56pl2zvabupryuosdj78     nop-test-2     Ready               Active                                  19.03.2
j5a4yz1kr2xke6b1ohoqlnbq5 *   nop-test-3     Ready               Active              Leader              19.03.2

E latelang, theha marang-rang:


$ docker network create --driver overlay --subnet 10.10.10.0/24 nw_swarm

Ka mor'a moo, re ile ra kopanya li-node tsa Gitlab-CI le Swarm mabapi le tsamaiso e hole ea li-node tse tsoang ho CI: ho kenya litifikeiti, ho beha mefuta-futa ea sekhukhu, le ho theha tšebeletso ea Docker ho seva sa tsamaiso. Eona ena sehlooho re bolokile nako e ngata.

Ka mor'a moo, re kentse mesebetsi bakeng sa ho theha le ho senya stack ho .gitlab-ci .yml.

Mesebetsi e meng e mmalwa e kentswe ho .gitlab-ci .yml

## staging stage
deploy_staging:
  stage: testing
  before_script:
    - echo "override global 'before_script'"
  image: "REGISTRY:5000/docker:latest"
  environment: staging
  dependencies: []
  variables:
    DOCKER_CERT_PATH: "/certs"
    DOCKER_HOST: tcp://10.50.173.107:2376
    DOCKER_TLS_VERIFY: 1
    CI_BIN_DEPENDENCIES_JOB: "release.centos.7"
  script:
    - mkdir -p $DOCKER_CERT_PATH
    - echo "$TLSCACERT" > $DOCKER_CERT_PATH/ca.pem
    - echo "$TLSCERT" > $DOCKER_CERT_PATH/cert.pem
    - echo "$TLSKEY" > $DOCKER_CERT_PATH/key.pem
    - docker stack deploy -c docker-compose.yml ${CI_ENVIRONMENT_NAME}_${CI_COMMIT_REF_NAME} --with-registry-auth
    - rm -rf $DOCKER_CERT_PATH
  when: manual

## stop staging stage
stop_staging:
  stage: testing
  before_script:
    - echo "override global 'before_script'"
  image: "REGISTRY:5000/docker:latest"
  environment: staging
  dependencies: []
  variables:
    DOCKER_CERT_PATH: "/certs"
    DOCKER_HOST: tcp://10.50.173.107:2376
    DOCKER_TLS_VERIFY: 1
  script:
    - mkdir -p $DOCKER_CERT_PATH
    - echo "$TLSCACERT" > $DOCKER_CERT_PATH/ca.pem
    - echo "$TLSCERT" > $DOCKER_CERT_PATH/cert.pem
    - echo "$TLSKEY" > $DOCKER_CERT_PATH/key.pem
    - docker stack rm ${CI_ENVIRONMENT_NAME}_${CI_COMMIT_REF_NAME}
    # TODO: need check that stopped
  when: manual

Ho tsoa ho snippet ea khoutu e kaholimo, u ka bona hore likonopo tse peli (deploy_staging, stop_staging) li kentsoe ho Pipelines, tse hlokang ketso ea letsoho.

Kenya lits'ebetso u sebelisa Docker Swarm
Lebitso la stack le tsamaisana le lebitso la lekala mme ho ikhetha hona ho lokela ho lekana. Litšebeletso tse ka har'a stack li fumana liaterese tse ikhethang tsa ip, le likou, li-directory, joalo-joalo. e tla aroloa, empa e ts'oanang ho tloha ho stack ho ea ho stack (hobane faele ea tlhophiso e tšoana bakeng sa mekotla eohle) - seo re neng re se batla. Re tsamaisa stack (sehlopha) re sebelisa docker-compose.yml, e hlalosang sehlopha sa rona.

docker-compose.yml

---
version: '3'

services:
  userprop:
    image: redis:alpine
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:
  celery_bcd:
    image: redis:alpine
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  schedulerdb:
    image: mariadb:latest
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
      MYSQL_DATABASE: schedulerdb
      MYSQL_USER: ****
      MYSQL_PASSWORD: ****
    command: ['--character-set-server=utf8mb4', '--collation-server=utf8mb4_unicode_ci', '--explicit_defaults_for_timestamp=1']
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  celerydb:
    image: mariadb:latest
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
      MYSQL_DATABASE: celerydb
      MYSQL_USER: ****
      MYSQL_PASSWORD: ****
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

  cluster:
    image: $CENTOS7
    environment:
      - CENTOS
      - CI_ENVIRONMENT_NAME
      - CI_API_V4_URL
      - CI_REPOSITORY_URL
      - CI_PROJECT_ID
      - CI_PROJECT_URL
      - CI_PROJECT_PATH
      - CI_PROJECT_NAME
      - CI_COMMIT_REF_NAME
      - CI_BIN_DEPENDENCIES_JOB
    command: >
      sudo -u myusername -H /bin/bash -c ". /etc/profile &&
        mkdir -p /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME &&
        cd /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME &&
            git clone -b $CI_COMMIT_REF_NAME $CI_REPOSITORY_URL . &&
            curl $CI_API_V4_URL/projects/$CI_PROJECT_ID/jobs/artifacts/$CI_COMMIT_REF_NAME/download?job=$CI_BIN_DEPENDENCIES_JOB -o artifacts.zip &&
            unzip artifacts.zip ;
        cd /storage1/$CI_COMMIT_REF_NAME/$CI_PROJECT_NAME/scripts/deploy/ &&
            python3 createconfig.py -s $CI_ENVIRONMENT_NAME &&
            /bin/bash install_venv.sh -d -r ../../requirements.txt &&
            python3 prepare_init.d.py &&
            python3 deploy.py -s $CI_ENVIRONMENT_NAME"
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    tty: true
    stdin_open: true
    networks:
      nw_swarm:

networks:
  nw_swarm:
    external: true

Mona u ka bona hore likarolo li kopantsoe ke marang-rang a le mong (nw_swarm) 'me li fumaneha ho tse ling.

Likarolo tsa tsamaiso (tse thehiloeng ho redis, mysql) li arohane le letamo le akaretsang la likarolo tse tloaelehileng (ka meralo, likarolo tsa moetlo li boetse li arotsoe e le litšebeletso). Mokhahlelo oa phepelo ea sehlopha sa rona o shebahala joalo ka ho fetisetsa CMD ho setšoantšo sa rona se le seng se seholo se hlophisitsoeng, 'me, ka kakaretso, ha se fapane hole le phepelo e hlalositsoeng Karolong ea I. Ke tla hatisa liphapang:

  • git clone... - fumana lifaele tse hlokahalang ho li sebelisa (createconfig.py, install_venv.sh, joalo-joalo)
  • curl... && bula... - Khoasolla le ho notlolla lintho tsa khale tsa khale (lisebelisoa tse hlophisitsoeng)

Ho na le bothata bo le bong feela bo sa hlalosoang: likarolo tse nang le sebopeho sa webo ha li fumanehe ho tsoa ho libatli tsa bahlahisi. Re rarolla bothata bona ka ho sebelisa proxy ea reverse, ka hona:

Ho .gitlab-ci.yml, ka mor'a hore re sebelise stack ea lihlopha, re eketsa moeli oa ho tsamaisa balancer (eo, ha e etsoa, ​​e ntlafatsang tlhophiso ea eona feela (e etsa lifaele tse ncha tsa tlhophiso ea nginx ho latela template: /etc/nginx/conf. d/${CI_COMMIT_REF_NAME}.conf) - bona khoutu docker-compose-nginx.yml)

    - docker stack deploy -c docker-compose-nginx.yml ${CI_ENVIRONMENT_NAME} --with-registry-auth

docker-compose-nginx.yml

---
version: '3'

services:
  nginx:
    image: nginx:latest
    environment:
      CI_COMMIT_REF_NAME: ${CI_COMMIT_REF_NAME}
      NGINX_CONFIG: |-
            server {
                listen 8080;
                server_name staging_${CI_COMMIT_REF_NAME}_cluster.dev;

                location / {
                    proxy_pass http://staging_${CI_COMMIT_REF_NAME}_cluster:8080;
                }
            }
            server {
                listen 5555;
                server_name staging_${CI_COMMIT_REF_NAME}_cluster.dev;

                location / {
                    proxy_pass http://staging_${CI_COMMIT_REF_NAME}_cluster:5555;
                }
            }
    volumes:
      - /tmp/staging/nginx:/etc/nginx/conf.d
    command:
      /bin/bash -c "echo -e "$$NGINX_CONFIG" > /etc/nginx/conf.d/${CI_COMMIT_REF_NAME}.conf;
        nginx -g "daemon off;";
        /etc/init.d/nginx reload"
    ports:
      - 8080:8080
      - 5555:5555
      - 3000:3000
      - 443:443
      - 80:80
    deploy:
      replicas: 1
      placement:
        constraints: [node.id == kilqc94pi2upzvabttikrfr5d]
      restart_policy:
        condition: none
    networks:
      nw_swarm:

networks:
  nw_swarm:
    external: true

Ho likhomphutha tsa nts'etsopele, ntlafatsa /etc/hosts; seta url ho nginx:

10.50.173.106 staging_BRANCH-1831_cluster.dev

Kahoo, ho romelloa ha lihlopha tse ikhethileng ho se ho kentsoe tšebetsong 'me bahlahisi ba ka khona ho li tsamaisa ka palo efe kapa efe e lekaneng ho lekola mesebetsi ea bona.

Merero ea kamoso:

  • Arola likarolo tsa rona e le litšebeletso
  • Theha Dockerfile bakeng sa e 'ngoe le e' ngoe
  • Fumana ka bo eona li-node tse sa keneng ka har'a stack
  • Hlalosa li-node u sebelisa template ea mabitso (ho fapana le ho sebelisa id joalo ka sengoloa)
  • Kenya cheke hore stack e senyehile
  • ...

Liteboho tse khethehileng bakeng sa sengoloa.

Source: www.habr.com

Eketsa ka tlhaloso