ProHoster > Blog > Pulega > GitLab Shell Runner. Fa'ataua fa'atauva'a o auaunaga fa'ata'ita'i e fa'aaoga ai le Docker Compose
GitLab Shell Runner. Fa'ataua fa'atauva'a o auaunaga fa'ata'ita'i e fa'aaoga ai le Docker Compose
O lenei tusiga o le a fiafia i tagata suʻesuʻe ma atinaʻe, ae e faʻamoemoe tele mo tagata tomai faʻapitoa o loʻo feagai ma le faʻafitauli o le faʻatulagaina o le GitLab CI / CD mo le tuʻufaʻatasia o suʻega i tulaga o le le lava o punaoa tetele ma / poʻo le leai o se pusa. tulaga fa'atutusa. O le a ou taʻu atu ia te oe le faʻatulagaina o le faʻatulagaina o siosiomaga suʻega e faʻaaoga ai le docker compose i luga o le tasi GitLab shell runner ma ina ia a faʻapipiʻiina ni siosiomaga, o le faʻalauiloaina o auaunaga e le faʻalavelave le tasi i le isi.
I laʻu faʻataʻitaʻiga, e masani ona tupu o suʻega tuʻufaʻatasia na "togafitia" i luga o poloketi. Ma e masani lava o le faʻafitauli muamua ma sili ona taua o le paipa CI, lea e suʻeina ai le tuʻufaʻatasia ua atiina ae 'au'aunaga (s) o lo'o fa'atinoina i se si'osi'omaga dev/stage. O lenei mea na mafua ai ni nai faʻafitauli:
Ona o faaletonu i se auaunaga faapitoa i le taimi o suʻega tuʻufaʻatasia, e ono faʻaleagaina le suʻega suʻega i faʻamaumauga gau. Sa i ai mataupu pe a tuʻuina atu se talosaga ma se JSON ua malepe na faʻalavelaveina le tautua, lea na faʻaalia ai le le atoatoa o le tu.
Fa'agesegese ole su'ega fa'ata'ita'iga a'o fa'ateleina fa'amaumauga ole su'ega. Ou te manatu e leai se uiga e faʻamatalaina se faʻataʻitaʻiga i le faʻamamaina / toe faʻafoʻi i tua le database. I laʻu faʻataʻitaʻiga, ou te leʻi feiloaʻi i se poloketi na sologa lelei ai lenei faiga.
Tulaga lamatia o le faʻalavelaveina o le faʻatinoga o le suʻega suʻega pe a suʻeina faʻatulagaga lautele. Mo se faʻataʻitaʻiga, tagata faʻaoga / vaega / faʻaupuga / faiga faʻaoga.
O fa'amaumauga o su'ega mai su'ega fa'a otometi e faigata ai le olaga mo su'ega tusi.
O nisi o le a fai mai o suʻega taʻavale lelei e tatau ona faʻamamāina faʻamaumauga pe a uma i latou. E i ai a'u finauga faasaga i:
Tu'u malosi e matua faigofie ona fa'aoga.
E le mafai ona aveese mea uma mai le faiga e ala ile API. Mo se faʻataʻitaʻiga, o se valaau e tape se mea e leʻi faʻatinoina ona e feteʻenaʻi ma faʻamatalaga pisinisi.
Pe a fatuina se mea e ala i le API, e mafai ona faia se aofaiga tele o metadata, lea e faʻafitauli e tape.
Afai o suʻega e iai faʻalagolago ia i latou lava, o le faagasologa o le faʻamamaina o faʻamaumauga pe a uma suʻega suʻega e liua i le tiga o le ulu.
Faʻaopoopo (ma, i loʻu manatu, e le faʻamaonia) valaʻau i le API.
Ma o le finauga autu: pe a amata ona kilia saʻo faʻamaumauga o suʻega mai le faʻamaumauga. O le mea lea ua liua i se PK/FK circus moni! Matou te faʻalogo mai le au atiaʻe: "Na ou faʻaopoopoina / aveese / toe faʻaigoaina se faʻailoga, aisea na maua ai le 100500 suʻega tuʻufaʻatasia?"
I loʻu manatu, o le fofo sili ona lelei o se siosiomaga malosi.
E to'atele tagata latou te fa'aogaina le docker-compose e fa'ata'ita'i ai se si'osi'omaga o su'ega, ae to'aitiiti tagata latou te fa'aogaina le fa'ailoga-tusi pe a fa'atautaia su'ega tu'ufa'atasi ile CI/CD. Ma o iinei ou te le o manatu i le kubernetes, swarm ma isi faʻasalalauga faʻapipiʻi pusa. E le o kamupani uma e iai. E manaia pe a fai o le docker-compose.yml e lautele.
E tusa lava pe iai a tatou lava tagata tamoʻe QA, e faʻapefea ona tatou mautinoa o auaunaga na faʻalauiloaina e ala ile docker-compose e le faʻalavelave le tasi i le isi?
E fa'afefea ona aoina mai ogalaau o auaunaga fa'ata'ita'i?
E fa'afefea ona fufulu le tagata tamo'e?
E iai laʻu lava tamoʻe GitLab mo aʻu galuega faatino ma sa ou feagai ma nei fesili i le taimi o le atinaʻe Java client mo TestRail. E sili atu le saʻo, pe a faʻatautaia suʻega tuʻufaʻatasia. I lalo o le a matou foia nei mataupu e faʻaaoga ai faʻataʻitaʻiga mai lenei poloketi.
Mo se tagata tamo'e, ou te fautuaina se masini komepiuta Linux ma 4 vCPU, 4 GB RAM, 50 GB HDD.
E tele faʻamatalaga i le faʻatulagaina o le gitlab-runner i luga ole Initaneti, faʻapuupuu:
Ulufale i le masini e ala i le SSH
Afai e itiiti ifo i le 8 GB RAM, ona ou fautuaina lea fai swap 10 GBina ia le sau le fasioti tagata OOM ma fasioti a matou galuega ona o le leai o se RAM. E mafai ona tupu lenei mea pe a sili atu i le 5 galuega e faʻalauiloa i le taimi e tasi. O galuega o le a sili atu lemu, ae tumau.
Fa'ata'ita'iga ma le OOM killer
Afai e te vaʻai i faʻamaumauga o galuega bash: line 82: 26474 Killed, ona na'o le fa'atinoina lea o le tagata tamo'e sudo dmesg | grep 26474
[26474] 1002 26474 1061935 123806 339 0 0 java
Out of memory: Kill process 26474 (java) score 127 or sacrifice child
Killed process 26474 (java) total-vm:4247740kB, anon-rss:495224kB, file-rss:0kB, shmem-rss:0kB
Ma afai o le ata e foliga faʻapenei, ona faʻaopoopo lea o le swap pe faʻaopoopo le RAM.
Ole mea lea ole a fa'atagaina oe e fai galuega tutusa ile tasi tagata tamo'e. Faitau atili iinei.
Afai e iai sau masini e sili atu le malosi, mo se faʻataʻitaʻiga 8 vCPU, 16 GB RAM, ona mafai lea ona faia nei fuainumera ia le itiiti ifo i le 2 taimi sili atu. Ae e fa'alagolago lava i le mea tonu o le a fa'alauiloa i lenei tagata tamo'e ma pe o le a le aofa'i.
O le galuega autu o se universal docker-compose.yml, lea e mafai e le au atinaʻe / suʻega ona faʻaoga uma i le lotoifale ma le CI pipeline.
Muamua, matou te faia igoa faʻapitoa mo le CI. O se tasi o fesuiaiga tulaga ese i GitLab CI o le fesuiaiga CI_JOB_ID. Afai e te faʻamaonia container_name ma le uiga "service-${CI_JOB_ID:-local}", ona i ai lea i le tulaga:
pe afai CI_JOB_ID e le'o fa'amatalaina i fesuiaiga o le siosiomaga,
ona avea ai lea o le igoa tautua service-local
pe afai CI_JOB_ID fa'amalamalamaina i fesuiaiga o le si'osi'omaga (fa'ata'ita'iga 123),
ona avea ai lea o le igoa tautua service-123
Lona lua, matou te faia se feso'ota'iga masani mo auaunaga fa'alauiloa. Ole mea lea e tu'u'ese'ese ai i tatou ile tulaga ole feso'ota'iga pe a fa'atautaia ni si'osi'omaga su'ega se tele.
O le mea moni, o le laasaga muamua lea i le manuia =)
Fa'ata'ita'iga o la'u docker-compose.yml ma fa'amatalaga
version: "3"
# Для корректной работы web (php) и fmt нужно,
# чтобы контейнеры имели общий исполняемый контент.
# В нашем случае, это директория /var/www/testrail
volumes:
static-content:
# Изолируем окружение на сетевом уровне
networks:
default:
external:
name: testrail-network-${CI_JOB_ID:-local}
services:
db:
image: mysql:5.7.22
# Каждый container_name содержит ${CI_JOB_ID:-local}
container_name: "testrail-mysql-${CI_JOB_ID:-local}"
environment:
MYSQL_HOST: db
MYSQL_DATABASE: mydb
MYSQL_ROOT_PASSWORD: 1234
SKIP_GRANT_TABLES: 1
SKIP_NETWORKING: 1
SERVICE_TAGS: dev
SERVICE_NAME: mysql
networks:
- default
migration:
image: registry.gitlab.com/touchbit/image/testrail/migration:latest
container_name: "testrail-migration-${CI_JOB_ID:-local}"
links:
- db
depends_on:
- db
networks:
- default
fpm:
image: registry.gitlab.com/touchbit/image/testrail/fpm:latest
container_name: "testrail-fpm-${CI_JOB_ID:-local}"
volumes:
- static-content:/var/www/testrail
links:
- db
networks:
- default
web:
image: registry.gitlab.com/touchbit/image/testrail/web:latest
container_name: "testrail-web-${CI_JOB_ID:-local}"
# Если переменные TR_HTTP_PORT или TR_HTTPS_PORTS не определены,
# то сервис поднимается на 80 и 443 порту соответственно.
ports:
- ${TR_HTTP_PORT:-80}:80
- ${TR_HTTPS_PORT:-443}:443
volumes:
- static-content:/var/www/testrail
links:
- db
- fpm
networks:
- default
Integration:
stage: test
tags:
- my-shell-runner
before_script:
# Аутентифицируемся в registry
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
# Генерируем псевдоуникальные TR_HTTP_PORT и TR_HTTPS_PORT
- export TR_HTTP_PORT=$(shuf -i10000-60000 -n1)
- export TR_HTTPS_PORT=$(shuf -i10000-60000 -n1)
# создаем директорию с идентификатором задачи
- mkdir ${CI_JOB_ID}
# копируем в созданную директорию наш docker-compose.yml
# чтобы контекст был разный для каждой задачи
- cp .indirect/docker-compose.yml ${CI_JOB_ID}/docker-compose.yml
script:
# поднимаем наше окружение
- make docker-up
# запускаем тесты исполняемым jar (у меня так)
- java -jar itest.jar --http-port ${TR_HTTP_PORT} --https-port ${TR_HTTPS_PORT}
# или в контейнере
- docker run --network=testrail-network-${CI_JOB_ID:-local} --rm itest
after_script:
# собираем логи
- make docker-logs
# останавливаем окружение
- make docker-kill
artifacts:
# сохраняем логи
when: always
paths:
- logs
expire_in: 30 days
O se taunuuga o le fa'atinoina o sea galuega, o le lisi o ogalaau i totonu o mea taua o le a aofia ai le tautua ma le su'ega ogalaau. Lea e faigofie tele i tulaga o mea sese. O suʻega taʻitasi i le tutusa e tusia lana lava ogalaau, ae o le a ou talanoa faʻapitoa e uiga i lenei mea.
$ docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /home/gitlab-runner/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
$ export TR_HTTP_PORT=$(shuf -i10000-60000 -n1)
$ export TR_HTTPS_PORT=$(shuf -i10000-60000 -n1)
$ mkdir ${CI_JOB_ID}
$ cp .indirect/docker-compose.yml ${CI_JOB_ID}/docker-compose.yml
$ make docker-up
docker-compose -f ${CI_JOB_ID:-.indirect}/docker-compose.yml kill
docker network rm testrail-network-${CI_JOB_ID:-local} || true
Error: No such network: testrail-network-204645172
docker network create testrail-network-${CI_JOB_ID:-local}
0a59552b4464b8ab484de6ae5054f3d5752902910bacb0a7b5eca698766d0331
docker-compose -f ${CI_JOB_ID:-.indirect}/docker-compose.yml pull
Pulling web ... done
Pulling fpm ... done
Pulling migration ... done
Pulling db ... done
docker-compose -f ${CI_JOB_ID:-.indirect}/docker-compose.yml up --force-recreate --renew-anon-volumes -d
Creating volume "204645172_static-content" with default driver
Creating testrail-mysql-204645172 ...
Creating testrail-mysql-204645172 ... done
Creating testrail-migration-204645172 ... done
Creating testrail-fpm-204645172 ... done
Creating testrail-web-204645172 ... done
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c6b76f9135ed registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" 13 seconds ago Up 1 second 0.0.0.0:51148->80/tcp, 0.0.0.0:25426->443/tcp testrail-web-204645172
01d303262d8e registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" 16 seconds ago Up 13 seconds 9000/tcp testrail-fpm-204645172
2cdab1edbf6a registry.gitlab.com/touchbit/image/testrail/migration:latest "docker-entrypoint.s…" 16 seconds ago Up 13 seconds 3306/tcp, 33060/tcp testrail-migration-204645172
826aaf7c0a29 mysql:5.7.22 "docker-entrypoint.s…" 18 seconds ago Up 16 seconds 3306/tcp testrail-mysql-204645172
6dbb3fae0322 registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" 36 seconds ago Up 22 seconds 0.0.0.0:44202->80/tcp, 0.0.0.0:20151->443/tcp testrail-web-204645084
3540f8d448ce registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" 38 seconds ago Up 35 seconds 9000/tcp testrail-fpm-204645084
70fea72aa10d mysql:5.7.22 "docker-entrypoint.s…" 40 seconds ago Up 37 seconds 3306/tcp testrail-mysql-204645084
d8aa24b2892d registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" About a minute ago Up 53 seconds 0.0.0.0:31103->80/tcp, 0.0.0.0:43872->443/tcp testrail-web-204644881
6d4ccd910fad registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" About a minute ago Up About a minute 9000/tcp testrail-fpm-204644881
685d8023a3ec mysql:5.7.22 "docker-entrypoint.s…" About a minute ago Up About a minute 3306/tcp testrail-mysql-204644881
1cdfc692003a registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" About a minute ago Up About a minute 0.0.0.0:44752->80/tcp, 0.0.0.0:23540->443/tcp testrail-web-204644793
6f26dfb2683e registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" About a minute ago Up About a minute 9000/tcp testrail-fpm-204644793
029e16b26201 mysql:5.7.22 "docker-entrypoint.s…" About a minute ago Up About a minute 3306/tcp testrail-mysql-204644793
c10443222ac6 registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" 5 hours ago Up 5 hours 0.0.0.0:57123->80/tcp, 0.0.0.0:31657->443/tcp testrail-web-204567103
04339229397e registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" 5 hours ago Up 5 hours 9000/tcp testrail-fpm-204567103
6ae0accab28d mysql:5.7.22 "docker-entrypoint.s…" 5 hours ago Up 5 hours 3306/tcp testrail-mysql-204567103
b66b60d79e43 registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" 5 hours ago Up 5 hours 0.0.0.0:56321->80/tcp, 0.0.0.0:58749->443/tcp testrail-web-204553690
033b1f46afa9 registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" 5 hours ago Up 5 hours 9000/tcp testrail-fpm-204553690
a8879c5ef941 mysql:5.7.22 "docker-entrypoint.s…" 5 hours ago Up 5 hours 3306/tcp testrail-mysql-204553690
069954ba6010 registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" 5 hours ago Up 5 hours 0.0.0.0:32869->80/tcp, 0.0.0.0:16066->443/tcp testrail-web-204553539
ed6b17d911a5 registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" 5 hours ago Up 5 hours 9000/tcp testrail-fpm-204553539
1a1eed057ea0 mysql:5.7.22 "docker-entrypoint.s…" 5 hours ago Up 5 hours 3306/tcp testrail-mysql-204553539
Ua mae'a lelei galuega uma
O galuega fa'atino e iai fa'amaumauga o auaunaga ma su'ega
E foliga mai e matagofie mea uma, ae o loʻo i ai se faʻaaliga. E mafai ona fa'aleaogaina fa'amalosi le laina paipa a'o fa'agasolo su'ega tu'ufa'atasi, ma o lea tulaga e le taofia ai atigipusa. Mai lea taimi i lea taimi e te manaʻomia le fufuluina o le tagata tamoʻe. Ae paga lea, o le galuega mo le faʻaleleia atili o le GitLab CE o loʻo i ai pea i le tulaga tatala
Ae ua matou faʻaopoopoina le faʻalauiloaina o se galuega e tusa ai ma se faʻatulagaga, ma e leai se tasi e faʻasaina i matou e faʻatautaia ma le lima.
Alu i la matou poloketi -> CI / CD -> Faʻasologa ma faʻatino le galuega Clean runner
Aofai:
E tasi le matou tamo'e atigi.
E leai ni feteenaiga i le va o galuega ma le siosiomaga.
Matou te fa'atautaia galuega fa'atasi ma su'ega tu'ufa'atasi.
E mafai ona e fa'atautaia su'ega tu'ufa'atasia pe i totonu o le lotoifale po'o totonu o se koneteina.
O lo'o aoina mai fa'amaumauga o auaunaga ma su'ega ma fa'apipi'i ile galuega ole paipa.
E mafai ona fa'amama le tagata tamo'e mai ata tuai o Docker.
Taimi fa'atulagaina e ~2 itula.
Na'o le mea moni lava. O le a ou fiafia e maua ni manatu faaalia.