ProHoster > Блог > Isakoso > GitLab Shell Runner. Ifilọlẹ idije ti awọn iṣẹ idanwo ni lilo Docker Compose
GitLab Shell Runner. Ifilọlẹ idije ti awọn iṣẹ idanwo ni lilo Docker Compose
Nkan yii yoo jẹ anfani si awọn oludanwo ati awọn olupilẹṣẹ, ṣugbọn o pinnu ni pataki fun awọn alamọja adaṣe ti o dojuko iṣoro ti iṣeto GitLab CI/CD fun idanwo isọpọ ni awọn ipo ti awọn orisun amayederun ti ko to ati/tabi isansa ti orchestration eiyan Syeed. Emi yoo sọ fun ọ bi o ṣe le ṣeto imuṣiṣẹ ti awọn agbegbe idanwo nipa lilo kikọ docker lori olusare ikarahun GitLab kan ṣoṣo ati pe nigba gbigbe awọn agbegbe lọpọlọpọ, awọn iṣẹ ifilọlẹ ko dabaru pẹlu ara wọn.
Ninu iṣe mi, o ma n ṣẹlẹ nigbagbogbo pe idanwo iṣọpọ ni a “ṣe itọju” lori awọn iṣẹ akanṣe. Ati nigbagbogbo iṣoro akọkọ ati pataki julọ ni opo gigun ti epo CI, ninu eyiti idanwo iṣọpọ ni idagbasoke iṣẹ (awọn) ni a ṣe ni agbegbe dev/ipele. Eyi fa awọn iṣoro pupọ:
Nitori awọn abawọn ninu iṣẹ kan pato lakoko idanwo isọpọ, Circuit idanwo le bajẹ nipasẹ data fifọ. Awọn ọran wa nigba fifiranṣẹ ibeere kan pẹlu ọna kika JSON ti o bajẹ ti kọlu iṣẹ naa, eyiti o jẹ ki iduro naa ko ṣiṣẹ patapata.
Ilọkuro ti Circuit idanwo bi data idanwo n pọ si. Mo ro pe ko ni oye lati ṣe apejuwe apẹẹrẹ pẹlu mimọ / yiyi data data pada. Ninu iṣe mi, Emi ko pade iṣẹ akanṣe kan nibiti ilana yii ti lọ laisiyonu.
Ewu ti idalọwọduro iṣẹ ṣiṣe ti Circuit idanwo nigba idanwo awọn eto eto gbogbogbo. Fun apẹẹrẹ, olumulo/ẹgbẹ/ọrọ igbaniwọle/eto imulo.
Awọn data idanwo lati awọn idanwo adaṣe jẹ ki igbesi aye nira fun awọn oludanwo afọwọṣe.
Diẹ ninu awọn yoo sọ pe awọn adaṣe ti o dara yẹ ki o sọ di mimọ data lẹhin ara wọn. Mo ni awọn ariyanjiyan lodi si:
Awọn iduro ti o ni agbara jẹ rọrun pupọ lati lo.
Kii ṣe gbogbo nkan ni a le yọ kuro ninu eto nipasẹ API. Fun apẹẹrẹ, ipe kan lati pa ohun kan jẹ ko ṣe imuse nitori pe o tako ọgbọn iṣowo.
Nigbati o ba ṣẹda ohun kan nipasẹ API, iye nla ti metadata le ṣẹda, eyiti o jẹ iṣoro lati paarẹ.
Ti awọn idanwo ba ni awọn igbẹkẹle laarin ara wọn, lẹhinna ilana ti mimọ data lẹhin ṣiṣe awọn idanwo yipada sinu orififo.
Awọn ipe ni afikun (ati, ni ero mi, kii ṣe idalare) si API.
Ati ariyanjiyan akọkọ: nigbati data idanwo bẹrẹ lati sọ di mimọ taara lati ibi ipamọ data. Eyi n yi pada si Sakosi PK/FK gidi kan! A gbọ lati ọdọ awọn olupilẹṣẹ: “Mo ṣẹṣẹ ṣafikun/yọkuro/ tun lorukọ ami kan, kilode ti awọn idanwo isọpọ 100500 ṣe mu?”
Ni ero mi, ojutu ti o dara julọ jẹ agbegbe ti o ni agbara.
Ọpọlọpọ eniyan lo docker-compose lati ṣiṣẹ agbegbe idanwo kan, ṣugbọn diẹ eniyan lo docker-compose nigbati wọn nṣe idanwo isọpọ ni CI/CD. Ati pe nibi Emi ko ṣe akiyesi awọn kubernetes, swarm ati awọn iru ẹrọ orchestration miiran. Ko gbogbo ile-iṣẹ ni wọn. Yoo dara ti docker-compose.yml jẹ gbogbo agbaye.
Paapa ti a ba ni olusare QA tiwa, bawo ni a ṣe le rii daju pe awọn iṣẹ ti a ṣe ifilọlẹ nipasẹ docker-compose ko dabaru pẹlu ara wa?
Bii o ṣe le gba awọn akọọlẹ ti awọn iṣẹ idanwo?
Bawo ni lati nu olusare?
Mo ni asare GitLab ti ara mi fun awọn iṣẹ akanṣe mi ati pe Mo pade awọn ibeere wọnyi lakoko idagbasoke Onibara Java fun TestRail. Ni deede diẹ sii, nigbati o nṣiṣẹ awọn idanwo isọpọ. Ni isalẹ a yoo yanju awọn oran wọnyi nipa lilo awọn apẹẹrẹ lati inu iṣẹ yii.
Fun olusare kan, Mo ṣeduro ẹrọ foju Linux kan pẹlu 4 vCPU, 4 GB Ramu, 50 GB HDD.
Alaye pupọ wa lori iṣeto gitlab-runner lori Intanẹẹti, nitorinaa ni ṣoki:
Wọle si ẹrọ nipasẹ SSH
Ti o ba ni kere ju 8 GB Ramu, lẹhinna Mo ṣeduro ṣe swap 10 GBki apaniyan OOM ko ba wa pa awọn iṣẹ wa nitori aini Ramu. Eyi le ṣẹlẹ nigbati diẹ sii ju awọn iṣẹ-ṣiṣe 5 ti ṣe ifilọlẹ ni nigbakannaa. Awọn iṣẹ-ṣiṣe yoo ni ilọsiwaju diẹ sii laiyara, ṣugbọn ni imurasilẹ.
Apeere pẹlu OOM apaniyan
Ti o ba ri ninu awọn akọọlẹ iṣẹ-ṣiṣe bash: line 82: 26474 Killed, lẹhinna kan ṣiṣẹ lori olusare sudo dmesg | grep 26474
[26474] 1002 26474 1061935 123806 339 0 0 java
Out of memory: Kill process 26474 (java) score 127 or sacrifice child
Killed process 26474 (java) total-vm:4247740kB, anon-rss:495224kB, file-rss:0kB, shmem-rss:0kB
Ati pe ti aworan ba dabi nkan bi eyi, lẹhinna boya ṣafikun swap tabi ṣafikun Ramu.
Eyi yoo gba ọ laaye lati ṣiṣe awọn iṣẹ-ṣiṣe ti o jọra lori olusare kan. Ka siwaju nibi.
Ti o ba ni ẹrọ ti o lagbara diẹ sii, fun apẹẹrẹ 8 vCPU, 16 GB Ramu, lẹhinna awọn nọmba wọnyi le ṣe o kere ju awọn akoko 2 tobi. Ṣugbọn gbogbo rẹ da lori kini gangan yoo ṣe ifilọlẹ lori olusare yii ati ni iye wo.
Iṣẹ akọkọ jẹ docker-compose.yml ti gbogbo agbaye, eyiti awọn olupilẹṣẹ / awọn idanwo le lo mejeeji ni agbegbe ati ni opo gigun ti epo CI.
Ni akọkọ, a ṣe awọn orukọ iṣẹ alailẹgbẹ fun CI. Ọkan ninu awọn oniyipada alailẹgbẹ ni GitLab CI jẹ oniyipada CI_JOB_ID. Ti o ba pato container_name pẹlu itumo "service-${CI_JOB_ID:-local}", lẹhinna ninu ọran naa:
ti o ba ti CI_JOB_ID ko ṣe alaye ni awọn oniyipada ayika,
lẹhinna orukọ iṣẹ yoo jẹ service-local
ti o ba ti CI_JOB_ID asọye ni awọn oniyipada ayika (fun apẹẹrẹ 123),
lẹhinna orukọ iṣẹ yoo jẹ service-123
Ni ẹẹkeji, a ṣẹda nẹtiwọọki ti o wọpọ fun awọn iṣẹ ifilọlẹ. Eyi fun wa ni ipinya ipele-nẹtiwọọki nigba ṣiṣe awọn agbegbe idanwo pupọ.
version: "3"
# Для корректной работы web (php) и fmt нужно,
# чтобы контейнеры имели общий исполняемый контент.
# В нашем случае, это директория /var/www/testrail
volumes:
static-content:
# Изолируем окружение на сетевом уровне
networks:
default:
external:
name: testrail-network-${CI_JOB_ID:-local}
services:
db:
image: mysql:5.7.22
# Каждый container_name содержит ${CI_JOB_ID:-local}
container_name: "testrail-mysql-${CI_JOB_ID:-local}"
environment:
MYSQL_HOST: db
MYSQL_DATABASE: mydb
MYSQL_ROOT_PASSWORD: 1234
SKIP_GRANT_TABLES: 1
SKIP_NETWORKING: 1
SERVICE_TAGS: dev
SERVICE_NAME: mysql
networks:
- default
migration:
image: registry.gitlab.com/touchbit/image/testrail/migration:latest
container_name: "testrail-migration-${CI_JOB_ID:-local}"
links:
- db
depends_on:
- db
networks:
- default
fpm:
image: registry.gitlab.com/touchbit/image/testrail/fpm:latest
container_name: "testrail-fpm-${CI_JOB_ID:-local}"
volumes:
- static-content:/var/www/testrail
links:
- db
networks:
- default
web:
image: registry.gitlab.com/touchbit/image/testrail/web:latest
container_name: "testrail-web-${CI_JOB_ID:-local}"
# Если переменные TR_HTTP_PORT или TR_HTTPS_PORTS не определены,
# то сервис поднимается на 80 и 443 порту соответственно.
ports:
- ${TR_HTTP_PORT:-80}:80
- ${TR_HTTPS_PORT:-443}:443
volumes:
- static-content:/var/www/testrail
links:
- db
- fpm
networks:
- default
Integration:
stage: test
tags:
- my-shell-runner
before_script:
# Аутентифицируемся в registry
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
# Генерируем псевдоуникальные TR_HTTP_PORT и TR_HTTPS_PORT
- export TR_HTTP_PORT=$(shuf -i10000-60000 -n1)
- export TR_HTTPS_PORT=$(shuf -i10000-60000 -n1)
# создаем директорию с идентификатором задачи
- mkdir ${CI_JOB_ID}
# копируем в созданную директорию наш docker-compose.yml
# чтобы контекст был разный для каждой задачи
- cp .indirect/docker-compose.yml ${CI_JOB_ID}/docker-compose.yml
script:
# поднимаем наше окружение
- make docker-up
# запускаем тесты исполняемым jar (у меня так)
- java -jar itest.jar --http-port ${TR_HTTP_PORT} --https-port ${TR_HTTPS_PORT}
# или в контейнере
- docker run --network=testrail-network-${CI_JOB_ID:-local} --rm itest
after_script:
# собираем логи
- make docker-logs
# останавливаем окружение
- make docker-kill
artifacts:
# сохраняем логи
when: always
paths:
- logs
expire_in: 30 days
Bi abajade ti nṣiṣẹ iru iṣẹ-ṣiṣe kan, awọn iwe-itọsọna igbasilẹ ninu awọn ohun-ọṣọ yoo ni iṣẹ ati awọn igbasilẹ idanwo. Eyi ti o rọrun pupọ ni ọran ti awọn aṣiṣe. Idanwo kọọkan ni afiwe kọ akọọlẹ tirẹ, ṣugbọn Emi yoo sọrọ nipa eyi lọtọ.
$ docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /home/gitlab-runner/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
$ export TR_HTTP_PORT=$(shuf -i10000-60000 -n1)
$ export TR_HTTPS_PORT=$(shuf -i10000-60000 -n1)
$ mkdir ${CI_JOB_ID}
$ cp .indirect/docker-compose.yml ${CI_JOB_ID}/docker-compose.yml
$ make docker-up
docker-compose -f ${CI_JOB_ID:-.indirect}/docker-compose.yml kill
docker network rm testrail-network-${CI_JOB_ID:-local} || true
Error: No such network: testrail-network-204645172
docker network create testrail-network-${CI_JOB_ID:-local}
0a59552b4464b8ab484de6ae5054f3d5752902910bacb0a7b5eca698766d0331
docker-compose -f ${CI_JOB_ID:-.indirect}/docker-compose.yml pull
Pulling web ... done
Pulling fpm ... done
Pulling migration ... done
Pulling db ... done
docker-compose -f ${CI_JOB_ID:-.indirect}/docker-compose.yml up --force-recreate --renew-anon-volumes -d
Creating volume "204645172_static-content" with default driver
Creating testrail-mysql-204645172 ...
Creating testrail-mysql-204645172 ... done
Creating testrail-migration-204645172 ... done
Creating testrail-fpm-204645172 ... done
Creating testrail-web-204645172 ... done
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c6b76f9135ed registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" 13 seconds ago Up 1 second 0.0.0.0:51148->80/tcp, 0.0.0.0:25426->443/tcp testrail-web-204645172
01d303262d8e registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" 16 seconds ago Up 13 seconds 9000/tcp testrail-fpm-204645172
2cdab1edbf6a registry.gitlab.com/touchbit/image/testrail/migration:latest "docker-entrypoint.s…" 16 seconds ago Up 13 seconds 3306/tcp, 33060/tcp testrail-migration-204645172
826aaf7c0a29 mysql:5.7.22 "docker-entrypoint.s…" 18 seconds ago Up 16 seconds 3306/tcp testrail-mysql-204645172
6dbb3fae0322 registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" 36 seconds ago Up 22 seconds 0.0.0.0:44202->80/tcp, 0.0.0.0:20151->443/tcp testrail-web-204645084
3540f8d448ce registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" 38 seconds ago Up 35 seconds 9000/tcp testrail-fpm-204645084
70fea72aa10d mysql:5.7.22 "docker-entrypoint.s…" 40 seconds ago Up 37 seconds 3306/tcp testrail-mysql-204645084
d8aa24b2892d registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" About a minute ago Up 53 seconds 0.0.0.0:31103->80/tcp, 0.0.0.0:43872->443/tcp testrail-web-204644881
6d4ccd910fad registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" About a minute ago Up About a minute 9000/tcp testrail-fpm-204644881
685d8023a3ec mysql:5.7.22 "docker-entrypoint.s…" About a minute ago Up About a minute 3306/tcp testrail-mysql-204644881
1cdfc692003a registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" About a minute ago Up About a minute 0.0.0.0:44752->80/tcp, 0.0.0.0:23540->443/tcp testrail-web-204644793
6f26dfb2683e registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" About a minute ago Up About a minute 9000/tcp testrail-fpm-204644793
029e16b26201 mysql:5.7.22 "docker-entrypoint.s…" About a minute ago Up About a minute 3306/tcp testrail-mysql-204644793
c10443222ac6 registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" 5 hours ago Up 5 hours 0.0.0.0:57123->80/tcp, 0.0.0.0:31657->443/tcp testrail-web-204567103
04339229397e registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" 5 hours ago Up 5 hours 9000/tcp testrail-fpm-204567103
6ae0accab28d mysql:5.7.22 "docker-entrypoint.s…" 5 hours ago Up 5 hours 3306/tcp testrail-mysql-204567103
b66b60d79e43 registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" 5 hours ago Up 5 hours 0.0.0.0:56321->80/tcp, 0.0.0.0:58749->443/tcp testrail-web-204553690
033b1f46afa9 registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" 5 hours ago Up 5 hours 9000/tcp testrail-fpm-204553690
a8879c5ef941 mysql:5.7.22 "docker-entrypoint.s…" 5 hours ago Up 5 hours 3306/tcp testrail-mysql-204553690
069954ba6010 registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" 5 hours ago Up 5 hours 0.0.0.0:32869->80/tcp, 0.0.0.0:16066->443/tcp testrail-web-204553539
ed6b17d911a5 registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" 5 hours ago Up 5 hours 9000/tcp testrail-fpm-204553539
1a1eed057ea0 mysql:5.7.22 "docker-entrypoint.s…" 5 hours ago Up 5 hours 3306/tcp testrail-mysql-204553539
Gbogbo awọn iṣẹ ṣiṣe pari ni aṣeyọri
Awọn ohun-ọṣọ iṣẹ-ṣiṣe ni iṣẹ ninu ati awọn iwe idanwo
Ohun gbogbo dabi pe o lẹwa, ṣugbọn nuance kan wa. Opopona le jẹ ifagile ni ipa lakoko ti awọn idanwo isọpọ n ṣiṣẹ, ninu eyiti awọn apoti ṣiṣiṣẹ kii yoo da duro. Lati igba de igba o nilo lati nu olusare. Laanu, iṣẹ-ṣiṣe fun ilọsiwaju ni GitLab CE tun wa ni ipo Open
Ṣugbọn a ti ṣafikun ifilọlẹ iṣẹ kan ni ibamu si iṣeto, ko si si ẹnikan ti o kọ wa lati ṣiṣẹ pẹlu ọwọ.
Lọ si iṣẹ akanṣe wa -> CI / CD -> Awọn iṣeto ati ṣiṣe iṣẹ naa Clean runner
Lapapọ:
A ni olusare ikarahun kan.
Ko si awọn ija laarin awọn iṣẹ-ṣiṣe ati ayika.
A ṣiṣe awọn iṣẹ-ṣiṣe pẹlu Integration igbeyewo ni afiwe.
O le ṣiṣe awọn idanwo isọpọ boya ni agbegbe tabi ninu apoti kan.
Iṣẹ ati awọn akọọlẹ idanwo ni a gba ati so mọ iṣẹ opo gigun ti epo.
O ṣee ṣe lati nu olusare lati awọn aworan Docker atijọ.
Akoko iṣeto jẹ ~ 2 wakati.
Iyẹn ni gbogbo, ni otitọ. Emi yoo dun lati gba esi.