ProHoster > Pūnaewele > Nā Administration > ʻO GitLab Shell Runner. Hoʻomaka hoʻokūkū o nā lawelawe i hoʻāʻo ʻia me ka hoʻohana ʻana iā Docker Compose
ʻO GitLab Shell Runner. Hoʻomaka hoʻokūkū o nā lawelawe i hoʻāʻo ʻia me ka hoʻohana ʻana iā Docker Compose
He mea hoihoi kēia ʻatikala i nā mea hoʻāʻo a me nā mea hoʻomohala, akā ua manaʻo nui ʻia no nā loea automation e kū nei i ka pilikia o ka hoʻonohonoho ʻana iā GitLab CI/CD no ka hoʻāʻo hoʻohui ʻana i nā kūlana o ka lawa ʻole o nā kumu waiwai a me ka ʻole o kahi pahu. kahua hoʻokani pila. E haʻi wau iā ʻoe pehea e hoʻonohonoho ai i ka hoʻonohonoho ʻana i nā kaiapuni hoʻāʻo e hoʻohana ana i ka docker compose ma kahi hoʻokahi ʻo GitLab shell runner a no laila i ka wā e kau ana i kekahi mau kaiapuni, ʻaʻole e hoʻopilikia nā lawelawe i hoʻokuʻu ʻia kekahi i kekahi.
I kaʻu hoʻomaʻamaʻa, ua ʻike pinepine ʻia ka hoʻāʻo ʻana i ka hoʻohui ʻana i "mālama ʻia" ma nā papahana. A pinepine ka pilikia mua a koʻikoʻi loa ʻo ka pipeline CI, kahi e hoʻāʻo ai i ka hoʻohui e hoʻomohala ʻia ana lawelawe ʻia (mau) lawelawe ma kahi kahua dev/stage. Ua pilikia kēia i kekahi mau pilikia:
Ma muli o nā hemahema o kahi lawelawe i ka wā o ka hoʻāʻo hoʻohui ʻana, hiki ke hōʻino ʻia ke kaapuni hoʻāʻo e ka ʻikepili haki. Aia kekahi mau hihia i ka hoʻouna ʻana i kahi noi me kahi ʻano JSON i haki ʻia i ka lawelawe, kahi i hiki ʻole ai ke kū.
Hoʻolohi o ke kaapuni hoʻāʻo i ka piʻi ʻana o ka ʻikepili hoʻāʻo. Manaʻo wau he mea kūpono ʻole ka wehewehe ʻana i kahi laʻana me ka hoʻomaʻemaʻe / hoʻihoʻi ʻana i ka waihona. I kaʻu hana, ʻaʻole au i ʻike i kahi papahana kahi i holo mālie ai kēia kaʻina hana.
Hiki ke hoʻopilikia i ka hana o ke kaapuni hoʻāʻo i ka wā e hoʻāʻo ai i nā hoʻonohonoho ʻōnaehana maʻamau. No ka laʻana, mea hoʻohana / pūʻulu / password / kulekele noi.
ʻO ka ʻikepili hoʻāʻo mai nā hoʻāʻo automated e paʻakikī ke ola no nā mea hōʻike lima.
E ʻōlelo kekahi e hoʻomaʻemaʻe nā autotest maikaʻi i ka ʻikepili ma hope o lākou iho. He mau hoʻopaʻapaʻa koʻu:
He mea maʻalahi ka hoʻohana ʻana i nā kuʻuna dynamic.
ʻAʻole hiki ke wehe ʻia nā mea āpau mai ka ʻōnaehana ma o ka API. No ka laʻana, ʻaʻole i hoʻokō ʻia kahi kelepona e holoi i kahi mea no ka mea kūʻē ia i ka loiloi ʻoihana.
I ka hana ʻana i kahi mea ma o ka API, hiki ke hana ʻia kahi nui o nā metadata, kahi pilikia e holoi ai.
Inā he mau hilinaʻi ko nā hoʻokolohua ma waena o lākou iho, a laila ke kaʻina hana o ka hoʻomaʻemaʻe ʻana i ka ʻikepili ma hope o ka holo ʻana i nā hoʻokolohua e lilo i poʻo.
Hoʻohui (a, i koʻu manaʻo, ʻaʻole i ʻāpono ʻia) nā kelepona i ka API.
A ʻo ka hoʻopaʻapaʻa nui: i ka wā e hoʻomaka ai ka hoʻomaʻemaʻe ʻana i ka ʻikepili mai ka waihona. Ke hoʻololi nei kēia i PK/FK circus maoli! Lohe mākou mai nā mea hoʻomohala: "Ua hoʻohui hou wau / wehe / inoa hou i kahi hōʻailona, no ke aha i loaʻa ai nā hoʻokolohua hoʻohui 100500?"
I koʻu manaʻo, ʻo ka hoʻonā maikaʻi loa he kaiapuni dynamic.
Nui ka poʻe e hoʻohana i ka docker-compose e holo i kahi kaiapuni hoʻāʻo, akā liʻiliʻi ka poʻe e hoʻohana i ka docker-compose i ka wā e hana ana i ka hoʻāʻo hoʻohui ʻana ma CI/CD. A ma ʻaneʻi ʻaʻole wau e noʻonoʻo i nā kubernetes, swarm a me nā kahua hoʻokani pahu pahu. ʻAʻole kēlā me kēia ʻoihana iā lākou. He mea maikaʻi inā ʻo docker-compose.yml ka honua.
ʻOiai inā loaʻa iā mākou kā mākou mea holo QA ponoʻī, pehea e hiki ai iā mākou ke hōʻoia i ka lawelawe ʻana ma o docker-compose ʻaʻole e hoʻopilikia kekahi i kekahi?
Pehea e ʻohi ai i nā lāʻau o nā lawelawe i hoʻāʻo ʻia?
Pehea e hoʻomaʻemaʻe ai i ka mea holo?
Loaʻa iaʻu kaʻu mea holo GitLab ponoʻī no kaʻu mau papahana a ua hālāwai wau i kēia mau nīnau i ka wā o ka hoʻomohala ʻana Mea kūʻai Java no ka mea, TestRail. ʻOi aku ka pololei, i ka wā e holo ana i nā hoʻokolohua hoʻohui. Ma lalo e hoʻoponopono mākou i kēia mau pilikia me ka hoʻohana ʻana i nā laʻana mai kēia papahana.
No ka mea holo, paipai wau i ka mīkini virtual Linux me 4 vCPU, 4 GB RAM, 50 GB HDD.
Nui ka ʻike e pili ana i ka hoʻonohonoho ʻana i ka gitlab-runner ma ka Pūnaewele, no laila pōkole:
E komo i ka mīkini ma o SSH
Inā loaʻa iā ʻoe ka liʻiliʻi ma mua o 8 GB RAM, a laila paipai wau e hoʻololi i 10 GBno laila ʻaʻole hele mai ka mea pepehi OOM a pepehi i kā mākou mau hana ma muli o ka nele o ka RAM. Hiki ke hana ʻia inā ʻoi aku ma mua o 5 mau hana i ka manawa like. E holo mālie ana nā hana, akā paʻa.
Laʻana me ka mea pepehi kanaka OOM
Inā ʻike ʻoe i nā moʻolelo hana bash: line 82: 26474 Killed, a laila hoʻokō wale i ka mea kukini sudo dmesg | grep 26474
[26474] 1002 26474 1061935 123806 339 0 0 java
Out of memory: Kill process 26474 (java) score 127 or sacrifice child
Killed process 26474 (java) total-vm:4247740kB, anon-rss:495224kB, file-rss:0kB, shmem-rss:0kB
A inā like ke kiʻi me kēia, a laila e hoʻohui i ka swap a i ʻole e hoʻohui i ka RAM.
E ʻae kēia iā ʻoe e holo i nā hana like ma luna o hoʻokahi mea holo. Heluhelu hou aku maanei.
Inā loaʻa iā ʻoe kahi mīkini ʻoi aku ka ikaika, no ka laʻana 8 vCPU, 16 GB RAM, a laila hiki ke hana ʻia kēia mau helu ma ka liʻiliʻi he 2 mau manawa nui. Akā, pili ia i ka mea e hoʻokuʻu ʻia ma luna o kēia mea holo a me ka nui.
ʻO ka hana nui he universal docker-compose.yml, hiki i nā mea hoʻomohala / nā mea hoʻāʻo ke hoʻohana ma ka ʻāina a ma ka pipeline CI.
ʻO ka mea mua, hana mākou i nā inoa lawelawe kūikawā no CI. ʻO kekahi o nā ʻokoʻa kūʻokoʻa ma GitLab CI ʻo ia ka loli CI_JOB_ID. Inā ʻoe e kuhikuhi container_name me ke ano "service-${CI_JOB_ID:-local}", a laila ma ka hihia:
inā CI_JOB_ID ʻaʻole i wehewehe ʻia i nā mea hoʻololi kaiapuni,
a laila ʻo ka inoa lawelawe service-local
inā CI_JOB_ID wehewehe ʻia ma nā ʻano hoʻololi kaiapuni (no ka laʻana 123),
a laila ʻo ka inoa lawelawe service-123
ʻO ka lua, hana mākou i kahi pūnaewele maʻamau no nā lawelawe i hoʻokuʻu ʻia. Hāʻawi kēia iā mākou i ka hoʻokaʻawale pae pūnaewele i ka wā e holo ana i nā kaiapuni hoʻāʻo he nui.
version: "3"
# Для корректной работы web (php) и fmt нужно,
# чтобы контейнеры имели общий исполняемый контент.
# В нашем случае, это директория /var/www/testrail
volumes:
static-content:
# Изолируем окружение на сетевом уровне
networks:
default:
external:
name: testrail-network-${CI_JOB_ID:-local}
services:
db:
image: mysql:5.7.22
# Каждый container_name содержит ${CI_JOB_ID:-local}
container_name: "testrail-mysql-${CI_JOB_ID:-local}"
environment:
MYSQL_HOST: db
MYSQL_DATABASE: mydb
MYSQL_ROOT_PASSWORD: 1234
SKIP_GRANT_TABLES: 1
SKIP_NETWORKING: 1
SERVICE_TAGS: dev
SERVICE_NAME: mysql
networks:
- default
migration:
image: registry.gitlab.com/touchbit/image/testrail/migration:latest
container_name: "testrail-migration-${CI_JOB_ID:-local}"
links:
- db
depends_on:
- db
networks:
- default
fpm:
image: registry.gitlab.com/touchbit/image/testrail/fpm:latest
container_name: "testrail-fpm-${CI_JOB_ID:-local}"
volumes:
- static-content:/var/www/testrail
links:
- db
networks:
- default
web:
image: registry.gitlab.com/touchbit/image/testrail/web:latest
container_name: "testrail-web-${CI_JOB_ID:-local}"
# Если переменные TR_HTTP_PORT или TR_HTTPS_PORTS не определены,
# то сервис поднимается на 80 и 443 порту соответственно.
ports:
- ${TR_HTTP_PORT:-80}:80
- ${TR_HTTPS_PORT:-443}:443
volumes:
- static-content:/var/www/testrail
links:
- db
- fpm
networks:
- default
Integration:
stage: test
tags:
- my-shell-runner
before_script:
# Аутентифицируемся в registry
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
# Генерируем псевдоуникальные TR_HTTP_PORT и TR_HTTPS_PORT
- export TR_HTTP_PORT=$(shuf -i10000-60000 -n1)
- export TR_HTTPS_PORT=$(shuf -i10000-60000 -n1)
# создаем директорию с идентификатором задачи
- mkdir ${CI_JOB_ID}
# копируем в созданную директорию наш docker-compose.yml
# чтобы контекст был разный для каждой задачи
- cp .indirect/docker-compose.yml ${CI_JOB_ID}/docker-compose.yml
script:
# поднимаем наше окружение
- make docker-up
# запускаем тесты исполняемым jar (у меня так)
- java -jar itest.jar --http-port ${TR_HTTP_PORT} --https-port ${TR_HTTPS_PORT}
# или в контейнере
- docker run --network=testrail-network-${CI_JOB_ID:-local} --rm itest
after_script:
# собираем логи
- make docker-logs
# останавливаем окружение
- make docker-kill
artifacts:
# сохраняем логи
when: always
paths:
- logs
expire_in: 30 days
Ma muli o ka holo ʻana o ia hana, e loaʻa i ka papa kuhikuhi logs i nā artifacts nā lawelawe a me nā log hoʻāʻo. He mea maʻalahi loa i ka hihia o nā hewa. Ke kākau nei kēlā me kēia hoʻāʻo i kāna log ponoʻī, akā e kamaʻilio kaʻawale wau e pili ana i kēia.
$ docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /home/gitlab-runner/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
$ export TR_HTTP_PORT=$(shuf -i10000-60000 -n1)
$ export TR_HTTPS_PORT=$(shuf -i10000-60000 -n1)
$ mkdir ${CI_JOB_ID}
$ cp .indirect/docker-compose.yml ${CI_JOB_ID}/docker-compose.yml
$ make docker-up
docker-compose -f ${CI_JOB_ID:-.indirect}/docker-compose.yml kill
docker network rm testrail-network-${CI_JOB_ID:-local} || true
Error: No such network: testrail-network-204645172
docker network create testrail-network-${CI_JOB_ID:-local}
0a59552b4464b8ab484de6ae5054f3d5752902910bacb0a7b5eca698766d0331
docker-compose -f ${CI_JOB_ID:-.indirect}/docker-compose.yml pull
Pulling web ... done
Pulling fpm ... done
Pulling migration ... done
Pulling db ... done
docker-compose -f ${CI_JOB_ID:-.indirect}/docker-compose.yml up --force-recreate --renew-anon-volumes -d
Creating volume "204645172_static-content" with default driver
Creating testrail-mysql-204645172 ...
Creating testrail-mysql-204645172 ... done
Creating testrail-migration-204645172 ... done
Creating testrail-fpm-204645172 ... done
Creating testrail-web-204645172 ... done
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c6b76f9135ed registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" 13 seconds ago Up 1 second 0.0.0.0:51148->80/tcp, 0.0.0.0:25426->443/tcp testrail-web-204645172
01d303262d8e registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" 16 seconds ago Up 13 seconds 9000/tcp testrail-fpm-204645172
2cdab1edbf6a registry.gitlab.com/touchbit/image/testrail/migration:latest "docker-entrypoint.s…" 16 seconds ago Up 13 seconds 3306/tcp, 33060/tcp testrail-migration-204645172
826aaf7c0a29 mysql:5.7.22 "docker-entrypoint.s…" 18 seconds ago Up 16 seconds 3306/tcp testrail-mysql-204645172
6dbb3fae0322 registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" 36 seconds ago Up 22 seconds 0.0.0.0:44202->80/tcp, 0.0.0.0:20151->443/tcp testrail-web-204645084
3540f8d448ce registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" 38 seconds ago Up 35 seconds 9000/tcp testrail-fpm-204645084
70fea72aa10d mysql:5.7.22 "docker-entrypoint.s…" 40 seconds ago Up 37 seconds 3306/tcp testrail-mysql-204645084
d8aa24b2892d registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" About a minute ago Up 53 seconds 0.0.0.0:31103->80/tcp, 0.0.0.0:43872->443/tcp testrail-web-204644881
6d4ccd910fad registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" About a minute ago Up About a minute 9000/tcp testrail-fpm-204644881
685d8023a3ec mysql:5.7.22 "docker-entrypoint.s…" About a minute ago Up About a minute 3306/tcp testrail-mysql-204644881
1cdfc692003a registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" About a minute ago Up About a minute 0.0.0.0:44752->80/tcp, 0.0.0.0:23540->443/tcp testrail-web-204644793
6f26dfb2683e registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" About a minute ago Up About a minute 9000/tcp testrail-fpm-204644793
029e16b26201 mysql:5.7.22 "docker-entrypoint.s…" About a minute ago Up About a minute 3306/tcp testrail-mysql-204644793
c10443222ac6 registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" 5 hours ago Up 5 hours 0.0.0.0:57123->80/tcp, 0.0.0.0:31657->443/tcp testrail-web-204567103
04339229397e registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" 5 hours ago Up 5 hours 9000/tcp testrail-fpm-204567103
6ae0accab28d mysql:5.7.22 "docker-entrypoint.s…" 5 hours ago Up 5 hours 3306/tcp testrail-mysql-204567103
b66b60d79e43 registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" 5 hours ago Up 5 hours 0.0.0.0:56321->80/tcp, 0.0.0.0:58749->443/tcp testrail-web-204553690
033b1f46afa9 registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" 5 hours ago Up 5 hours 9000/tcp testrail-fpm-204553690
a8879c5ef941 mysql:5.7.22 "docker-entrypoint.s…" 5 hours ago Up 5 hours 3306/tcp testrail-mysql-204553690
069954ba6010 registry.gitlab.com/touchbit/image/testrail/web:latest "nginx -g 'daemon of…" 5 hours ago Up 5 hours 0.0.0.0:32869->80/tcp, 0.0.0.0:16066->443/tcp testrail-web-204553539
ed6b17d911a5 registry.gitlab.com/touchbit/image/testrail/fpm:latest "docker-php-entrypoi…" 5 hours ago Up 5 hours 9000/tcp testrail-fpm-204553539
1a1eed057ea0 mysql:5.7.22 "docker-entrypoint.s…" 5 hours ago Up 5 hours 3306/tcp testrail-mysql-204553539
Pau nā hana a pau
Loaʻa i nā mea hana hana nā moʻolelo lawelawe a me ka hoʻāʻo
He nani nā mea a pau, akā aia kahi nuance. Hiki ke hoʻopau ikaika ʻia ka paipu i ka wā e holo ana nā hoʻāʻo hoʻohui, i ia manawa ʻaʻole e pani ʻia nā ipu holo. I kēlā me kēia manawa pono ʻoe e hoʻomaʻemaʻe i ka mea holo. ʻO ka mea pōʻino, ke kūlana nei ka hana no ka hoʻomaikaʻi ʻana i ka GitLab CE Open
Akā ua hoʻohui mākou i ka hoʻomaka ʻana o kahi hana e like me ka papa kuhikuhi, a ʻaʻohe mea e pāpā iā mākou e holo lima.
E hele i kā mākou papahana -> CI / CD -> Hoʻonohonoho a holo i ka hana Clean runner
Hōʻuluʻulu:
Hoʻokahi kā mākou kukini pūpū.
ʻAʻohe paio ma waena o nā hana a me ke kaiapuni.
Holo mākou i nā hana me nā hoʻokolohua hoʻohui like.
Hiki iā ʻoe ke holo i nā hoʻokolohua hoʻohui ʻāina a i ʻole i loko o kahi pahu.
ʻOhi ʻia nā lāʻau lawelawe a me ka hoʻāʻo a hoʻopili ʻia i ka hana pipeline.
Hiki ke hoʻomaʻemaʻe i ka mea holo mai nā kiʻi Docker kahiko.
ʻO ka manawa hoʻonohonoho he ~ 2 hola.
ʻO ia wale nō, ʻoiaʻiʻo. E hauʻoli wau i ka loaʻa ʻana o nā manaʻo.