🥇JUnit ma GitLab CI me Kubernetes

ʻOiai ʻo ka ʻike pono ʻana o nā mea āpau he mea nui a pono hoʻi ka hoʻāʻo ʻana i kāu polokalamu, a he nui ka poʻe i hana maʻalahi ia no ka manawa lōʻihi, i ka nui o Habr ʻaʻohe mea ʻai hoʻokahi no ka hoʻonohonoho ʻana i kahi hui o ia mau huahana kaulana i ʻO kēia niche e like me (kā mākou punahele) GitLab a me JUnit . E hoopiha kakou i keia hakahaka!

Hoʻolauna

ʻO ka mea mua, e hāʻawi wau i kekahi pōʻaiapili:

No ka holo ʻana o kā mākou mau noi āpau ma Kubernetes, e noʻonoʻo mākou i ka holo ʻana i nā hoʻokolohua ma nā ʻōnaehana kūpono.
No ka hui ʻana a me ka hoʻolaha ʻana mākou e hoʻohana ai werf (ma ka ʻōlelo o nā ʻāpana hoʻolālā, ʻo ia hoʻi ke ʻano o ke komo ʻana o Helm).
ʻAʻole wau e hele i loko o nā kikoʻī o ka hana maoli ʻana o nā hoʻokolohua: i kā mākou hihia, kākau ka mea kūʻai aku i nā hoʻokolohua iā ia iho, a hōʻoia wale mākou i kā lākou hoʻomaka ʻana (a me ke ʻano o kahi hōʻike pili i ka noi hui).

He aha ke ʻano o ke kaʻina hana maʻamau?

Ke kūkulu ʻana i ka noi - e haʻalele mākou i ka wehewehe ʻana o kēia pae.
E hoʻolālā i ka palapala noi i kahi inoa ʻokoʻa o ka pūʻulu Kubernetes a hoʻomaka i ka hoʻāʻo.
Ke ʻimi nei i nā mea artifact a me ka hoʻopau ʻana i nā hōʻike JUnit me GitLab.
Holoi i kahi inoa inoa i hana mua ʻia.

I kēia manawa - i ka hoʻokō!

hoʻoponopono

GitLab CI

E hoʻomaka kākou me kahi ʻāpana .gitlab-ci.yaml, e wehewehe ana i ka lawe ʻana i ka noi a me nā hoʻāʻo holo. Ua lilo ka papa inoa i mea nui loa, no laila ua hoʻopiha piha ʻia me nā manaʻo:

variables:
# объявляем версию werf, которую собираемся использовать
  WERF_VERSION: "1.0 beta"

.base_deploy: &base_deploy
  script:
# создаем namespace в K8s, если его нет
    - kubectl --context="${WERF_KUBE_CONTEXT}" get ns ${CI_ENVIRONMENT_SLUG} || kubectl create ns ${CI_ENVIRONMENT_SLUG}
# загружаем werf и деплоим — подробнее об этом см. в документации
# (https://werf.io/how_to/gitlab_ci_cd_integration.html#deploy-stage)
    - type multiwerf && source <(multiwerf use ${WERF_VERSION})
    - werf version
    - type werf && source <(werf ci-env gitlab --tagging-strategy tag-or-branch --verbose)
    - werf deploy --stages-storage :local
      --namespace ${CI_ENVIRONMENT_SLUG}
      --set "global.commit_ref_slug=${CI_COMMIT_REF_SLUG:-''}"
# передаем переменную `run_tests`
# она будет использоваться в рендере Helm-релиза
      --set "global.run_tests=${RUN_TESTS:-no}"
      --set "global.env=${CI_ENVIRONMENT_SLUG}"
# изменяем timeout (бывают долгие тесты) и передаем его в релиз
      --set "global.ci_timeout=${CI_TIMEOUT:-900}"
     --timeout ${CI_TIMEOUT:-900}
  dependencies:
    - Build

.test-base: &test-base
  extends: .base_deploy
  before_script:
# создаем директорию для будущего отчета, исходя из $CI_COMMIT_REF_SLUG
    - mkdir /mnt/tests/${CI_COMMIT_REF_SLUG} || true
# вынужденный костыль, т.к. GitLab хочет получить артефакты в своем build-dir’е
    - mkdir ./tests || true
    - ln -s /mnt/tests/${CI_COMMIT_REF_SLUG} ./tests/${CI_COMMIT_REF_SLUG}
  after_script:
# после окончания тестов удаляем релиз вместе с Job’ом
# (и, возможно, его инфраструктурой)
    - type multiwerf && source <(multiwerf use ${WERF_VERSION})
    - werf version
    - type werf && source <(werf ci-env gitlab --tagging-strategy tag-or-branch --verbose)
    - werf dismiss --namespace ${CI_ENVIRONMENT_SLUG} --with-namespace
# мы разрешаем падения, но вы можете сделать иначе
  allow_failure: true
  variables:
    RUN_TESTS: 'yes'
# задаем контекст в werf
# (https://werf.io/how_to/gitlab_ci_cd_integration.html#infrastructure)
    WERF_KUBE_CONTEXT: 'admin@stage-cluster'
  tags:
# используем раннер с тегом `werf-runner`
    - werf-runner
  artifacts:
# требуется собрать артефакт для того, чтобы его можно было увидеть
# в пайплайне и скачать — например, для более вдумчивого изучения
    paths:
      - ./tests/${CI_COMMIT_REF_SLUG}/*
# артефакты старше недели будут удалены
    expire_in: 7 day
# важно: эти строки отвечают за парсинг отчета GitLab’ом
    reports:
      junit: ./tests/${CI_COMMIT_REF_SLUG}/report.xml

# для упрощения здесь показаны всего две стадии
# в реальности же у вас их будет больше — как минимум из-за деплоя
stages:
  - build
  - tests

build:
  stage: build
  script:
# сборка — снова по документации по werf
# (https://werf.io/how_to/gitlab_ci_cd_integration.html#build-stage)
    - type multiwerf && source <(multiwerf use ${WERF_VERSION})
    - werf version
    - type werf && source <(werf ci-env gitlab --tagging-strategy tag-or-branch --verbose)
    - werf build-and-publish --stages-storage :local
  tags:
    - werf-runner
  except:
    - schedules

run tests:
  <<: *test-base
  environment:
# "сама соль" именования namespace’а
# (https://docs.gitlab.com/ce/ci/variables/predefined_variables.html)
    name: tests-${CI_COMMIT_REF_SLUG}
  stage: tests
  except:
    - schedules

Kubernetes

I kēia manawa i ka papa kuhikuhi .helm/templates e hana kākou iā YAML me Job - tests-job.yaml - e holo i nā hoʻokolohua a me nā kumuwaiwai Kubernetes e pono ai. E ʻike i nā wehewehe ma hope o ka papa inoa ʻana:

{{- if eq .Values.global.run_tests "yes" }}
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: tests-script
data:
  tests.sh: |
    echo "======================"
    echo "${APP_NAME} TESTS"
    echo "======================"

    cd /app
    npm run test:ci
    cp report.xml /app/test_results/${CI_COMMIT_REF_SLUG}/

    echo ""
    echo ""
    echo ""

    chown -R 999:999 /app/test_results/${CI_COMMIT_REF_SLUG}
---
apiVersion: batch/v1
kind: Job
metadata:
  name: {{ .Chart.Name }}-test
  annotations:
    "helm.sh/hook": post-install,post-upgrade
    "helm.sh/hook-weight": "2"
    "werf/watch-logs": "true"
spec:
  activeDeadlineSeconds: {{ .Values.global.ci_timeout }}
  backoffLimit: 1
  template:
    metadata:
      name: {{ .Chart.Name }}-test
    spec:
      containers:
      - name: test
        command: ['bash', '-c', '/app/tests.sh']
{{ tuple "application" . | include "werf_container_image" | indent 8 }}
        env:
        - name: env
          value: {{ .Values.global.env }}
        - name: CI_COMMIT_REF_SLUG
          value: {{ .Values.global.commit_ref_slug }}
       - name: APP_NAME
          value: {{ .Chart.Name }}
{{ tuple "application" . | include "werf_container_env" | indent 8 }}
        volumeMounts:
        - mountPath: /app/test_results/
          name: data
        - mountPath: /app/tests.sh
          name: tests-script
          subPath: tests.sh
      tolerations:
      - key: dedicated
        operator: Exists
      - key: node-role.kubernetes.io/master
        operator: Exists
      restartPolicy: OnFailure
      volumes:
      - name: data
        persistentVolumeClaim:
          claimName: {{ .Chart.Name }}-pvc
      - name: tests-script
        configMap:
          name: tests-script
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: {{ .Chart.Name }}-pvc
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 10Mi
  storageClassName: {{ .Chart.Name }}-{{ .Values.global.commit_ref_slug }}
  volumeName: {{ .Values.global.commit_ref_slug }}

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: {{ .Values.global.commit_ref_slug }}
spec:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 10Mi
  local:
    path: /mnt/tests/
  nodeAffinity:
   required:
     nodeSelectorTerms:
     - matchExpressions:
       - key: kubernetes.io/hostname
         operator: In
         values:
         - kube-master
  persistentVolumeReclaimPolicy: Delete
  storageClassName: {{ .Chart.Name }}-{{ .Values.global.commit_ref_slug }}
{{- end }}

He aha ke ʻano o nā kumuwaiwai i wehewehe ʻia ma kēia hoʻonohonoho? I ka hoʻolālā ʻana, hana mākou i kahi inoa inoa kūikawā no ka papahana (hōʻike ʻia kēia ma .gitlab-ci.yaml - tests-${CI_COMMIT_REF_SLUG}) a ʻōwili i waho:

ConfigMap me ka palapala hōʻike;
Ioba me ka wehewehe ʻana o ka pod a me ke kuhikuhi i kuhikuhi ʻia command, e holo wale ana i na hoao;
PV a me PVC, hiki iā ʻoe ke mālama i ka ʻikepili hoʻāʻo.

E hoʻolohe i ke kūlana hoʻolauna me if i ka hoʻomaka ʻana o ka hōʻike - no laila, pono e hoʻopili ʻia nā faila YAML ʻē aʻe o ka pakuhi Helm me ka noi. huli kua hoʻolālā i ʻole lākou e kau ʻia i ka wā hoʻāʻo. ʻo ia:

{{- if ne .Values.global.run_tests "yes" }}
---
я другой ямлик
{{- end }}

Eia naʻe, inā nā hoʻokolohua koi i kekahi mau hana (no ka laʻana, Redis, RabbitMQ, Mongo, PostgreSQL...) - hiki i kā lākou YAML ole pio. E hoʻolālā iā lākou i loko o kahi hoʻokolohua hoʻāʻo pū kekahi ... hoʻoponopono iā lākou e like me kāu e ʻike pono ai, ʻoiaʻiʻo.

Paʻi hope

No ka mea ka hui a me ka hoʻolaha ʻana me ka hoʻohana ʻana i nā hana werf i kēia manawa wale ma ke kikowaena kūkulu (me ka gitlab-runner), a ua hoʻokuʻu ʻia ka pod me nā hoʻokolohua ma ka haku, pono ʻoe e hana i kahi papa kuhikuhi. /mnt/tests ma luna o ka haku a hāʻawi aku i ke kukini. no ka laʻana, ma o NFS. Hiki ke loaʻa kahi laʻana kikoʻī me nā wehewehe ma K8s palapala.

ʻO ka hopena:

user@kube-master:~$ cat /etc/exports | grep tests
/mnt/tests    IP_gitlab-builder/32(rw,nohide,insecure,no_subtree_check,sync,all_squash,anonuid=999,anongid=998)

user@gitlab-runner:~$ cat /etc/fstab | grep tests
IP_kube-master:/mnt/tests    /mnt/tests   nfs4    _netdev,auto  0       0

ʻAʻohe mea i pāpā i ka hana ʻana i ka māhele NFS ma luna o gitlab-runner, a laila kau ʻia i loko o nā pods.

i hoʻopuka

Ke nīnau nei paha ʻoe no ke aha e hoʻopili ai i nā mea āpau ma ka hoʻokumu ʻana i kahi Job inā hiki iā ʻoe ke holo i kahi palapala me nā hoʻāʻo pololei ma ka mea holo shell? He mea ʻole ka pane...

Pono kekahi mau ho'āʻo e komo i ka ʻōnaehana (MongoDB, RabbitMQ, PostgreSQL, a me nā mea ʻē aʻe) e hōʻoia i kā lākou hana pono. Hoʻohui mākou i ka hoʻāʻo ʻana - me kēia ala, lilo ia i mea maʻalahi e hoʻokomo i kēlā mau hui hou. Ma waho aʻe o kēia, loaʻa iā mākou kūlana ke ala hoʻolālā (ʻoiai inā e hoʻohana ana i ka NFS, ke kau ʻana o nā papa kuhikuhi).

hopena

He aha kā mākou e ʻike ai ke hoʻohana mākou i ka hoʻonohonoho i hoʻomākaukau ʻia?

E hōʻike ka noi hoʻohui i nā helu helu hōʻuluʻulu no nā hoʻokolohua e holo ana i kāna pipeline hou loa:

Hiki ke kaomi i kēlā me kēia hewa no nā kikoʻī:

NB: E hoʻomaopopo ka mea heluhelu e hoʻāʻo nei mākou i kahi noi NodeJS, a ma nā kiʻi kiʻi - .NET ... Mai kāhāhā: ʻo ia wale nō i ka wā e hoʻomākaukau ai i ka ʻatikala, ʻaʻohe hewa i ʻike ʻia i ka hoʻāʻo ʻana i ka noi mua, akā. ua loaa ma kekahi.

hopena

E like me kāu e ʻike ai, ʻaʻohe mea paʻakikī!

Ma ke kumu, inā loaʻa iā ʻoe kahi ʻohi pūpū a hana ia, akā ʻaʻole pono ʻoe i nā Kubernetes, ʻoi aku ka maʻalahi o ka hoʻopili ʻana i ka hoʻāʻo iā ia ma mua o ka wehewehe ʻana ma aneʻi. A i loko Nā palapala GitLab CI e ʻike ʻoe i nā hiʻohiʻona no Ruby, Go, Gradle, Maven a me kekahi mau mea ʻē aʻe.

PS

E heluhelu pū ma kā mākou blog:

Source: www.habr.com

JUnit ma GitLab CI me Kubernetes

Hoʻolauna

hoʻoponopono

GitLab CI

Kubernetes

Paʻi hope

i hoʻopuka

hopena

hopena

PS

Pākuʻi i ka manaʻo hoʻopuka ana pane