🥇JUnit i GitLab CI ma Kubernetes

E ui i le mea moni e iloa lelei e tagata uma o le suʻeina o lau polokalama e taua ma talafeagai, ma o le toʻatele o loʻo faia otometi mo se taimi umi, i le lautele o Habr e leai se fua e tasi mo le faʻatulagaina o se tuufaatasiga o oloa lauiloa i totonu. lenei faʻailoga e pei o (matou fiafia) GitLab ma JUnit . Sei o tatou faatumu lenei avanoa!

Folasaga

Muamua, seʻi ou tuʻuina atu se faʻamatalaga:

Talu ai o a matou talosaga uma e faʻaoga i luga o Kubernetes, matou te mafaufau e faʻatino suʻega i luga o atinaʻe talafeagai.
Mo le faʻapipiʻiina ma le faʻapipiʻiina matou te faʻaaogaina werf (e tusa ai ma vaega tetele, o lona uiga otometi foi o loʻo aʻafia ai Helm).
O le a ou le alu i faʻamatalaga o le faia moni o suʻega: i la matou tulaga, e tusia e le kalani ia suʻega ia lava, ma matou naʻo le faʻamautinoaina o latou faʻalauiloa (ma le i ai o se lipoti talafeagai i le talosaga tuʻufaʻatasia).

O a foliga o le fa'asologa lautele o gaioiga?

Fausiaina o le talosaga - o le a matou aveese le faʻamatalaga o lenei laasaga.
Tu'u le talosaga i se isi igoa avanoa o le vaega Kubernetes ma amata su'ega.
Su'esu'e mo mea fa'apitoa ma fa'asalalau lipoti JUnit ma GitLab.
Ave'ese se igoa avanoa na faia muamua.

I le taimi nei - i le faʻatinoga!

tonu

GitLab CI

Tatou amata i se vaega .gitlab-ci.yaml, o loʻo faʻamatalaina le faʻaogaina o le talosaga ma faʻataʻitaʻiga suʻega. O le lisi na foliga mai e matua tele lava, o lea na faʻaopoopoina atoatoa i faʻamatalaga:

variables:
# объявляем версию werf, которую собираемся использовать
  WERF_VERSION: "1.0 beta"

.base_deploy: &base_deploy
  script:
# создаем namespace в K8s, если его нет
    - kubectl --context="${WERF_KUBE_CONTEXT}" get ns ${CI_ENVIRONMENT_SLUG} || kubectl create ns ${CI_ENVIRONMENT_SLUG}
# загружаем werf и деплоим — подробнее об этом см. в документации
# (https://werf.io/how_to/gitlab_ci_cd_integration.html#deploy-stage)
    - type multiwerf && source <(multiwerf use ${WERF_VERSION})
    - werf version
    - type werf && source <(werf ci-env gitlab --tagging-strategy tag-or-branch --verbose)
    - werf deploy --stages-storage :local
      --namespace ${CI_ENVIRONMENT_SLUG}
      --set "global.commit_ref_slug=${CI_COMMIT_REF_SLUG:-''}"
# передаем переменную `run_tests`
# она будет использоваться в рендере Helm-релиза
      --set "global.run_tests=${RUN_TESTS:-no}"
      --set "global.env=${CI_ENVIRONMENT_SLUG}"
# изменяем timeout (бывают долгие тесты) и передаем его в релиз
      --set "global.ci_timeout=${CI_TIMEOUT:-900}"
     --timeout ${CI_TIMEOUT:-900}
  dependencies:
    - Build

.test-base: &test-base
  extends: .base_deploy
  before_script:
# создаем директорию для будущего отчета, исходя из $CI_COMMIT_REF_SLUG
    - mkdir /mnt/tests/${CI_COMMIT_REF_SLUG} || true
# вынужденный костыль, т.к. GitLab хочет получить артефакты в своем build-dir’е
    - mkdir ./tests || true
    - ln -s /mnt/tests/${CI_COMMIT_REF_SLUG} ./tests/${CI_COMMIT_REF_SLUG}
  after_script:
# после окончания тестов удаляем релиз вместе с Job’ом
# (и, возможно, его инфраструктурой)
    - type multiwerf && source <(multiwerf use ${WERF_VERSION})
    - werf version
    - type werf && source <(werf ci-env gitlab --tagging-strategy tag-or-branch --verbose)
    - werf dismiss --namespace ${CI_ENVIRONMENT_SLUG} --with-namespace
# мы разрешаем падения, но вы можете сделать иначе
  allow_failure: true
  variables:
    RUN_TESTS: 'yes'
# задаем контекст в werf
# (https://werf.io/how_to/gitlab_ci_cd_integration.html#infrastructure)
    WERF_KUBE_CONTEXT: 'admin@stage-cluster'
  tags:
# используем раннер с тегом `werf-runner`
    - werf-runner
  artifacts:
# требуется собрать артефакт для того, чтобы его можно было увидеть
# в пайплайне и скачать — например, для более вдумчивого изучения
    paths:
      - ./tests/${CI_COMMIT_REF_SLUG}/*
# артефакты старше недели будут удалены
    expire_in: 7 day
# важно: эти строки отвечают за парсинг отчета GitLab’ом
    reports:
      junit: ./tests/${CI_COMMIT_REF_SLUG}/report.xml

# для упрощения здесь показаны всего две стадии
# в реальности же у вас их будет больше — как минимум из-за деплоя
stages:
  - build
  - tests

build:
  stage: build
  script:
# сборка — снова по документации по werf
# (https://werf.io/how_to/gitlab_ci_cd_integration.html#build-stage)
    - type multiwerf && source <(multiwerf use ${WERF_VERSION})
    - werf version
    - type werf && source <(werf ci-env gitlab --tagging-strategy tag-or-branch --verbose)
    - werf build-and-publish --stages-storage :local
  tags:
    - werf-runner
  except:
    - schedules

run tests:
  <<: *test-base
  environment:
# "сама соль" именования namespace’а
# (https://docs.gitlab.com/ce/ci/variables/predefined_variables.html)
    name: tests-${CI_COMMIT_REF_SLUG}
  stage: tests
  except:
    - schedules

Kubernetes

I le taimi nei i le lisi .helm/templates tatou faia le YAML ma Iopu - tests-job.yaml - e faʻatino suʻega ma punaoa Kubernetes e manaʻomia. Va'ai fa'amatalaga pe a mae'a lisi:

{{- if eq .Values.global.run_tests "yes" }}
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: tests-script
data:
  tests.sh: |
    echo "======================"
    echo "${APP_NAME} TESTS"
    echo "======================"

    cd /app
    npm run test:ci
    cp report.xml /app/test_results/${CI_COMMIT_REF_SLUG}/

    echo ""
    echo ""
    echo ""

    chown -R 999:999 /app/test_results/${CI_COMMIT_REF_SLUG}
---
apiVersion: batch/v1
kind: Job
metadata:
  name: {{ .Chart.Name }}-test
  annotations:
    "helm.sh/hook": post-install,post-upgrade
    "helm.sh/hook-weight": "2"
    "werf/watch-logs": "true"
spec:
  activeDeadlineSeconds: {{ .Values.global.ci_timeout }}
  backoffLimit: 1
  template:
    metadata:
      name: {{ .Chart.Name }}-test
    spec:
      containers:
      - name: test
        command: ['bash', '-c', '/app/tests.sh']
{{ tuple "application" . | include "werf_container_image" | indent 8 }}
        env:
        - name: env
          value: {{ .Values.global.env }}
        - name: CI_COMMIT_REF_SLUG
          value: {{ .Values.global.commit_ref_slug }}
       - name: APP_NAME
          value: {{ .Chart.Name }}
{{ tuple "application" . | include "werf_container_env" | indent 8 }}
        volumeMounts:
        - mountPath: /app/test_results/
          name: data
        - mountPath: /app/tests.sh
          name: tests-script
          subPath: tests.sh
      tolerations:
      - key: dedicated
        operator: Exists
      - key: node-role.kubernetes.io/master
        operator: Exists
      restartPolicy: OnFailure
      volumes:
      - name: data
        persistentVolumeClaim:
          claimName: {{ .Chart.Name }}-pvc
      - name: tests-script
        configMap:
          name: tests-script
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: {{ .Chart.Name }}-pvc
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 10Mi
  storageClassName: {{ .Chart.Name }}-{{ .Values.global.commit_ref_slug }}
  volumeName: {{ .Values.global.commit_ref_slug }}

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: {{ .Values.global.commit_ref_slug }}
spec:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 10Mi
  local:
    path: /mnt/tests/
  nodeAffinity:
   required:
     nodeSelectorTerms:
     - matchExpressions:
       - key: kubernetes.io/hostname
         operator: In
         values:
         - kube-master
  persistentVolumeReclaimPolicy: Delete
  storageClassName: {{ .Chart.Name }}-{{ .Values.global.commit_ref_slug }}
{{- end }}

O le a le ituaiga punaoa faamatalaina i lenei faatulagaga? A faʻapipiʻi, matou te fatuina se igoa tulaga ese mo le poloketi (o loʻo faʻaalia i .gitlab-ci.yaml - tests-${CI_COMMIT_REF_SLUG}) ma taʻavale i fafo:

ConfigMap fa'atasi ai ma le su'ega tusi;
gāluega faʻatasi ai ma faʻamatalaga o le pod ma le faʻatonuga faʻapitoa command, lea na'o le fa'atinoina o su'ega;
PV ma le PVC, lea e mafai ai ona e teuina faʻamaumauga o suʻega.

Fa'alogo lelei ile tulaga tomua ma if i le amataga o le faʻaaliga - e tusa ai, o isi faila YAML o le siata Helm ma le talosaga e tatau ona afifi i totonu liliu mamanu ina ia latou le maua deployed i le taimi o suʻega. O lena lava:

{{- if ne .Values.global.run_tests "yes" }}
---
я другой ямлик
{{- end }}

Ae peitaʻi, afai o suʻega mana'omia nisi atina'e (mo se faʻataʻitaʻiga, Redis, RabbitMQ, Mongo, PostgreSQL...) - o latou YAML e mafai ona lē tape. Fa'asoa i latou i se siosiomaga fa'ata'ita'i fa'apea fo'i... fetuutuuna'i pe a e mana'o e fetaui, ioe.

pa'i mulimuli

Aua fa'apotopotoga ma le fa'aogaina e fa'aaoga ai galuega werf mo le taimi nei ae i luga o le fale faufale (faʻatasi ai ma gitlab-runner), ma o le pod ma suʻega e faʻalauiloa i luga o le matai, e tatau ona e fatuina se lisi /mnt/tests i luga o le matai ma avatu i le tagata tamoʻe, mo se faʻataʻitaʻiga, e ala i le NFS. O se faʻataʻitaʻiga faʻapitoa ma faʻamatalaga e mafai ona maua ile K8s fa'amaumauga.

O le taunuuga o le a:

user@kube-master:~$ cat /etc/exports | grep tests
/mnt/tests    IP_gitlab-builder/32(rw,nohide,insecure,no_subtree_check,sync,all_squash,anonuid=999,anongid=998)

user@gitlab-runner:~$ cat /etc/fstab | grep tests
IP_kube-master:/mnt/tests    /mnt/tests   nfs4    _netdev,auto  0       0

E leai se tasi e faʻasaina le faia saʻo o se NFS i luga o le gitlab-runner, ona faʻapipiʻi lea i totonu o pods.

mataʻi

Atonu e te fesili pe aisea e faʻalavelave ai mea uma e ala i le fatuina o se Iopu pe afai e mafai ona e faʻataʻitaʻiina se tusitusiga ma suʻega saʻo i luga o le atigi tamoʻe? O le tali e fai si faatauvaa...

O nisi o su'ega e mana'omia le avanoa i mea tetele (MongoDB, RabbitMQ, PostgreSQL, ma isi) e fa'amaonia ai o lo'o sa'o lelei. Matou te faia faʻataʻitaʻiga faʻatasi - faʻatasi ai ma lenei faiga, e faigofie ai ona faʻapipiʻi ia mea faʻaopoopo. E le gata i lea, matou te maua fua auala fa'apipi'i (tusa lava pe fa'aaoga le NFS, fa'aopoopo fa'aopoopo o fa'atonuga).

iʻuga

O le a le mea o le a tatou vaʻaia pe a tatou faʻaogaina le faʻatulagaga ua saunia?

O le talosaga tu'ufa'atasia o le a fa'aalia ai fa'amaumauga o fa'amaumauga mo su'ega o lo'o fa'atinoina i lana paipa fou:

O mea sese ta'itasi e mafai ona kiliki iinei mo fa'amatalaga:

NB: O le a matauina e le tagata faitau faʻalogo o loʻo matou suʻeina se talosaga NodeJS, ma i le screenshots - .NET ... Aua e te ofo: naʻo le taimi lava e saunia ai le tusiga, e leai ni mea sese na maua i le suʻeina o le talosaga muamua, ae latou na maua i se isi.

iʻuga

E pei ona e vaʻaia, e leai se mea faigata!

I le mataupu faavae, afai ua i ai sau atigi aoina ma e aoga, ae e te le manaʻomia Kubernetes, faʻapipiʻi suʻega o le a sili atu ona faigofie nai lo le faʻamatalaina iinei. Ma i totonu GitLab CI faʻamaumauga o le ae mauaina faʻataʻitaʻiga mo Ruby, Go, Gradle, Maven ma isi.

SALA

Faitau foi i la matou blog:

puna: www.habr.com

JUnit i GitLab CI ma Kubernetes

Folasaga

tonu

GitLab CI

Kubernetes

pa'i mulimuli

mataʻi

iʻuga

iʻuga

SALA

Faaopoopo i ai se faamatalaga faalēaogāina tali