🥇JMeteri testide käivitamine OpenShiftis, kasutades Jenkins Pipeline

Tere kõigile!

Selles artiklis tahan jagada üht võimalust JMeteri jõudlusteste käivitamiseks OpenShiftis, kasutades automatiseeritult Jenkinsi. Kõigepealt teeme kõik vajalikud sammud (loomine ImageStreams, BuildConfig, Job jne) käsitsi režiimis. Pärast seda kirjutame Jenkins Pipeline.

Lähtepunktina peaks meil olema:

töötab OpenShift (v3.11) klastris
Jenkinsi server koos konfigureeritud mandaatidega OpenShiftis töötamiseks
faili apache-jmeter-5.2.tgz

Testidena saab see olema lihtne HTTP Request edasi ya.ru ühes voolus.

Projekti loomine OpenShiftis

Alustame uue keskkonna loomisega. Loome perftest ümbritsetud meeskonnast:

$ oc new-project perftest --display-name="Performance Tests" --description="Performance Tests - JMeter"

Meid viiakse automaatselt üle vastloodud keskkonda perftest, kontrollime, kas see on nii:

$ oc project
Using project "perftest" on server "https://127.0.0.1:8443".

Salvestusruumi loomine

Testiaruanded salvestatakse ühiselt veebiserveriga ja jmeter-meter'koht - /jmeter/reports.

Parem on luua Storajs kohe, sest POD-id seotakse nendega jmeter-web и jmeter-master.

Täpsemat teavet ladustamise kohta leiate ametlikust dokumentatsioonist Püsiv ladustamine.

Loome jaoks yaml-failid PV и PVC.

pv.yaml

$ tee pv.yaml<<EOF
apiVersion: v1
kind: PersistentVolume
metadata:
  name: jmeter-reports
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteMany
  glusterfs:
    endpoints: glusterfs-cluster
    path: /jmeter/reports
    readOnly: false
  persistentVolumeReclaimPolicy: Retain
EOF

pvc.yaml

$ tee pvc.yaml<<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jmeter-reports
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
EOF

Loome PV и PVC ümbritsetud OpenShiftiga:

$ oc create -f pv.yaml -n perftest
$ oc create -f pvc.yaml -n perftest

Oleku kontrollimine PVC:

$ oc get pvc -n perftest
NAME             STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS        AGE
jmeter-reports   Bound     pvc-b0e5f152-db4b-11ea-a497-566f75280024   10Gi       RWX            glusterfs-storage   8m

GUI-s näeb see välja järgmine:

JMeteri baaspildi loomine

Liigume edasi loomise juurde ImageStream и BuildConfig.

Kogu vajaliku teabe leiate dokumentatsioonist - Ehitised ja pildivood.

Pildi loomise strateegia on Docker kohalikust allikast.

Loome aluspildi jmeter-base, mis on aluseks jmeter-master.

dockerfile

FROM openjdk:8u212-jdk

ARG JMETER_VER="5.2"
ENV JMETER_HOME /jmeter/apache-jmeter-$JMETER_VER
ENV PATH $JMETER_HOME/bin:$PATH

RUN mkdir -p /jmeter/results 
    && mkdir /jmeter/tests

WORKDIR /jmeter

COPY apache-jmeter-$JMETER_VER.tgz .

RUN tar -xzf $JMETER_HOME.tgz 
    && rm $JMETER_HOME.tgz 
    && ls -la

RUN sed -i s/#server.rmi.ssl.disable=false/server.rmi.ssl.disable=true/ $JMETER_HOME/bin/jmeter.properties

EXPOSE 60000

is.yaml

$ tee is.yaml<<EOF
apiVersion: v1
kind: ImageStream
metadata:
  labels:
    build: jmeter-base
  name: jmeter-base
EOF

bc.yaml

$ tee bc.yaml<<EOF
apiVersion: v1
kind: BuildConfig
metadata:
  name: jmeter-base
spec:
  failedBuildsHistoryLimit: 5
  nodeSelector: null
  output:
    to:
      kind: ImageStreamTag
      name: 'jmeter-base:latest'
  postCommit: {}
  resources: {}
  runPolicy: Serial
  source:
    binary: {}
    type: Binary
  strategy:
    dockerStrategy:
      from:
        kind: ImageStreamTag
        name: 'openjdk:8u212-jdk'
    type: Docker
  successfulBuildsHistoryLimit: 5
EOF

Loome objekte IS и BC:

$ oc create -f is.yaml -n perftest
$ oc create -f bc.yaml -n perftest

Nüüd paneme kokku põhipildi jmeter-base:

$ oc start-build jmeter-base -n perftest --from-dir=. --follow

JMeteri VEEB

jmeter-web See on Apache veebiserver. Selle ülesandeks on anda vaatamiseks kataloog koos testitulemustega.

Valmistatud Dockerfile ja konfiguratsioonifail httpd.conf. Direktiivseks DocumentRoot väärtuse komplekt /jmeter/reports, st. kataloog, kuhu testitulemused salvestatakse.

dockerfile

$ tee Dockerfile<<EOF
FROM httpd:2.4

COPY httpd.conf /usr/local/apache2/conf/httpd.conf
RUN chmod -R 777 /usr/local/apache2/logs

EXPOSE 8080

CMD ["httpd", "-D", "FOREGROUND"]
EOF

is.yaml

$ tee is.yaml<<EOF
apiVersion: v1
kind: ImageStream
metadata:
  generation: 1
  labels:
    build: jmeter-web
  name: jmeter-web
EOF

bc.yaml

$ tee bc.yaml<<EOF
apiVersion: v1
kind: BuildConfig
metadata:
  name: jmeter-web
spec:
  failedBuildsHistoryLimit: 5
  nodeSelector: null
  output:
    to:
      kind: ImageStreamTag
      name: 'jmeter-web:latest'
  runPolicy: Serial
  source:
    binary: {}
    type: Binary
  strategy:
    dockerStrategy:
      from:
        kind: ImageStreamTag
        name: 'httpd:2.4'
    type: Docker
  successfulBuildsHistoryLimit: 5
EOF

Loome ImageStream и BuildConfig objektid:

$ oc create -f is.yaml -n perftest
$ oc create -f bc.yaml -n perftest

Pildi kogumine kohast Dockerfile:

$ oc start-build jmeter-web -n perftest --from-dir=. --follow

dc.yaml

$ tee dc.yaml<<EOF
apiVersion: apps.openshift.io/v1
kind: DeploymentConfig
metadata:
  name: jmeter-web
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: jmeter-web
    spec:
      containers:
        - image: 172.30.1.1:5000/perftest/jmeter-web
          name: jmeter-web
          volumeMounts:
            - mountPath: /jmeter/reports
              name: jmeter-reports
          ports:
            - containerPort: 80
              protocol: TCP
            - containerPort: 8080
              protocol: TCP
      volumes:
        - name: jmeter-reports
          persistentVolumeClaim:
            claimName: jmeter-reports
EOF

sc.yaml

$ tee sc.yaml<<EOF
apiVersion: v1
kind: Service
metadata:
  labels:
    app: jmeter-web
  name: jmeter-web
spec:
  ports:
    - name: 8080-tcp
      port: 8080
      protocol: TCP
      targetPort: 8080
  selector:
    deploymentconfig: jmeter-web
  sessionAffinity: None
  type: ClusterIP
EOF

Loome objekte Service и DeploymentConfig:

$ oc create -f sc.yaml -n perftest
$ oc create -f dc.yaml -n perftest

Jmeter-meister

Alustame Apache veebiserveri juurutamist.

See on Dockerfile jmeter-master'a, põhineb jmeter-base, mis käivitab testid ja salvestab tulemused salvestusruumi.

dockerfile

Dockerfile eest jmeter-master, põhineb jmeter-base.

FROM jmeter-base

ARG JMETER_VER="5.2"
ENV JMETER_HOME /jmeter/apache-jmeter-$JMETER_VER
ENV PATH $JMETER_HOME/bin:$PATH

WORKDIR /jmeter
COPY run.sh /jmeter/
COPY tests/*.jmx /jmeter/tests/
RUN chmod +x /jmeter/run.sh

ENTRYPOINT ["/bin/bash"]
CMD ["/jmeter/run.sh"]

jooksma.sh

run.sh see on skript, mis käivitab JMeteri ja salvestab tulemused kataloogi files.

Iga kord, kui skript käivitatakse, kustutab see eelmised testid, nii et saate töötada ainult uusimate andmetega. Kuid see pole probleem, sest saate seda vastavalt oma vajadustele muuta.

#!/bin/bash

set -e

if [ -d "/jmeter/reports/files" ]
then
    echo "Directory /jmeter/reports/files exist - OK"
else
    echo "Creating /jmeter/reports/files directory"
    mkdir /jmeter/reports/files
fi

if [ -d "/jmeter/reports/dashboards" ]
then
    echo "Directory /jmeter/reports/dashboards exist"
else
    echo "Creating /jmeter/reports/dashboards directory"
    mkdir /jmeter/reports/dashboards
fi

echo "*** JMeter START Tests ***"

for item in $(ls -1 /jmeter/tests | grep jmx)
do
    echo "*** Removing dashboard directory for $item"
    rm -rdf /jmeter/reports/dashboards/${item}*

    echo "*** Removing tests directory for $item"
    rm -rdf /jmeter/reports/files/${item}*

    echo "*** Testing a $item file ***"
    jmeter -n -t /jmeter/tests/${item} -l /jmeter/reports/files/${item}-report.jtl -e -o /jmeter/reports/dashboards/${item}-dash
done

is.yaml

$ tee is.yaml<<EOF
apiVersion: image.openshift.io/v1
kind: ImageStream
metadata:
  generation: 1
  labels:
    build: jmeter-master
  name: jmeter-master
EOF

bc.yaml

$ tee bc.yaml<<EOF
apiVersion: build.openshift.io/v1
kind: BuildConfig
metadata:
  name: jmeter-master
spec:
  failedBuildsHistoryLimit: 5
  nodeSelector: null
  output:
    to:
      kind: ImageStreamTag
      name: 'jmeter-master:latest'
  runPolicy: Serial
  source:
    binary: {}
    type: Binary
  strategy:
    dockerStrategy:
      from:
        kind: ImageStreamTag
        name: 'jmeter-base:latest'
    type: Docker
  successfulBuildsHistoryLimit: 5
EOF

Loome IS и BC objektid:

$ oc create -f is.yaml -n perftest
$ oc create -f bc.yaml -n perftest

Kogume jmeter-master pilt:

$ oc start-build jmeter-master -n perftest --from-dir=. --follow

töö

Jobon kasutusel OpenShift'e selleks, et käivitada üks või mitu PODja tagavad nende eduka lõpuleviimise pärast käsu/skripti täitmist.

$ tee job.yaml<<EOF
apiVersion: batch/v1
kind: Job
metadata:
  name: jmeter-master
  labels:
    jobName: jmeter-master
spec:
  completions: 1
  parallelism: 1
  template:
    metadata:
      name: jmeter-master
      labels:
        jobName: jmeter-master
    spec:
      containers:
        - name: jmeter-master
          image: 172.30.1.1:5000/perftest/jmeter-master:latest
          volumeMounts:
            - mountPath: /jmeter/reports
              name: jmeter-reports
          imagePullPolicy: Always
      volumes:
        - name: jmeter-reports
          persistentVolumeClaim:
            claimName: jmeter-reports
      restartPolicy: Never
      terminationGracePeriodSeconds: 30
EOF

Loo objekt Job:

$ oc create -f job.yaml -n perftest

Kontrollime töö olekut:

$ oc get jobs -n perftest
NAME            DESIRED   SUCCESSFUL   AGE
jmeter-master   1         1            5m

Kustutamiseks Job kasutame käsku:

$ oc delete jobs/jmeter-master -n perftest --ignore-not-found=true

Jenkinsi torujuhe

Nüüd automatiseerimine. Vaatame sammud uuesti üle:

git clone
oc whoami -t
oc start-build ...
oc delete jobs/jmeter-master
oc create -f job.yaml -n perftest

Allpool on konveier, milles teostatakse hoidla kloonimine, kustutamine ja OpenShift loomine Job's.

#!groovy

pipeline {

    agent any

    stages {

        stage('Start Notifications') {
            steps {
                echo "Sending Email Notification"
            }
            post {
                always {
                    echo "STARTED - Performance Tests"
                    mail(to: '[email protected]', from: "[email protected]", subject: "START - Performance Tests",mimeType: "text/html", body: "<strong>START - Performance Tests</strong><br /><br />Project: Name of Project<br />Environment: PerfTest<br />Build number: ${env.BUILD_NUMBER}<br />Build URL:   ${env.BUILD_URL}"
                }
            }
        }

        stage('Git checkout') {
            steps {
                ...
            }
        }

        stage('Perf Tests') {
            steps {
                script {
                    sh '''
                        OC_CMD1="oc login -u=username -p=PASS -n=perftest 
                        --server=https://...:8443"

                        $OC_CMD1

                        OC_TOKEN=`oc whoami -t`

                        OC_CMD2="oc --token=$OC_TOKEN --server=https://...:8443 
                        start-build jmeter-master -n=perftest --from-dir=./master 
                        --follow=true"

                        OC_CMD3="oc --token=$OC_TOKEN --server=https://...:8443 
                        delete jobs/jmeter-master -n=perftest --ignore-not-found=true"

                        OC_CMD4="oc--token=$OC_TOKEN --server=https://...:8443 
                        create -f ./master/job.yaml -n=perftest"

                        $OC_CMD2
                        $OC_CMD3
                        $OC_CMD4
                    '''
                }
            }
        }

        post {
            failure {
                echo "FAILED - Performance Tests"
                mail(to: '[email protected]', from: "[email protected]", subject: "FAILED - Performance Tests",mimeType: "text/html", body: "<strong>FAILED - Performance Tests</strong><br /><br />Project: Name of Project<br />Environment: PerfTest<br />Build number: ${env.BUILD_NUMBER}<br />Build URL: ${env.BUILD_URL}"
                }

            success {
                echo "SUCCESSED - Performance Tests"
                mail(to: '[email protected]', from: "[email protected]", subject: "SUCCESSED - Performance Tests",mimeType: "text/html", body: "<strong>SUCCESSED - Performance Tests</strong><br /><br />Project: Name of Project<br />Environment: PerfTest<br />Build number: ${env.BUILD_NUMBER}<br />Build URL:   ${env.BUILD_URL}"
            }
        }

    }
}

Pärast seda, kui Pipeline on oma töö lõpetanud, saame e-posti teel teate '[email protected] pärit [email protected].

Lingil klõpsates http://jmeter-web.127.0.0.1.nip.io/ me näeme kataloogi files, mis salvestab testiaruanded:

Faili sisu ya.HTTP.Request.jmx-report.jtk:

timeStamp,elapsed,label,responseCode,responseMessage,threadName,dataType,success,failureMessage,bytes,sentBytes,grpThreads,allThreads,URL,Latency,IdleTime,Connect
1597311456443,569,Yandex - HTTP Request,200,Ok,Thread Group 1-1,text,true,,59449,220,1,1,https://ya.ru/,145,0,57
1597311456443,147,Yandex - HTTP Request-0,302,Found,Thread Group 1-1,,true,,478,110,1,1,http://ya.ru/,145,0,57
1597311456592,420,Yandex - HTTP Request-1,200,Ok,Thread Group 1-1,text,true,,58971,110,1,1,https://ya.ru/,370,0,259

Järeldus

See artikkel tutvustas üht võimalust JMeteri testide käitamiseks OpenShift keskkonnas. Tegime kõik sammud käsitsi läbi, mille järel lõime Jenkinsi torujuhtme, et automatiseerida testide käitamise protsessi.

Allikad ja dokumentatsioon

Allikas: www.habr.com

JMeteri testide käivitamine OpenShiftis, kasutades Jenkins Pipeline'i

Projekti loomine OpenShiftis

Salvestusruumi loomine

JMeteri baaspildi loomine

JMeteri VEEB

Jmeter-meister

Jenkinsi torujuhe

Järeldus

Allikad ja dokumentatsioon

Lisa kommentaar Tühista vastus