ProHoster > Blog > Bestjoer > JMeter-tests útfiere yn OpenShift mei Jenkins Pipeline

JMeter-tests útfiere yn OpenShift mei Jenkins Pipeline

Hello everyone!

Yn dit artikel wol ik ien fan 'e manieren diele om JMeter-prestaasjestests yn OpenShift út te fieren mei Jenkins as automatisearring. Earst sille wy alle nedige stappen dwaan (meitsje ImageStreams, BuildConfig, Job ensfh.) yn hânmjittich modus. Lit ús dêrnei Jenkins Pipeline skriuwe.

As útgongspunt moatte wy hawwe:

  1. running OpenShift (v3.11) kluster
  2. Jenkins-tsjinner mei ynstelde referinsjes om te wurkjen yn OpenShift
  3. file apache-jmeter-5.2.tgz

As tests sil it ienfâldich wêze HTTP Request op ya.ru yn ien stream.

It meitsjen fan in projekt yn OpenShift

Litte wy begjinne mei it meitsjen fan in nije omjouwing. Lit ús meitsje perftest omjûn troch in team:

$ oc new-project perftest --display-name="Performance Tests" --description="Performance Tests - JMeter"

Wy wurde automatysk oerbrocht nei de nij oanmakke omjouwing perftest, lit ús kontrolearje dat dit sa is:

$ oc project
Using project "perftest" on server "https://127.0.0.1:8443".

Skepping fan Storage

Test rapporten sille wurde opslein mienskiplik mei de web tsjinner en jmeter-meter'in plak - /jmeter/reports.

It is no better om Storajs te meitsjen, om't POD's deroan wurde bûn jmeter-web и jmeter-master.

Jo sille mear detaillearre ynformaasje fine oer opslach yn 'e offisjele dokumintaasje Persistente opslach.

Litte wy yaml-bestannen meitsje foar PV и PVC.

pv.yaml

$ tee pv.yaml<<EOF
apiVersion: v1
kind: PersistentVolume
metadata:
  name: jmeter-reports
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteMany
  glusterfs:
    endpoints: glusterfs-cluster
    path: /jmeter/reports
    readOnly: false
  persistentVolumeReclaimPolicy: Retain
EOF

pvc.yaml

$ tee pvc.yaml<<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jmeter-reports
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
EOF

Lit ús meitsje PV и PVC omjûn troch OpenShift:

$ oc create -f pv.yaml -n perftest
$ oc create -f pvc.yaml -n perftest

Kontrolearje de status foar PVC:

$ oc get pvc -n perftest
NAME             STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS        AGE
jmeter-reports   Bound     pvc-b0e5f152-db4b-11ea-a497-566f75280024   10Gi       RWX            glusterfs-storage   8m

Hjir is hoe't it der útsjen sil yn 'e GUI:

JMeter-tests útfiere yn OpenShift mei Jenkins Pipeline

It meitsjen fan in JMeter Base Image

Litte wy trochgean mei it meitsjen ImageStream и BuildConfig.

Jo kinne alle nedige ynformaasje fine yn 'e dokumintaasje - Builds en Image Streams.

De byldboustrategy is Docker út in lokale boarne.

Litte wy in basisôfbylding meitsje jmeter-base, dêr't de basis foar wêze sil jmeter-master.

dockerfile

FROM openjdk:8u212-jdk

ARG JMETER_VER="5.2"
ENV JMETER_HOME /jmeter/apache-jmeter-$JMETER_VER
ENV PATH $JMETER_HOME/bin:$PATH

RUN mkdir -p /jmeter/results 
    && mkdir /jmeter/tests

WORKDIR /jmeter

COPY apache-jmeter-$JMETER_VER.tgz .

RUN tar -xzf $JMETER_HOME.tgz 
    && rm $JMETER_HOME.tgz 
    && ls -la

RUN sed -i s/#server.rmi.ssl.disable=false/server.rmi.ssl.disable=true/ $JMETER_HOME/bin/jmeter.properties

EXPOSE 60000

is.yaml

$ tee is.yaml<<EOF
apiVersion: v1
kind: ImageStream
metadata:
  labels:
    build: jmeter-base
  name: jmeter-base
EOF

bc.yaml

$ tee bc.yaml<<EOF
apiVersion: v1
kind: BuildConfig
metadata:
  name: jmeter-base
spec:
  failedBuildsHistoryLimit: 5
  nodeSelector: null
  output:
    to:
      kind: ImageStreamTag
      name: 'jmeter-base:latest'
  postCommit: {}
  resources: {}
  runPolicy: Serial
  source:
    binary: {}
    type: Binary
  strategy:
    dockerStrategy:
      from:
        kind: ImageStreamTag
        name: 'openjdk:8u212-jdk'
    type: Docker
  successfulBuildsHistoryLimit: 5
EOF

Litte wy objekten meitsje IS и BC:

$ oc create -f is.yaml -n perftest
$ oc create -f bc.yaml -n perftest

Litte wy no de basisôfbylding gearstalle jmeter-base:

$ oc start-build jmeter-base -n perftest --from-dir=. --follow

JMeter WEB

jmeter-web Dit is in Apache-webserver. Syn taak is om in map te leverjen mei testresultaten foar besjen.

Tariede Dockerfile en konfiguraasjetriem httpd.conf. Foar rjochtline DocumentRoot wearde set /jmeter/reports, d.w.s. de map wêryn testresultaten wurde opslein.

dockerfile

$ tee Dockerfile<<EOF
FROM httpd:2.4

COPY httpd.conf /usr/local/apache2/conf/httpd.conf
RUN chmod -R 777 /usr/local/apache2/logs

EXPOSE 8080

CMD ["httpd", "-D", "FOREGROUND"]
EOF

is.yaml

$ tee is.yaml<<EOF
apiVersion: v1
kind: ImageStream
metadata:
  generation: 1
  labels:
    build: jmeter-web
  name: jmeter-web
EOF

bc.yaml

$ tee bc.yaml<<EOF
apiVersion: v1
kind: BuildConfig
metadata:
  name: jmeter-web
spec:
  failedBuildsHistoryLimit: 5
  nodeSelector: null
  output:
    to:
      kind: ImageStreamTag
      name: 'jmeter-web:latest'
  runPolicy: Serial
  source:
    binary: {}
    type: Binary
  strategy:
    dockerStrategy:
      from:
        kind: ImageStreamTag
        name: 'httpd:2.4'
    type: Docker
  successfulBuildsHistoryLimit: 5
EOF

Lit ús meitsje ImageStream и BuildConfig objekten:

$ oc create -f is.yaml -n perftest
$ oc create -f bc.yaml -n perftest

Sammelje in ôfbylding fan Dockerfile:

$ oc start-build jmeter-web -n perftest --from-dir=. --follow

dc.yaml

$ tee dc.yaml<<EOF
apiVersion: apps.openshift.io/v1
kind: DeploymentConfig
metadata:
  name: jmeter-web
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: jmeter-web
    spec:
      containers:
        - image: 172.30.1.1:5000/perftest/jmeter-web
          name: jmeter-web
          volumeMounts:
            - mountPath: /jmeter/reports
              name: jmeter-reports
          ports:
            - containerPort: 80
              protocol: TCP
            - containerPort: 8080
              protocol: TCP
      volumes:
        - name: jmeter-reports
          persistentVolumeClaim:
            claimName: jmeter-reports
EOF

sc.yaml

$ tee sc.yaml<<EOF
apiVersion: v1
kind: Service
metadata:
  labels:
    app: jmeter-web
  name: jmeter-web
spec:
  ports:
    - name: 8080-tcp
      port: 8080
      protocol: TCP
      targetPort: 8080
  selector:
    deploymentconfig: jmeter-web
  sessionAffinity: None
  type: ClusterIP
EOF

Litte wy objekten meitsje Service и DeploymentConfig:

$ oc create -f sc.yaml -n perftest
$ oc create -f dc.yaml -n perftest

Jmeter-master

Litte wy begjinne mei it ynsetten fan de Apache-webserver.

Dit is de Dockerfile jmeter-master'a, basearre op jmeter-base, dy't tests útfiere en de resultaten opslaan yn opslach.

dockerfile

Dockerfile foar jmeter-master, basearre op jmeter-base.

FROM jmeter-base

ARG JMETER_VER="5.2"
ENV JMETER_HOME /jmeter/apache-jmeter-$JMETER_VER
ENV PATH $JMETER_HOME/bin:$PATH

WORKDIR /jmeter
COPY run.sh /jmeter/
COPY tests/*.jmx /jmeter/tests/
RUN chmod +x /jmeter/run.sh

ENTRYPOINT ["/bin/bash"]
CMD ["/jmeter/run.sh"]

run.sh

run.sh dit is in skript dat rint JMeter en bewarret de resultaten yn in map files.

Elke kear as it skript wurdt lansearre, wisket it eardere tests, sadat jo allinich wurkje kinne mei de lêste gegevens. Mar dit is gjin probleem, om't jo it kinne feroarje nei jo behoeften.

#!/bin/bash

set -e

if [ -d "/jmeter/reports/files" ]
then
    echo "Directory /jmeter/reports/files exist - OK"
else
    echo "Creating /jmeter/reports/files directory"
    mkdir /jmeter/reports/files
fi

if [ -d "/jmeter/reports/dashboards" ]
then
    echo "Directory /jmeter/reports/dashboards exist"
else
    echo "Creating /jmeter/reports/dashboards directory"
    mkdir /jmeter/reports/dashboards
fi

echo "*** JMeter START Tests ***"

for item in $(ls -1 /jmeter/tests | grep jmx)
do
    echo "*** Removing dashboard directory for $item"
    rm -rdf /jmeter/reports/dashboards/${item}*

    echo "*** Removing tests directory for $item"
    rm -rdf /jmeter/reports/files/${item}*

    echo "*** Testing a $item file ***"
    jmeter -n -t /jmeter/tests/${item} -l /jmeter/reports/files/${item}-report.jtl -e -o /jmeter/reports/dashboards/${item}-dash
done

is.yaml

$ tee is.yaml<<EOF
apiVersion: image.openshift.io/v1
kind: ImageStream
metadata:
  generation: 1
  labels:
    build: jmeter-master
  name: jmeter-master
EOF

bc.yaml

$ tee bc.yaml<<EOF
apiVersion: build.openshift.io/v1
kind: BuildConfig
metadata:
  name: jmeter-master
spec:
  failedBuildsHistoryLimit: 5
  nodeSelector: null
  output:
    to:
      kind: ImageStreamTag
      name: 'jmeter-master:latest'
  runPolicy: Serial
  source:
    binary: {}
    type: Binary
  strategy:
    dockerStrategy:
      from:
        kind: ImageStreamTag
        name: 'jmeter-base:latest'
    type: Docker
  successfulBuildsHistoryLimit: 5
EOF

Lit ús meitsje IS и BC objekten:

$ oc create -f is.yaml -n perftest
$ oc create -f bc.yaml -n perftest

Wy sammelje jmeter-master byld:

$ oc start-build jmeter-master -n perftest --from-dir=. --follow

Taak

Job's wurde brûkt yn OpenShift'e om ien of mear te rinnen POD's en garandearje harren suksesfolle foltôging nei it útfieren fan it kommando/skript.

$ tee job.yaml<<EOF
apiVersion: batch/v1
kind: Job
metadata:
  name: jmeter-master
  labels:
    jobName: jmeter-master
spec:
  completions: 1
  parallelism: 1
  template:
    metadata:
      name: jmeter-master
      labels:
        jobName: jmeter-master
    spec:
      containers:
        - name: jmeter-master
          image: 172.30.1.1:5000/perftest/jmeter-master:latest
          volumeMounts:
            - mountPath: /jmeter/reports
              name: jmeter-reports
          imagePullPolicy: Always
      volumes:
        - name: jmeter-reports
          persistentVolumeClaim:
            claimName: jmeter-reports
      restartPolicy: Never
      terminationGracePeriodSeconds: 30
EOF

Meitsje in foarwerp Job:

$ oc create -f job.yaml -n perftest

Litte wy de status fan 'e baan kontrolearje:

$ oc get jobs -n perftest
NAME            DESIRED   SUCCESSFUL   AGE
jmeter-master   1         1            5m

Om te wiskjen Job lit ús it kommando brûke:

$ oc delete jobs/jmeter-master -n perftest --ignore-not-found=true

Jenkins Pipeline

No automatisearring. Litte wy de stappen nochris oergean:

  1. git clone
  2. oc whoami -t
  3. oc start-build ...
  4. oc delete jobs/jmeter-master
  5. oc create -f job.yaml -n perftest

Hjirûnder is in pipeline wêryn repository klonen, wiskjen en OpenShift oanmeitsjen wurde útfierd Job's.

#!groovy

pipeline {

    agent any

    stages {

        stage('Start Notifications') {
            steps {
                echo "Sending Email Notification"
            }
            post {
                always {
                    echo "STARTED - Performance Tests"
                    mail(to: 'username@srv.net', from: "jenkins@srv.net", subject: "START - Performance Tests",mimeType: "text/html", body: "<strong>START - Performance Tests</strong><br /><br />Project: Name of Project<br />Environment: PerfTest<br />Build number: ${env.BUILD_NUMBER}<br />Build URL:   ${env.BUILD_URL}"
                }
            }
        }

        stage('Git checkout') {
            steps {
                ...
            }
        }

        stage('Perf Tests') {
            steps {
                script {
                    sh '''
                        OC_CMD1="oc login -u=username -p=PASS -n=perftest 
                        --server=https://...:8443"

                        $OC_CMD1

                        OC_TOKEN=`oc whoami -t`

                        OC_CMD2="oc --token=$OC_TOKEN --server=https://...:8443 
                        start-build jmeter-master -n=perftest --from-dir=./master 
                        --follow=true"

                        OC_CMD3="oc --token=$OC_TOKEN --server=https://...:8443 
                        delete jobs/jmeter-master -n=perftest --ignore-not-found=true"

                        OC_CMD4="oc--token=$OC_TOKEN --server=https://...:8443 
                        create -f ./master/job.yaml -n=perftest"

                        $OC_CMD2
                        $OC_CMD3
                        $OC_CMD4
                    '''
                }
            }
        }

        post {
            failure {
                echo "FAILED - Performance Tests"
                mail(to: 'username@srv.net', from: "jenkins@srv.net", subject: "FAILED - Performance Tests",mimeType: "text/html", body: "<strong>FAILED - Performance Tests</strong><br /><br />Project: Name of Project<br />Environment: PerfTest<br />Build number: ${env.BUILD_NUMBER}<br />Build URL: ${env.BUILD_URL}"
                }

            success {
                echo "SUCCESSED - Performance Tests"
                mail(to: 'username@srv.net', from: "jenkins@srv.net", subject: "SUCCESSED - Performance Tests",mimeType: "text/html", body: "<strong>SUCCESSED - Performance Tests</strong><br /><br />Project: Name of Project<br />Environment: PerfTest<br />Build number: ${env.BUILD_NUMBER}<br />Build URL:   ${env.BUILD_URL}"
            }
        }

    }
}

Nei't Pipeline syn operaasje foltôge hat, krije wy in notifikaasje per e-post 'username@srv.net от jenkins@srv.net.

Troch de link te folgjen http://jmeter-web.127.0.0.1.nip.io/ wy sille de map sjen files, dy't testrapporten opslaat:

JMeter-tests útfiere yn OpenShift mei Jenkins Pipeline

Triem ynhâld ya.HTTP.Request.jmx-report.jtk:

timeStamp,elapsed,label,responseCode,responseMessage,threadName,dataType,success,failureMessage,bytes,sentBytes,grpThreads,allThreads,URL,Latency,IdleTime,Connect
1597311456443,569,Yandex - HTTP Request,200,Ok,Thread Group 1-1,text,true,,59449,220,1,1,https://ya.ru/,145,0,57
1597311456443,147,Yandex - HTTP Request-0,302,Found,Thread Group 1-1,,true,,478,110,1,1,http://ya.ru/,145,0,57
1597311456592,420,Yandex - HTTP Request-1,200,Ok,Thread Group 1-1,text,true,,58971,110,1,1,https://ya.ru/,370,0,259

konklúzje

Dit artikel toande ien fan 'e opsjes foar it útfieren fan JMeter-tests yn' e OpenShift-omjouwing. Wy hawwe alle stappen mei de hân foltôge, wêrnei't wy Jenkins Pipeline hawwe makke om it proses fan it útfieren fan tests te automatisearjen.

Boarnen en dokumintaasje

Boarne: www.habr.com