Khiav JMeter xeem hauv OpenShift siv Jenkins Pipeline

Nyob zoo txhua leej txhua tus!

Hauv tsab xov xwm no kuv xav qhia ib txoj hauv kev los khiav JMeter kev xeem ua haujlwm hauv OpenShift siv Jenkins ua automation. Ua ntej peb yuav ua tag nrho cov kauj ruam tsim nyog (tsim ImageStreams, BuildConfig, Job etc.) nyob rau hauv phau ntawv hom. Tom qab ntawd, cia peb sau Jenkins Pipeline.

Raws li qhov pib taw tes peb yuav tsum muaj:

  1. khiav OpenShift (v3.11) pawg
  2. Jenkins server nrog cov ntawv pov thawj teeb tsa los ua haujlwm hauv OpenShift
  3. cov ntaub ntawv apache-jmeter-5.2.tgz

Raws li kev ntsuam xyuas nws yuav yooj yim HTTP Request rau ya.ru hauv ib lub kwj dej.

Tsim ib qhov project hauv OpenShift

Cia peb pib los ntawm kev tsim ib puag ncig tshiab. Wb tsim perftest ncig los ntawm ib pab neeg:

$ oc new-project perftest --display-name="Performance Tests" --description="Performance Tests - JMeter"

Peb yuav cia li hloov mus rau qhov chaw tsim tshiab perftest, cia peb xyuas seb qhov no yog li cas:

$ oc project
Using project "perftest" on server "https://127.0.0.1:8443".

Tsim Kev Cia

Cov ntaub ntawv xeem yuav muab khaws cia ua ke nrog lub vev xaib server thiab jmeter-meter'ib- /jmeter/reports.

Nws yog qhov zoo dua los tsim Storajs tam sim no, vim tias PODs yuav raug khi rau lawv jmeter-web ΠΈ jmeter-master.

Koj yuav pom cov ntaub ntawv ntxaws ntxiv ntawm kev khaws cia hauv cov ntaub ntawv raug cai Kev khaws cia tsis tu ncua.

Wb tsim yaml cov ntaub ntawv rau PV ΠΈ PVC.

pv yam

$ tee pv.yaml<<EOF
apiVersion: v1
kind: PersistentVolume
metadata:
  name: jmeter-reports
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteMany
  glusterfs:
    endpoints: glusterfs-cluster
    path: /jmeter/reports
    readOnly: false
  persistentVolumeReclaimPolicy: Retain
EOF

pvc yam

$ tee pvc.yaml<<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jmeter-reports
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
EOF

Wb tsim PV ΠΈ PVC nyob ib puag ncig ntawm OpenShift:

$ oc create -f pv.yaml -n perftest
$ oc create -f pvc.yaml -n perftest

Tshawb xyuas cov xwm txheej rau PVC:

$ oc get pvc -n perftest
NAME             STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS        AGE
jmeter-reports   Bound     pvc-b0e5f152-db4b-11ea-a497-566f75280024   10Gi       RWX            glusterfs-storage   8m

Nov yog qhov nws yuav zoo li hauv GUI:

Khiav JMeter xeem hauv OpenShift siv Jenkins Pipeline

Tsim ib daim duab JMeter Base

Wb mus rau kev tsim ImageStream ΠΈ BuildConfig.

Koj tuaj yeem pom tag nrho cov ntaub ntawv tsim nyog hauv cov ntaub ntawv - Tsim thiab duab kwj.

Cov duab tsim tswv yim yog Docker los ntawm ib qho chaw hauv zos.

Cia peb tsim ib lub hauv paus duab jmeter-base, uas yuav yog lub hauv paus rau jmeter-master.

Dockerfile

FROM openjdk:8u212-jdk

ARG JMETER_VER="5.2"
ENV JMETER_HOME /jmeter/apache-jmeter-$JMETER_VER
ENV PATH $JMETER_HOME/bin:$PATH

RUN mkdir -p /jmeter/results 
    && mkdir /jmeter/tests

WORKDIR /jmeter

COPY apache-jmeter-$JMETER_VER.tgz .

RUN tar -xzf $JMETER_HOME.tgz 
    && rm $JMETER_HOME.tgz 
    && ls -la

RUN sed -i s/#server.rmi.ssl.disable=false/server.rmi.ssl.disable=true/ $JMETER_HOME/bin/jmeter.properties

EXPOSE 60000

yam

$ tee is.yaml<<EOF
apiVersion: v1
kind: ImageStream
metadata:
  labels:
    build: jmeter-base
  name: jmeter-base
EOF

bc yam

$ tee bc.yaml<<EOF
apiVersion: v1
kind: BuildConfig
metadata:
  name: jmeter-base
spec:
  failedBuildsHistoryLimit: 5
  nodeSelector: null
  output:
    to:
      kind: ImageStreamTag
      name: 'jmeter-base:latest'
  postCommit: {}
  resources: {}
  runPolicy: Serial
  source:
    binary: {}
    type: Binary
  strategy:
    dockerStrategy:
      from:
        kind: ImageStreamTag
        name: 'openjdk:8u212-jdk'
    type: Docker
  successfulBuildsHistoryLimit: 5
EOF

Cia peb tsim cov khoom IS ΠΈ BC:

$ oc create -f is.yaml -n perftest
$ oc create -f bc.yaml -n perftest

Tam sim no cia peb sib sau ua ke lub hauv paus duab jmeter-base:

$ oc start-build jmeter-base -n perftest --from-dir=. --follow

JMeter WEB

jmeter-web Nov yog Apache web server. Nws txoj hauj lwm yog los muab ib daim ntawv teev cov txiaj ntsig rau kev saib.

Npaj Dockerfile thiab configuration file httpd.conf. Rau cov lus qhia DocumentRoot tus nqi teev /jmeter/reports, i.e. cov directory nyob rau hauv qhov kev xeem tau txais kev cawmdim.

Dockerfile

$ tee Dockerfile<<EOF
FROM httpd:2.4

COPY httpd.conf /usr/local/apache2/conf/httpd.conf
RUN chmod -R 777 /usr/local/apache2/logs

EXPOSE 8080

CMD ["httpd", "-D", "FOREGROUND"]
EOF

yam

$ tee is.yaml<<EOF
apiVersion: v1
kind: ImageStream
metadata:
  generation: 1
  labels:
    build: jmeter-web
  name: jmeter-web
EOF

bc yam

$ tee bc.yaml<<EOF
apiVersion: v1
kind: BuildConfig
metadata:
  name: jmeter-web
spec:
  failedBuildsHistoryLimit: 5
  nodeSelector: null
  output:
    to:
      kind: ImageStreamTag
      name: 'jmeter-web:latest'
  runPolicy: Serial
  source:
    binary: {}
    type: Binary
  strategy:
    dockerStrategy:
      from:
        kind: ImageStreamTag
        name: 'httpd:2.4'
    type: Docker
  successfulBuildsHistoryLimit: 5
EOF

Wb tsim ImageStream ΠΈ BuildConfig khoom:

$ oc create -f is.yaml -n perftest
$ oc create -f bc.yaml -n perftest

Sau ib daim duab los ntawm Dockerfile:

$ oc start-build jmeter-web -n perftest --from-dir=. --follow

dc yam

$ tee dc.yaml<<EOF
apiVersion: apps.openshift.io/v1
kind: DeploymentConfig
metadata:
  name: jmeter-web
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: jmeter-web
    spec:
      containers:
        - image: 172.30.1.1:5000/perftest/jmeter-web
          name: jmeter-web
          volumeMounts:
            - mountPath: /jmeter/reports
              name: jmeter-reports
          ports:
            - containerPort: 80
              protocol: TCP
            - containerPort: 8080
              protocol: TCP
      volumes:
        - name: jmeter-reports
          persistentVolumeClaim:
            claimName: jmeter-reports
EOF

sc. yam

$ tee sc.yaml<<EOF
apiVersion: v1
kind: Service
metadata:
  labels:
    app: jmeter-web
  name: jmeter-web
spec:
  ports:
    - name: 8080-tcp
      port: 8080
      protocol: TCP
      targetPort: 8080
  selector:
    deploymentconfig: jmeter-web
  sessionAffinity: None
  type: ClusterIP
EOF

Cia peb tsim cov khoom Service ΠΈ DeploymentConfig:

$ oc create -f sc.yaml -n perftest
$ oc create -f dc.yaml -n perftest

Jmeter-tus tswv

Cia peb pib siv Apache web server.

Qhov no yog Dockerfile jmeter-master'a, raws li jmeter-base, uas yuav khiav cov kev ntsuam xyuas thiab txuag cov txiaj ntsig rau kev khaws cia.

Dockerfile

Dockerfile rau jmeter-master, raws li jmeter-base.

FROM jmeter-base

ARG JMETER_VER="5.2"
ENV JMETER_HOME /jmeter/apache-jmeter-$JMETER_VER
ENV PATH $JMETER_HOME/bin:$PATH

WORKDIR /jmeter
COPY run.sh /jmeter/
COPY tests/*.jmx /jmeter/tests/
RUN chmod +x /jmeter/run.sh

ENTRYPOINT ["/bin/bash"]
CMD ["/jmeter/run.sh"]

run.sh

run.sh qhov no yog ib tsab ntawv uas khiav JMeter thiab khaws cov txiaj ntsig rau hauv phau ntawv teev npe files.

Txhua lub sij hawm tsab ntawv raug tso tawm, nws tshem tawm cov kev xeem dhau los, yog li koj tuaj yeem ua haujlwm nrog cov ntaub ntawv tshiab. Tab sis qhov no tsis yog qhov teeb meem, vim tias koj tuaj yeem hloov nws kom haum rau koj cov kev xav tau.

#!/bin/bash

set -e

if [ -d "/jmeter/reports/files" ]
then
    echo "Directory /jmeter/reports/files exist - OK"
else
    echo "Creating /jmeter/reports/files directory"
    mkdir /jmeter/reports/files
fi

if [ -d "/jmeter/reports/dashboards" ]
then
    echo "Directory /jmeter/reports/dashboards exist"
else
    echo "Creating /jmeter/reports/dashboards directory"
    mkdir /jmeter/reports/dashboards
fi

echo "*** JMeter START Tests ***"

for item in $(ls -1 /jmeter/tests | grep jmx)
do
    echo "*** Removing dashboard directory for $item"
    rm -rdf /jmeter/reports/dashboards/${item}*

    echo "*** Removing tests directory for $item"
    rm -rdf /jmeter/reports/files/${item}*

    echo "*** Testing a $item file ***"
    jmeter -n -t /jmeter/tests/${item} -l /jmeter/reports/files/${item}-report.jtl -e -o /jmeter/reports/dashboards/${item}-dash
done

yam

$ tee is.yaml<<EOF
apiVersion: image.openshift.io/v1
kind: ImageStream
metadata:
  generation: 1
  labels:
    build: jmeter-master
  name: jmeter-master
EOF

bc yam

$ tee bc.yaml<<EOF
apiVersion: build.openshift.io/v1
kind: BuildConfig
metadata:
  name: jmeter-master
spec:
  failedBuildsHistoryLimit: 5
  nodeSelector: null
  output:
    to:
      kind: ImageStreamTag
      name: 'jmeter-master:latest'
  runPolicy: Serial
  source:
    binary: {}
    type: Binary
  strategy:
    dockerStrategy:
      from:
        kind: ImageStreamTag
        name: 'jmeter-base:latest'
    type: Docker
  successfulBuildsHistoryLimit: 5
EOF

Wb tsim IS ΠΈ BC khoom:

$ oc create -f is.yaml -n perftest
$ oc create -f bc.yaml -n perftest

Peb sau jmeter-master duab:

$ oc start-build jmeter-master -n perftest --from-dir=. --follow

txoj hauj lwm

Job's yog siv rau hauv OpenShift'e nyob rau hauv thiaj li yuav khiav ib los yog ntau tshaj POD's thiab lav lawv qhov ua tiav tiav tom qab ua tiav cov lus txib / ntawv.

$ tee job.yaml<<EOF
apiVersion: batch/v1
kind: Job
metadata:
  name: jmeter-master
  labels:
    jobName: jmeter-master
spec:
  completions: 1
  parallelism: 1
  template:
    metadata:
      name: jmeter-master
      labels:
        jobName: jmeter-master
    spec:
      containers:
        - name: jmeter-master
          image: 172.30.1.1:5000/perftest/jmeter-master:latest
          volumeMounts:
            - mountPath: /jmeter/reports
              name: jmeter-reports
          imagePullPolicy: Always
      volumes:
        - name: jmeter-reports
          persistentVolumeClaim:
            claimName: jmeter-reports
      restartPolicy: Never
      terminationGracePeriodSeconds: 30
EOF

Tsim ib yam khoom Job:

$ oc create -f job.yaml -n perftest

Cia peb tshawb xyuas qhov xwm txheej ntawm txoj haujlwm:

$ oc get jobs -n perftest
NAME            DESIRED   SUCCESSFUL   AGE
jmeter-master   1         1            5m

Rho tawm Job cia peb siv cov lus txib:

$ oc delete jobs/jmeter-master -n perftest --ignore-not-found=true

Jenkins Pipeline

Tam sim no automation. Cia wb mus hla cov kauj ruam dua:

  1. git clone
  2. oc whoami -t
  3. oc start-build ...
  4. oc delete jobs/jmeter-master
  5. oc create -f job.yaml -n perftest

Hauv qab no yog cov raj xa dej hauv qhov chaw khaws cia cloning, tshem tawm thiab tsim OpenShift tau ua Job's.

#!groovy

pipeline {

    agent any

    stages {

        stage('Start Notifications') {
            steps {
                echo "Sending Email Notification"
            }
            post {
                always {
                    echo "STARTED - Performance Tests"
                    mail(to: '[email protected]', from: "[email protected]", subject: "START - Performance Tests",mimeType: "text/html", body: "<strong>START - Performance Tests</strong><br /><br />Project: Name of Project<br />Environment: PerfTest<br />Build number: ${env.BUILD_NUMBER}<br />Build URL:   ${env.BUILD_URL}"
                }
            }
        }

        stage('Git checkout') {
            steps {
                ...
            }
        }

        stage('Perf Tests') {
            steps {
                script {
                    sh '''
                        OC_CMD1="oc login -u=username -p=PASS -n=perftest 
                        --server=https://...:8443"

                        $OC_CMD1

                        OC_TOKEN=`oc whoami -t`

                        OC_CMD2="oc --token=$OC_TOKEN --server=https://...:8443 
                        start-build jmeter-master -n=perftest --from-dir=./master 
                        --follow=true"

                        OC_CMD3="oc --token=$OC_TOKEN --server=https://...:8443 
                        delete jobs/jmeter-master -n=perftest --ignore-not-found=true"

                        OC_CMD4="oc--token=$OC_TOKEN --server=https://...:8443 
                        create -f ./master/job.yaml -n=perftest"

                        $OC_CMD2
                        $OC_CMD3
                        $OC_CMD4
                    '''
                }
            }
        }

        post {
            failure {
                echo "FAILED - Performance Tests"
                mail(to: '[email protected]', from: "[email protected]", subject: "FAILED - Performance Tests",mimeType: "text/html", body: "<strong>FAILED - Performance Tests</strong><br /><br />Project: Name of Project<br />Environment: PerfTest<br />Build number: ${env.BUILD_NUMBER}<br />Build URL: ${env.BUILD_URL}"
                }

            success {
                echo "SUCCESSED - Performance Tests"
                mail(to: '[email protected]', from: "[email protected]", subject: "SUCCESSED - Performance Tests",mimeType: "text/html", body: "<strong>SUCCESSED - Performance Tests</strong><br /><br />Project: Name of Project<br />Environment: PerfTest<br />Build number: ${env.BUILD_NUMBER}<br />Build URL:   ${env.BUILD_URL}"
            }
        }

    }
}

Tom qab Pipeline tau ua tiav nws txoj haujlwm, peb yuav tau txais kev ceeb toom los ntawm email '[email protected] los ntawm [email protected].

Nyem rau ntawm qhov txuas http://jmeter-web.127.0.0.1.nip.io/ peb mam li pom cov directory files, uas khaws cov ntaub ntawv xeem:

Khiav JMeter xeem hauv OpenShift siv Jenkins Pipeline

Cov ntaub ntawv txheem ya.HTTP.Request.jmx-report.jtk:

timeStamp,elapsed,label,responseCode,responseMessage,threadName,dataType,success,failureMessage,bytes,sentBytes,grpThreads,allThreads,URL,Latency,IdleTime,Connect
1597311456443,569,Yandex - HTTP Request,200,Ok,Thread Group 1-1,text,true,,59449,220,1,1,https://ya.ru/,145,0,57
1597311456443,147,Yandex - HTTP Request-0,302,Found,Thread Group 1-1,,true,,478,110,1,1,http://ya.ru/,145,0,57
1597311456592,420,Yandex - HTTP Request-1,200,Ok,Thread Group 1-1,text,true,,58971,110,1,1,https://ya.ru/,370,0,259

xaus

Kab lus no tau pom ib qho ntawm cov kev xaiv rau kev khiav JMeter kev xeem hauv OpenShift ib puag ncig. Peb ua tiav tag nrho cov kauj ruam manually, tom qab ntawd peb tsim Jenkins Pipeline los ua kom cov txheej txheem ntawm kev xeem khiav.

Cov peev txheej thiab cov ntaub ntawv

Tau qhov twg los: www.hab.com

Ntxiv ib saib