Peb lub tuam txhab tsim cov kav dej tsis siv neeg tag nrho rau kev nthuav tawm cov ntawv thov tshiab rau hauv qhov chaw tsim khoom. Ntawm chav kawm, qhov no yuav tsum tau automated functional tests. Hauv qab no qhov kev txiav yog ib zaj dab neeg hais txog yuav ua li cas, pib nrog ib leeg-xov kev sim ntawm lub tshuab hauv zos, peb tau mus txog lub ntsiab lus ntawm ntau txoj xov autotest khiav ntawm Selenoid hauv kev tsim cov raj xa dej nrog Allure daim ntawv qhia ntawm GitLab nplooj ntawv thiab nws thiaj li tau txais cov cuab yeej automation txias. tias yav tom ntej cov neeg tuaj yeem siv pab pawg.
Peb pib qhov twg?
Txhawm rau siv autotests thiab sib koom ua ke rau hauv cov raj xa dej, peb xav tau cov txheej txheem automation uas tuaj yeem hloov pauv hloov tau raws li peb xav tau. Qhov zoo tshaj plaws, kuv xav tau ib qho qauv rau lub cav autotesting, yoog rau embedding autotests rau hauv cov kav dej. Rau kev siv peb xaiv cov technologies hauv qab no:
- Java,
- Maven,
- Selenium,
- Cucumber + JUNIT 4,
- Allure,
- GitLab.
Vim li cas qhov teeb meem tshwj xeeb no? Java yog ib hom lus nrov tshaj plaws rau kev xeem automated, thiab txhua pab pawg neeg hais lus. Selenium yog qhov kev daws teeb meem pom tseeb. Cucumber, ntawm lwm yam, yuav tsum tau ua kom muaj kev ntseeg siab rau cov txiaj ntsig ntawm kev ntsuas tsis siv neeg ntawm ib feem ntawm cov tuam tsev koom nrog hauv kev ntsuas phau ntawv.
Ib leeg threaded xeem
Txhawm rau kom tsis txhob rov tsim lub log, peb tau txais kev txhim kho los ntawm ntau qhov chaw cia khoom ntawm GitHub ua lub hauv paus rau lub moj khaum thiab hloov kho lawv rau peb tus kheej. Peb tsim ib lub chaw cia khoom rau lub tsev qiv ntawv loj nrog lub hauv paus ntawm autotest moj khaum thiab lub chaw cia khoom nrog tus piv txwv kub ntawm kev siv autotests ntawm peb cov tub ntxhais kawm. Txhua pab pawg yuav tsum coj cov duab Kub thiab tsim cov kev sim hauv nws, ua kom haum rau lawv qhov project. Peb xa nws mus rau GitLab-CI bank, uas peb teeb tsa:
- txhua hnub khiav ntawm tag nrho cov ntawv sau autotests rau txhua qhov project;
- launches nyob rau hauv lub build pipeline.
Thaum xub thawj muaj ob peb qhov kev sim, thiab lawv tau ua nyob rau hauv ib lub kwj deg. Ib leeg-threaded khiav ntawm lub Windows khiav GitLab haum rau peb zoo heev: cov kev ntsuam xyuas tau rub lub rooj sib tw sim me ntsis thiab siv yuav luag tsis muaj peev txheej.
Thaum lub sij hawm, tus naj npawb ntawm autotests tau ntau thiab ntau, thiab peb xav txog kev khiav lawv nyob rau hauv tib lub sij hawm, thaum ib tug tag nrho khiav pib siv li peb teev. Lwm yam teeb meem kuj tshwm sim:
- peb tsis tuaj yeem txheeb xyuas tau tias cov kev sim no ruaj khov;
- cov kev sim uas tau khiav ob peb zaug ua ke ntawm lub tshuab hauv zos qee zaum poob hauv CI.
Piv txwv ntawm kev teeb tsa autotests:
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20</version>
<configuration>
<skipTests>${skipTests}</skipTests>
<testFailureIgnore>false</testFailureIgnore>
<argLine>
-javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
-Dcucumber.options="--tags ${TAGS} --plugin io.qameta.allure.cucumber2jvm.AllureCucumber2Jvm --plugin pretty"
</argLine>
</configuration>
<dependencies>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjweaver</artifactId>
<version>${aspectj.version}</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>io.qameta.allure</groupId>
<artifactId>allure-maven</artifactId>
<version>2.9</version>
</plugin>
</plugins>
Allure qhia piv txwv
Runner load thaum kuaj (8 cores, 8 GB RAM, 1 xov)
Pros ntawm ib leeg-threaded xeem:
- yooj yim teeb tsa thiab khiav;
- launches hauv CI yog xyaum tsis txawv ntawm lub zos launches;
- kev ntsuam xyuas tsis cuam tshuam rau ib leeg;
- yam tsawg kawg nkaus yuav tsum tau rau cov kev pab khiav.
Disadvantages ntawm ib leeg-threaded tests:
- siv sijhawm ntev heev los ua kom tiav;
- ntev stabilization ntawm kev ntsuam xyuas;
- Kev siv tsis zoo ntawm cov khoom siv khiav, kev siv tsawg heev.
Tests ntawm JVM forks
Txij li thaum peb tsis tau saib xyuas cov xov-kev nyab xeeb code thaum siv lub hauv paus moj khaum, txoj kev pom tseeb tshaj plaws los khiav hauv qhov sib npaug yog rau Maven. Lub plugin yooj yim rau kev teeb tsa, tab sis rau kev ua haujlwm sib luag kom raug, autotests yuav tsum tau khiav hauv nyias browsers. Tsis muaj dab tsi ua, Kuv yuav tsum tau siv Selenoid.
Selenoid server tau tsim tawm ntawm lub tshuab nrog 32 cores thiab 24 GB ntawm RAM. Qhov kev txwv tau teeb tsa ntawm 48 browsers - 1,5 threads ib core thiab txog 400 MB ntawm RAM. Raws li qhov tshwm sim, lub sijhawm kuaj tau raug txo los ntawm peb teev mus rau 40 feeb. Kev ua kom nrawm nrawm tau pab daws qhov teeb meem ruaj khov: tam sim no peb tuaj yeem khiav ceev autotests tshiab 20-30 zaug kom txog thaum peb paub tseeb tias lawv khiav tau zoo.
Thawj qhov tsis zoo ntawm cov kev daws teeb meem yog kev siv siab ntawm cov peev txheej khiav nrog cov xov tooj me me ntawm cov xov sib txuas: ntawm 4 cores thiab 8 GB ntawm RAM, cov kev ntsuam xyuas tau ua haujlwm tsis muaj ntau tshaj 6 xov. Qhov thib ob qhov tsis zoo: lub plugin tsim cov chav kawm khiav rau txhua qhov xwm txheej, txawm li cas los xij lawv tau pib.
Tseem ceeb! Tsis txhob dhau qhov sib txawv nrog cov cim rau argLine, piv txwv li no:
<argLine>-Dcucumber.options="--tags ${TAGS} --plugin io.qameta.allure.cucumber2jvm.AllureCucumber2Jvm --plugin pretty"</argLine>
β¦
Mvn βDTAGS="@smoke"
Yog tias koj dhau lub cim npe hauv txoj kev no, lub plugin yuav tsim cov neeg sib tw rau txhua qhov kev xeem, uas yog, nws yuav sim khiav txhua qhov kev xeem, hla lawv tam sim ntawd tom qab tso tawm thiab tsim ntau JVM rab rawg.
Nws yog qhov tseeb los pov qhov sib txawv nrog lub cim rau hauv cim npe hauv cov kev teeb tsa plugin, saib piv txwv hauv qab no. Lwm txoj hauv kev uas peb tau sim muaj teeb meem txuas rau Allure plugin.
Piv txwv ntawm lub sijhawm ua haujlwm rau 6 qhov kev xeem luv luv nrog kev teeb tsa tsis raug:
[INFO] Total time: 03:17 min
Piv txwv ntawm lub sijhawm xeem yog tias koj ncaj qha hloov lub cim rau mvn... βDcucumber.options:
[INFO] Total time: 44.467 s
Piv txwv ntawm kev teeb tsa autotests:
<profiles>
<profile>
<id>parallel</id>
<build>
<plugins>
<plugin>
<groupId>com.github.temyers</groupId>
<artifactId>cucumber-jvm-parallel-plugin</artifactId>
<version>5.0.0</version>
<executions>
<execution>
<id>generateRunners</id>
<phase>generate-test-sources</phase>
<goals>
<goal>generateRunners</goal>
</goals>
<configuration>
<tags>
<tag>${TAGS}</tag>
</tags>
<glue>
<package>stepdefs</package>
</glue>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.21.0</version>
<configuration>
<forkCount>12</forkCount>
<reuseForks>false</reuseForks>
<includes>**/*IT.class</includes>
<testFailureIgnore>false</testFailureIgnore>
<!--suppress UnresolvedMavenProperty -->
<argLine>
-javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar" -Dcucumber.options="--plugin io.qameta.allure.cucumber2jvm.AllureCucumber2Jvm TagPFAllureReporter --plugin pretty"
</argLine>
</configuration>
<dependencies>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjweaver</artifactId>
<version>${aspectj.version}</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
</profile>
Piv txwv ntawm Allure tsab ntawv ceeb toom (qhov kev sim tsis ruaj khov tshaj plaws, 4 reruns)
Runner load thaum kuaj (8 cores, 8 GB RAM, 12 xov)
Tshaj:
- yooj yim teeb - koj tsuas yog yuav tsum tau ntxiv ib lub plugin;
- muaj peev xwm ua ib txhij ua ntau qhov kev xeem;
- acceleration ntawm kev xeem stabilization ua tsaug rau kauj ruam 1.
Txais:
- Ntau OS / ntim yuav tsum tau;
- kev siv nyiaj ntau rau txhua tus diav rawg;
- Lub plugin yog outdated thiab tsis txhawb ntxiv lawm.
Yuav ua li cas kov yeej instability
Test lub rooj zaum tsis zoo tagnrho, ib yam li cov autotests lawv tus kheej. Nws tsis yog qhov xav tsis thoob tias peb muaj ntau qhov kev xeem flacky. Tuaj cawm, uas tawm ntawm lub thawv txhawb nqa rov pib qhov kev xeem ua tsis tiav. Koj yuav tsum tau hloov kho lub plugin version kom tsawg kawg 2.21 thiab sau ib kab nrog tus naj npawb ntawm rov pib dua hauv pom cov ntaub ntawv lossis dhau nws qhov kev sib cav rau Maven.
Piv txwv ntawm kev teeb tsa autotests:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.21.0</version>
<configuration>
β¦.
<rerunFailingTestsCount>2</rerunFailingTestsCount>
β¦.
</configuration>
</plugin>
Los yog thaum pib: mvn β¦ -Dsurefire.rerunFailingTestsCount=2 β¦
Raws li kev xaiv, teeb tsa Maven kev xaiv rau PowerShell tsab ntawv (PS1):
Set-Item Env:MAVEN_OPTS "-Dfile.encoding=UTF-8 -Dsurefire.rerunFailingTestsCount=2"Tshaj:
- tsis tas yuav nkim sijhawm txheeb xyuas qhov kev sim tsis ruaj khov thaum nws tsoo;
- test bench stability teeb meem tuaj yeem txo tau.
Txais:
- floating tsis xws luag yuav plam;
- khiav lub sij hawm nce.
Kev xeem sib npaug nrog lub tsev qiv ntawv Cucumber 4
Tus naj npawb ntawm kev xeem tau nce txhua hnub. Peb rov xav txog kev ua kom nrawm dua. Tsis tas li ntawd, kuv xav koom nrog ntau qhov kev xeem raws li qhov ua tau rau hauv daim ntawv thov kev sib txuas ua ke. Qhov tseem ceeb tshaj plaws yog tias tiam ntawm cov neeg khiav dej num tau siv sijhawm ntev heev thaum ua haujlwm sib luag siv Maven plugin.
Lub sijhawm ntawd, Cucumber 4 twb tau tso tawm lawm, yog li peb txiav txim siab rov sau cov ntsiav rau qhov version no. Nyob rau hauv cov ntawv tso tawm peb tau cog lus thaum uas tig mus tso rau ntawm xov theem. Raws li txoj cai, qhov no yuav tsum yog:
- ua kom nrawm nrawm ntawm kev khiav ntawm autotests los ntawm kev nce cov xov tooj;
- tshem tawm qhov poob ntawm lub sijhawm ntawm kev tsim cov neeg khiav dej num rau txhua qhov autotest.
Optimizing lub moj khaum rau multi-threaded autotests muab tawm tsis yog li ntawd nyuaj. Cucumber 4 khiav txhua qhov kev sim ntawm ib tus xov tooj ntawm qhov pib kom tiav, yog li qee yam zoo li qub tau hloov dua siab tshiab rau ThreadLocal variables.
Qhov loj tshaj plaws thaum hloov pauv siv lub tswv yim refactoring cov cuab yeej yog los xyuas qhov chaw uas qhov sib txawv ntawm qhov sib piv (piv txwv li, kuaj rau null). Tsis tas li ntawd, koj yuav tsum ntxiv Allure plugin rau Junit Runner class annotation.
Piv txwv ntawm kev teeb tsa autotests:
<profile>
<id>parallel</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M3</version>
<configuration>
<useFile>false</useFile>
<testFailureIgnore>false</testFailureIgnore>
<parallel>methods</parallel>
<threadCount>6</threadCount>
<perCoreThreadCount>true</perCoreThreadCount>
<argLine>
-javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
</argLine>
</configuration>
<dependencies>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjweaver</artifactId>
<version>${aspectj.version}</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
</profile>
Piv txwv ntawm Allure tsab ntawv ceeb toom (qhov kev sim tsis ruaj khov tshaj plaws, 5 reruns)
Khiav thauj thaum kuaj (8 cores, 8 GB RAM, 24 xov)
Tshaj:
- kev siv nyiaj tsawg;
- haiv neeg kev txhawb nqa los ntawm Cucumber - tsis muaj cov cuab yeej ntxiv;
- muaj peev xwm khiav ntau tshaj 6 threads ib tug processor core.
Txais:
- koj yuav tsum xyuas kom meej tias cov cai txhawb ntau txoj xov zoo;
- qhov pib nkag nce.
Allure qhia ntawm nplooj ntawv GitLab
Tom qab qhia txog kev ua tiav ntau txoj xov, peb pib siv sijhawm ntau ntxiv los tshuaj xyuas cov ntaub ntawv. Lub sijhawm ntawd, peb yuav tsum rub tawm txhua daim ntawv tshaj tawm ua khoom cuav rau GitLab, tom qab ntawd rub tawm nws thiab qhib nws. Nws tsis yooj yim heev thiab siv sijhawm ntev. Thiab yog tias lwm tus xav saib daim ntawv tshaj tawm rau lawv tus kheej, ces lawv yuav tsum tau ua tib yam haujlwm. Peb xav tau txais kev tawm tswv yim sai dua, thiab peb nrhiav kev daws teeb meem - nplooj ntawv GitLab. Qhov no yog qhov ua tau zoo uas muaj nyob rau hauv lub thawv hauv txhua qhov tsis ntev los no ntawm GitLab. Tso cai rau koj siv cov chaw zoo li qub ntawm koj lub server thiab nkag mus rau lawv ntawm qhov txuas ncaj qha.
Tag nrho cov screenshots ntawm Allure cov ntaub ntawv tau raug coj los ntawm nplooj ntawv GitLab. Tsab ntawv rau xa daim ntawv tshaj tawm rau GitLab nplooj ntawv - hauv Windows PowerShell (ua ntej no koj yuav tsum tau khiav autotests):
New-Item -ItemType directory -Path $testresulthistory | Out-Null
try {Invoke-WebRequest -Uri $hst -OutFile $outputhst}
Catch{echo "fail copy history"}
try {Invoke-WebRequest -Uri $hsttrend -OutFile $outputhsttrnd}
Catch{echo "fail copy history trend"}
mvn allure:report
#mvn assembly:single -PzipAllureReport
xcopy $buildlocationtargetsiteallure-maven-plugin* $buildlocationpublic /s /i /Y
Dab tsi kawg
Yog li, yog tias koj tab tom xav txog seb koj puas xav tau Xov xov nyab xeeb hauv lub Cucumber autotest moj khaum, tam sim no cov lus teb yog qhov pom tseeb - nrog Cucumber 4 nws yooj yim rau kev siv nws, yog li ua kom cov xov tooj ntawm cov xov tau pib ib txhij. Nrog rau txoj kev no ntawm kev xeem khiav, cov lus nug tam sim no dhau los txog kev ua haujlwm ntawm lub tshuab nrog Selenoid thiab lub rooj zaum xeem.
Kev xyaum tau qhia tias khiav autotests ntawm threads tso cai rau koj los txo cov peev txheej siv rau qhov tsawg kawg nkaus nrog qhov ua tau zoo tshaj plaws. Raws li tuaj yeem pom los ntawm cov duab kos, ob npaug ntawm cov xov tsis ua rau muaj kev nrawm zoo sib xws hauv kev ntsuas kev ua haujlwm. Txawm li cas los xij, peb muaj peev xwm ntxiv ntau dua 2 qhov kev xeem tsis siv neeg rau daim ntawv thov tsim, uas txawm tias 200 reruns khiav hauv li 5 feeb. Qhov no tso cai rau koj kom tau txais kev tawm tswv yim sai los ntawm lawv, thiab, yog tias tsim nyog, hloov pauv thiab rov ua cov txheej txheem dua.
Tau qhov twg los: www.hab.com