Travis CI is a distributed web service for building and testing software using GitHub as source code hosting. In addition to the above scenarios, you can add your own, thanks to the extensive configuration options. In this article, we will set up Travis CI to work with PVS-Studio using the PPSSPP code as an example.
Introduction
is a web service for building and testing software. It is usually used in conjunction with continuous integration practices.
PSP emulator. The program is able to emulate the launch of any games from disc images designed for Sony PSP. The release of the program took place on November 1, 2012. PPSSPP is distributed under the GPL v2 license. Anyone can make their own improvements to .
is a static code analyzer for finding errors and potential vulnerabilities in program code. In this article, for a change, we will launch PVS-Studio not locally on the developer's machine, but in the cloud, and look for errors in PPSSPP.
Setting up Travis CI
We will need a repository on GitHub, where the project we need is located, as well as a key for PVS-Studio (you can get or ).
Let's go to the site . After authorization with a GitHub account, we will have a list of repositories:
For testing, I made a fork of PPSSPP.
Activate the repository we want to build:
At the moment, Travis CI is unable to build our project as there are no build instructions. So it's time for the configuration.
During the analysis, we will need some variables, for example, the key for PVS-Studio, which it would be undesirable to specify in the configuration file. So let's add environment variables using the build setup in Travis CI:
We need:
- PVS_USERNAME - username
- PVS_KEY - key
- MAIL_USER - email that will be used to send the report
- MAIL_PASSWORD - email password
The last two are optional. They will be used to send results by mail. If you want to distribute the report in another way, then you do not need to specify them.
So, we have added the environment variables we need:
Now let's create a file .travis.yml and put it in the root of the project. PPSSPP already had a configuration file for Travis CI, however, it was too large and completely unsuitable for the example, so it had to be greatly simplified and only the main elements remained.
First, we specify the language, the version of Ubuntu Linux that we want to use in the virtual machine, and the necessary packages for building:
language: cpp
dist: xenial
addons:
apt:
update: true
packages:
- ant
- aria2
- build-essential
- cmake
- libgl1-mesa-dev
- libglu1-mesa-dev
- libsdl2-dev
- pv
- sendemail
- software-properties-common
sources:
- sourceline: 'ppa:ubuntu-toolchain-r/test'
- sourceline: 'ppa:ubuntu-sdk-team/ppa'All packages listed are for PPSSPP only.
Now we specify the assembly matrix:
matrix:
include:
- os: linux
compiler: "gcc"
env: PPSSPP_BUILD_TYPE=Linux PVS_ANALYZE=Yes
- os: linux
compiler: "clang"
env: PPSSPP_BUILD_TYPE=LinuxA little more about the section matrix. In Travis CI, there are two ways to create build variants: the first is to list compilers, operating system types, environment variables, etc., after which a matrix of all possible combinations will be generated; the second is an explicit indication of the matrix. Of course, you can combine these two approaches and add a unique case, or, on the contrary, exclude it using the section exclude. You can read more about this in .
It remains to specify the project-specific assembly instructions:
before_install:
- travis_retry bash .travis.sh travis_before_install
install:
- travis_retry bash .travis.sh travis_install
script:
- bash .travis.sh travis_script
after_success:
- bash .travis.sh travis_after_successTravis CI allows you to add custom commands for different stages of the life of a virtual machine. Section before_install executed before installing packages. Then install, which follows the installation of packages from the list addons.aptwhich we indicated above. The assembly itself takes place in script. If everything went well, then we are in after_success (it is in this section that we will run the static analysis). These are not all the steps that can be modified, if you need more, then you should look in .
For readability, the commands have been moved to a separate script. .travis.sh, which is placed at the root of the project.
So we have the following file .travis.yml:
language: cpp
dist: xenial
addons:
apt:
update: true
packages:
- ant
- aria2
- build-essential
- cmake
- libgl1-mesa-dev
- libglu1-mesa-dev
- libsdl2-dev
- pv
- sendemail
- software-properties-common
sources:
- sourceline: 'ppa:ubuntu-toolchain-r/test'
- sourceline: 'ppa:ubuntu-sdk-team/ppa'
matrix:
include:
- os: linux
compiler: "gcc"
env: PVS_ANALYZE=Yes
- os: linux
compiler: "clang"
before_install:
- travis_retry bash .travis.sh travis_before_install
install:
- travis_retry bash .travis.sh travis_install
script:
- bash .travis.sh travis_script
after_success:
- bash .travis.sh travis_after_successLet's update the submodules before installing the packages. This is needed to build PPSSPP. Let's add the first function to .travis.sh (note the extension):
travis_before_install() {
git submodule update --init --recursive
}Now we have come directly to configuring automatic start of PVS-Studio in Travis CI. First we need to install the PVS-Studio package on the system:
travis_install() {
if [ "$CXX" = "g++" ]; then
sudo apt-get install -qq g++-4.8
fi
if [ "$PVS_ANALYZE" = "Yes" ]; then
wget -q -O - https://files.viva64.com/etc/pubkey.txt
| sudo apt-key add -
sudo wget -O /etc/apt/sources.list.d/viva64.list
https://files.viva64.com/etc/viva64.list
sudo apt-get update -qq
sudo apt-get install -qq pvs-studio
libio-socket-ssl-perl
libnet-ssleay-perl
fi
download_extract
"https://cmake.org/files/v3.6/cmake-3.6.2-Linux-x86_64.tar.gz"
cmake-3.6.2-Linux-x86_64.tar.gz
}At the start of a function travis_install we install the compilers we need using environment variables. Then if the variable $PVS_ANALYZE stores the value Yes (we indicated it in the section env during build matrix configuration), we install the package pvs-studio. In addition to it, packages are also indicated libio-socket-ssl-perl ΠΈ libnet-ssleay-perl, however, they are needed to send results by mail, so they are not needed if you have chosen another way to deliver the report.
Function download_extract downloads and unpacks the specified archive:
download_extract() {
aria2c -x 16 $1 -o $2
tar -xf $2
}It's time to assemble the project. It happens in the section script:
travis_script() {
if [ -d cmake-3.6.2-Linux-x86_64 ]; then
export PATH=$(pwd)/cmake-3.6.2-Linux-x86_64/bin:$PATH
fi
CMAKE_ARGS="-DHEADLESS=ON ${CMAKE_ARGS}"
if [ "$PVS_ANALYZE" = "Yes" ]; then
CMAKE_ARGS="-DCMAKE_EXPORT_COMPILE_COMMANDS=On ${CMAKE_ARGS}"
fi
cmake $CMAKE_ARGS CMakeLists.txt
make
}In fact, this is a simplified original configuration, except for these lines:
if [ "$PVS_ANALYZE" = "Yes" ]; then
CMAKE_ARGS="-DCMAKE_EXPORT_COMPILE_COMMANDS=On ${CMAKE_ARGS}"
fiIn this piece of code, we set for cmake compilation command export flag. This is necessary for a static code analyzer. You can read more about this in the article "Β«.
If the assembly was successful, then we get into after_success, where we perform static analysis:
travis_after_success() {
if [ "$PVS_ANALYZE" = "Yes" ]; then
pvs-studio-analyzer credentials $PVS_USERNAME $PVS_KEY -o PVS-Studio.lic
pvs-studio-analyzer analyze -j2 -l PVS-Studio.lic
-o PVS-Studio-${CC}.log
--disableLicenseExpirationCheck
plog-converter -t html PVS-Studio-${CC}.log -o PVS-Studio-${CC}.html
sendemail -t [email protected]
-u "PVS-Studio $CC report, commit:$TRAVIS_COMMIT"
-m "PVS-Studio $CC report, commit:$TRAVIS_COMMIT"
-s smtp.gmail.com:587
-xu $MAIL_USER
-xp $MAIL_PASSWORD
-o tls=yes
-f $MAIL_USER
-a PVS-Studio-${CC}.log PVS-Studio-${CC}.html
fi
}Let's take a closer look at the following lines:
pvs-studio-analyzer credentials $PVS_USERNAME $PVS_KEY -o PVS-Studio.lic
pvs-studio-analyzer analyze -j2 -l PVS-Studio.lic
-o PVS-Studio-${CC}.log
--disableLicenseExpirationCheck
plog-converter -t html PVS-Studio-${CC}.log -o PVS-Studio-${CC}.htmlThe first line generates a license file from the username and key that we specified at the very beginning when setting the Travis CI environment variables.
The second line starts the analysis itself. Flag -j sets the number of threads for analysis, flag -l indicates license, flag -o defines a file for outputting logs, and the flag -disableLicenseExpirationCheck required for trial versions, since by default pvs-studio-analyzer will warn the user that the license is about to expire. To avoid this, you can specify this flag.
The log file contains raw output that cannot be read without conversion, so the first step is to make the file readable. Let's pass the logs through plog-converter, and the output is an html file.
In this example, I decided to send reports by mail using the command send email.
As a result, we have the following file .travis.sh:
#/bin/bash
travis_before_install() {
git submodule update --init --recursive
}
download_extract() {
aria2c -x 16 $1 -o $2
tar -xf $2
}
travis_install() {
if [ "$CXX" = "g++" ]; then
sudo apt-get install -qq g++-4.8
fi
if [ "$PVS_ANALYZE" = "Yes" ]; then
wget -q -O - https://files.viva64.com/etc/pubkey.txt
| sudo apt-key add -
sudo wget -O /etc/apt/sources.list.d/viva64.list
https://files.viva64.com/etc/viva64.list
sudo apt-get update -qq
sudo apt-get install -qq pvs-studio
libio-socket-ssl-perl
libnet-ssleay-perl
fi
download_extract
"https://cmake.org/files/v3.6/cmake-3.6.2-Linux-x86_64.tar.gz"
cmake-3.6.2-Linux-x86_64.tar.gz
}
travis_script() {
if [ -d cmake-3.6.2-Linux-x86_64 ]; then
export PATH=$(pwd)/cmake-3.6.2-Linux-x86_64/bin:$PATH
fi
CMAKE_ARGS="-DHEADLESS=ON ${CMAKE_ARGS}"
if [ "$PVS_ANALYZE" = "Yes" ]; then
CMAKE_ARGS="-DCMAKE_EXPORT_COMPILE_COMMANDS=On ${CMAKE_ARGS}"
fi
cmake $CMAKE_ARGS CMakeLists.txt
make
}
travis_after_success() {
if [ "$PVS_ANALYZE" = "Yes" ]; then
pvs-studio-analyzer credentials $PVS_USERNAME $PVS_KEY -o PVS-Studio.lic
pvs-studio-analyzer analyze -j2 -l PVS-Studio.lic
-o PVS-Studio-${CC}.log
--disableLicenseExpirationCheck
plog-converter -t html PVS-Studio-${CC}.log -o PVS-Studio-${CC}.html
sendemail -t [email protected]
-u "PVS-Studio $CC report, commit:$TRAVIS_COMMIT"
-m "PVS-Studio $CC report, commit:$TRAVIS_COMMIT"
-s smtp.gmail.com:587
-xu $MAIL_USER
-xp $MAIL_PASSWORD
-o tls=yes
-f $MAIL_USER
-a PVS-Studio-${CC}.log PVS-Studio-${CC}.html
fi
}
set -e
set -x
$1;It's time to add the changes to the git repository, after which Travis CI will automatically start the build. Click on "ppsspp" to go to build reports:
We will see an overview of the current assembly:
In case of successful completion of the build, we will receive an email with the results of static analysis. Of course, mailing is not the only way to get a report. You can choose any implementation method. But it is important to remember that after the build is completed, it will be impossible to access the files of the virtual machine.
Brief overview of errors
We have successfully completed the most difficult part. Now let's make sure that all our efforts paid off. Let's consider some interesting points from the static analysis report that came to me by mail (I pointed it out for a reason).
Dangerous optimization
void sha1( unsigned char *input, int ilen, unsigned char output[20] )
{
sha1_context ctx;
sha1_starts( &ctx );
sha1_update( &ctx, input, ilen );
sha1_finish( &ctx, output );
memset( &ctx, 0, sizeof( sha1_context ) );
}PVS-Studio warning: The compiler could delete the 'memset' function call, which is used to flush 'sum' buffer. The RtlSecureZeroMemory() function should be used to erase the private data. sha1.cpp 325
This piece of code is in the secure hashing module, however, it contains a serious security flaw (). Consider the assembler listing that is generated when compiling the Debug version:
; Line 355
mov r8d, 20
xor edx, edx
lea rcx, QWORD PTR sum$[rsp]
call memset
; Line 356Everything is in order, and the function memeset is executed, thereby overwriting important data in RAM, however, you should not rejoice yet. Consider the assembler listing of the Release version with optimization:
; 354 :
; 355 : memset( sum, 0, sizeof( sum ) );
; 356 :}As you can see from the listing, the compiler ignored the call memeset. This is due to the fact that in the function sha1 after the call memeset no more access to structure ctx. Therefore, the compiler sees no point in wasting processor time on overwriting memory that is not used in the future. You can fix this by using the function RtlSecureZeroMemory or her.
Correctly:
void sha1( unsigned char *input, int ilen, unsigned char output[20] )
{
sha1_context ctx;
sha1_starts( &ctx );
sha1_update( &ctx, input, ilen );
sha1_finish( &ctx, output );
RtlSecureZeroMemory(&ctx, sizeof( sha1_context ) );
} Superfluous comparison
static u32 sceAudioOutputPannedBlocking
(u32 chan, int leftvol, int rightvol, u32 samplePtr) {
int result = 0;
// For some reason, this is the only one that checks for negative.
if (leftvol > 0xFFFF || rightvol > 0xFFFF || leftvol < 0 || rightvol < 0) {
....
} else {
if (leftvol >= 0) {
chans[chan].leftVolume = leftvol;
}
if (rightvol >= 0) {
chans[chan].rightVolume = rightvol;
}
chans[chan].sampleAddress = samplePtr;
result = __AudioEnqueue(chans[chan], chan, true);
}
}PVS-Studio warning: Expression 'leftvol >= 0' is always true. sceAudio.cpp 120
Notice the else branch for the first one if. The code will be executed only if all the conditions leftvol > 0xFFFF || rightvol > 0xFFFF || leftvol < 0 || rightvol < 0 turn out to be false. Therefore, we get the following statements, which will be true for the else branch: leftvol <= 0xFFFF, rightvol <= 0xFFFF, leftvol >= 0 ΠΈ rightvol >= 0. Pay attention to the last two statements. Does it make sense to check what is a necessary condition for the execution of this piece of code?
So we can safely remove these conditional statements:
static u32 sceAudioOutputPannedBlocking
(u32 chan, int leftvol, int rightvol, u32 samplePtr) {
int result = 0;
// For some reason, this is the only one that checks for negative.
if (leftvol > 0xFFFF || rightvol > 0xFFFF || leftvol < 0 || rightvol < 0) {
....
} else {
chans[chan].leftVolume = leftvol;
chans[chan].rightVolume = rightvol;
chans[chan].sampleAddress = samplePtr;
result = __AudioEnqueue(chans[chan], chan, true);
}
}Another scenario. Behind these redundant conditions lies some kind of error. Perhaps they checked not what is required.
Ctrl+C Ctrl+V strikes back
static u32 scePsmfSetPsmf(u32 psmfStruct, u32 psmfData) {
if (!Memory::IsValidAddress(psmfData) ||
!Memory::IsValidAddress(psmfData)) {
return hleReportError(ME, SCE_KERNEL_ERROR_ILLEGAL_ADDRESS, "bad address");
}
....
}There are identical sub-expressions '!Memory::IsValidAddress(psmfData)' to the left and to the right of the '||' operator. scePsmf.cpp 703
Pay attention to the check inside if. Don't you think it's strange that we check if the address is valid? psmfData, two whole times? That seems strange to me too⦠In fact, we have, of course, a typo, and the idea was to check both input parameters.
Correct option:
static u32 scePsmfSetPsmf(u32 psmfStruct, u32 psmfData) {
if (!Memory::IsValidAddress(psmfStruct) ||
!Memory::IsValidAddress(psmfData)) {
return hleReportError(ME, SCE_KERNEL_ERROR_ILLEGAL_ADDRESS, "bad address");
}
....
}Forgotten variable
extern void ud_translate_att(
int size = 0;
....
if (size == 8) {
ud_asmprintf(u, "b");
} else if (size == 16) {
ud_asmprintf(u, "w");
} else if (size == 64) {
ud_asmprintf(u, "q");
}
....
}PVS-Studio warning: Expression 'size == 8' is always false. syn-att.c 195
This error is in the folder ext, so not exactly relevant to the project, but the bug was found before I paid attention to it, so I decided to leave it. After all, this article is not about an overview of errors, but about integration with Travis CI, and no analyzer settings were made.
Variable size is initialized with a constant, however, it is not used at all in the code, up to the operator if, which, of course, gives false while checking the condition, because, as we remember, size equals zero. Subsequent checks also do not make sense.
Apparently, the author of the code snippet forgot to overwrite the variable size before that.
Stop
On this, perhaps, we will end with errors. The purpose of this article is to demonstrate the work of PVS-Studio together with Travis CI, and not to analyze the project as carefully as possible. If you want bigger and prettier bugs, you can always admire them :).
Conclusion
Using web services to build projects together with the practice of incremental analysis allows you to detect many problems immediately after the code is merged. True, one build may not be enough, so setting up testing together with static analysis will significantly improve the quality of the code.
Useful links
If you want to share this article with an English-speaking audience, please use the translation link: Maxim Zvyagintsev. .
Source: habr.com