🥇Launching IntelliJ IDEA inspections on Jenkins

IntelliJ IDEA today has the most advanced static Java code analyzer, which, in terms of its capabilities, has left far behind such "veterans" as Check style и Spotbugs. Its numerous "inspections" check the code in various aspects, from coding style to characteristic bugs.

However, as long as the results of the analysis are displayed only in the developer's local IDE, they are of little use to the development process. Static Analysis must be carried out as the first step of the build pipeline, its results must determine the quality gates, and the build must fail if the quality gates fail. TeamCity CI is known to be integrated with IDEA. But even if you're not using TeamCity, you can still try running IDEA inspections on any other CI server. I propose to see how this can be done using IDEA Community Edition, Jenkins and Warnings NG plugin.

Step 1. Run the analysis in a container and get a report

At first, the idea of running an IDE (desktop application!) inside a CI system that does not have a graphical interface may seem dubious and very troublesome. Fortunately, the IDEA developers have provided the ability to run code formatting и inspections from the command line. Moreover, to run IDEA in this mode, a graphical subsystem is not required, and these tasks can be performed on servers with a text shell.

Inspections are launched using a script bin/inspect.sh from the IDEA installation directory. The parameters required are:

full path to the project (relative ones are not supported),
path to the .xml file with inspection settings (usually located inside the project in .idea/inspectionProfiles/Project_Default.xml),
full path to the folder where .xml files with analysis results reports will be stored.

In addition, it is expected that

the path to the Java SDK will be configured in the IDE, otherwise the analysis will not work. These settings are contained in the configuration file. jdk.table.xml in the IDEA global configuration folder. The default global IDEA configuration itself is in the user's home directory, but this location can be explicitly set in file idea.properties.
the analyzed project must be a valid IDEA project, for which some files that are usually ignored will have to be committed to version control, namely:
- .idea/inspectionProfiles/Project_Default.xml — analyzer settings, they will obviously be used when launching inspections in a container,
- .idea/modules.xml - otherwise we get the error 'This project contains no modules',
- .idea/misc.xml - otherwise we get the error 'The JDK is not configured properly for this project',
- *.iml-файлы - otherwise we will get an error about an unconfigured JDK in the module.

Although these files are usually included in .gitignore, they do not contain any information specific to the environment of a particular developer - unlike, for example, a file workspace.xml, where such information, just, is contained, and therefore it is not necessary to commit it.

By itself, the way out suggests itself to pack the JDK together with IDEA Community Edition into a container in a form ready to be "set" on the analyzed projects. Let's choose a suitable base container, and here is the Dockerfile we get:

Dockerfile

FROM openkbs/ubuntu-bionic-jdk-mvn-py3

ARG INTELLIJ_VERSION="ideaIC-2019.1.1"

ARG INTELLIJ_IDE_TAR=${INTELLIJ_VERSION}.tar.gz

ENV IDEA_PROJECT_DIR="/var/project"

WORKDIR /opt

COPY jdk.table.xml /etc/idea/config/options/

RUN wget https://download-cf.jetbrains.com/idea/${INTELLIJ_IDE_TAR} && 
    tar xzf ${INTELLIJ_IDE_TAR} && 
    tar tzf ${INTELLIJ_IDE_TAR} | head -1 | sed -e 's//.*//' | xargs -I{} ln -s {} idea && 
    rm ${INTELLIJ_IDE_TAR} && 
    echo idea.config.path=/etc/idea/config >> idea/bin/idea.properties && 
    chmod -R 777 /etc/idea

CMD idea/bin/inspect.sh ${IDEA_PROJECT_DIR} ${IDEA_PROJECT_DIR}/.idea/inspectionProfiles/Project_Default.xml ${IDEA_PROJECT_DIR}/target/idea_inspections -v2

With the option idea.config.path we made IDEA look for its global configuration in the folder /etc/idea, since the user's home folder in the working conditions in CI is an indefinite thing and often completely absent.

This is how the file copied to the container looks like jdk.table.xml, which contains the paths to the OpenJDK installed inside the container (it can be based on a similar file from your own IDEA settings directory):

jdk.table.xml

<application>
 <component name="ProjectJdkTable">
   <jdk version="2">
     <name value="1.8" />
     <type value="JavaSDK" />
     <version value="1.8" />
     <homePath value="/usr/java" />
     <roots>
       <annotationsPath>
         <root type="composite">
           <root url="jar://$APPLICATION_HOME_DIR$/lib/jdkAnnotations.jar!/" type="simple" />
         </root>
       </annotationsPath>
       <classPath>
         <root type="composite">
           <root url="jar:///usr/java/jre/lib/charsets.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/deploy.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/ext/access-bridge-64.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/ext/cldrdata.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/ext/dnsns.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/ext/jaccess.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/ext/jfxrt.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/ext/localedata.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/ext/nashorn.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/ext/sunec.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/ext/sunjce_provider.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/ext/sunmscapi.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/ext/sunpkcs11.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/ext/zipfs.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/javaws.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/jce.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/jfr.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/jfxswt.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/jsse.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/management-agent.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/plugin.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/resources.jar!/" type="simple" />
           <root url="jar:///usr/java/jre/lib/rt.jar!/" type="simple" />
         </root>
       </classPath>
     </roots>
     <additional />
   </jdk>
 </component>
</application>

Finished image available on Docker Hub.

Before moving on, let's test running the IDEA parser in the container:

docker run --rm -v <путь/к/вашему/проекту>:/var/project inponomarev/intellij-idea-analyzer

The analysis should work successfully, and numerous .xml files with analyzer reports should appear in the target/idea_inspections subfolder.

Now there is no longer any doubt that the IDEA analyzer can be run offline in any CI environment, and we move on to the second step.

Step 2. Display and analyze the report

Getting a report in the form of .xml files is half the battle, now it needs to be made human-readable. And also its results should be used in quality gates - the logic for determining whether the accepted change passes or does not pass according to quality criteria.

This will help us Jenkins Warnings NG Pluginwhich was released in January 2019. With its introduction, many separate plug-ins for working with static analysis results in Jenkins (CheckStyle, FindBugs, PMD, etc.) are now marked as obsolete.

The plugin consists of two parts:

numerous analyzer message collectors (full list includes all analyzers known to science from AcuCobol to ZPT Lint),
a single report viewer for all of them.

The list of things that Warnings NG can parse includes Java compiler warnings and warnings from Maven execution logs: although they are constantly in sight, they are rarely purposefully analyzed. IntelliJ IDEA reports are also included in the list of recognized formats.

Since the plugin is new, it initially interacts well with the Jenkins Pipeline. The build step with his participation will look like this (we just tell the plugin which report format we recognize and which files should be scanned):

stage ('Static analysis'){
    sh 'rm -rf target/idea_inspections'
    docker.image('inponomarev/intellij-idea-analyzer').inside {
       sh '/opt/idea/bin/inspect.sh $WORKSPACE $WORKSPACE/.idea/inspectionProfiles/Project_Default.xml $WORKSPACE/target/idea_inspections -v2'
    }
    recordIssues(
       tools: [ideaInspection(pattern: 'target/idea_inspections/*.xml')]
    )
}

The report interface looks like this:

It is convenient that this interface is universal for all recognizable parsers. It contains an interactive chart of the distribution of finds by category and a graph of the dynamics of changes in the number of finds. In the grid at the bottom of the page, you can perform a quick search. The only thing that did not work correctly for IDEA inspections was the ability to browse the code directly in Jenkins (although for other reports, such as Checkstyle, this plugin can do it beautifully). It appears to be a bug in the IDEA report parser that needs to be fixed.

Among the features of Warnings NG is the ability to aggregate findings from different sources in one report and program Quality Gates, including a ratchet for a reference assembly. Some Quality Gates programming documentation is available here - however, it is not complete, and you have to look at the source. On the other hand, for complete control over what is happening, the “ratchet” can be implemented independently (see my previous post about this theme).

Conclusion

Before starting to prepare this material, I decided to look: has anyone already written on this topic on Habré? I only found interview 2017 с masticwhere it says:

As far as I know, there is no integration with Jenkins or a maven plugin [...] In principle, any enthusiast could make IDEA Community Edition and Jenkins friends, many would only benefit from this.

Well, after two years we have the Warnings NG Plugin, and finally this friendship has come true!

Source: habr.com

Run IntelliJ IDEA Inspections on Jenkins

Step 1. Run the analysis in a container and get a report

Step 2. Display and analyze the report

Conclusion

Add a comment Отменить ответ