Static analysis and SonarQube web service

First if not already done, install Docker.

Then one can download the existing Docker image (~3.5 Go)

docker pull hpclib/sonarqube

Notice this image has been made with this Dockerfile.

Finally run interactively in the container with

docker run -it hpclib/sonarqube

In this environment all the analysis tools used in the following are available, sonar-scanner, coverity, cppcheck, clang, gcov, valgrind, etc. Thus you don't need to follow the "installation" recommandation given below. Still, the token creation is required, see next section.

To be able to publish results to SonarQube a token system is necessary for the authentication. Log in to https://sonarqube.inria.fr/sonarqube with your Inria LDAP credential and create a personal token (save it in your home for example in $HOME/.sonarqubetoken)

If you don't use the Docker image you will need to install sonar-scanner to perform a static analysis, index source code files, and publish the results. Skip this stage if you use the Docker image.

Please install the sonar-scanner program.

We provide some installation scripts for some well known operating systems, please refer to one of the install-sonar-scanner-*.sh scripts. wget, unzip, sudo, ln commands are required.

Lets try sonar-scanner with an "Hello World" python script.

First create the "hello world" python script

# This program prints Hello, world!

print('Hello, world!')

you can execute it if you have python installed

python hello_world.py

Optionnaly save your token in a file, for example

echo "34539dd39d77f225f552676191a23ff725116702" > ~/.sonarqubetoken

Then lets use sonar-scanner to analyze this source code

sonar-scanner -D"sonar.host.url=https://sonarqube.inria.fr/sonarqube" -D"sonar.login=$(cat ~/.sonarqubetoken)" -D"sonar.projectKey=sedbso:helloworld:test" -D"sonar.sources=./hello_word.py" -D"sonar.language=py"

Visit the sonarqube project page and look for your submitted project.

sonar-scanner is used in a shell through a CLI. One can pass the options directly in the command line or they can be stored in a file sonar-project.properties.

sonar.host.url=https://sonarqube.inria.fr/sonarqube
sonar.login=34539dd39d77f225f552676191a23ff725116702
sonar.projectKey=sedbso:helloworld:test
sonar.sources=./hello_world.py
sonar.language=py

Then you should be able to invoke sonar-scanner more simply

sonar-scanner

Note that one can set some options in the file and change their values at the level of the command line (take precedence).

Please skip this stage if you use the Docker image.

heat is a toy code written in C that will be used in this hands-on session to perform static and dynamic analysis.

We propose to install some dependencies of the heat program, git, C compiler (GCC), CMake, plus some useful analysis tools, Clang Static Analyzer, Cppcheck, GCOV, Valgrind, pytest, pylint …

We provide some scripts to do that depending on the OS, see install-dev-tools-*.sh scripts.

In a shell, git clone the Heat source code, this is a public git repository

# git command is required
git clone https://gitlab.inria.fr/sed-bso/heat.git
cd heat/

Configure and build Heat so that:

• we enable and save some chosen GCC warnings
• we enable GCOV coverage
• we perform a static analysis with Clang Static Analyzer

mkdir -p build
cd build
# make clean
CFLAGS="--coverage -fPIC -fdiagnostics-show-option -Wall -Wunused-parameter -Wundef -Wno-long-long -Wsign-compare -Wmissing-prototypes -Wstrict-prototypes -Wcomment -pedantic -g"
LDFLAGS="--coverage"
scan-build -v -plist --intercept-first --analyze-headers -o analyzer_reports cmake .. -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DCMAKE_C_FLAGS="$CFLAGS" -DCMAKE_EXE_LINKER_FLAGS="$LDFLAGS"
scan-build -v -plist --intercept-first --analyze-headers -o analyzer_reports make 2>&1 | tee heat-build.log

Remarks:

• CMAKE_VERBOSE_MAKEFILE is set to ON to get detailed compilation lines
• CMAKE_EXPORT_COMPILE_COMMANDS is set to ON to get compile commands used during the make process. It will be used later for the clang-tidy analysis and cppcheck
• "–coverage" is used for coverage analysis with gcov, see Instrumentation Options
• "-fPIC" should be used if a shared library is built
• "-fdiagnostics-show-option" is used to feed GCC warnings into SonarQube, see sonar-cxx wiki
• "-Wall -Wunused-parameter -Wundef -Wno-long-long -Wsign-compare -Wmissing-prototypes -Wstrict-prototypes -Wcomment -pedantic" are used to enable some GCC warnings (arbitrarily)
• scan-build program is used to perform a clang static analyzer scan, see scan-build
• we build with debug information "-g" to help scan-build (clang analyzer) to be more effective (avoid false positive), see scan-build recommended debug
• scan-build results are stored in analyzer_reports
• "-plist" option is used for getting some report files compatible with SonarQube

We perform unitary tests and this generates gcov coverage files thanks to the "–coverage" flags used just before, see resulting *.gcda and *.gcno files

ctest -V

find . -regex '.*\.\(gcda\|gcno\)'

SonarQube performs a static analysis when we invoke sonar-scanner (see the next section "SonarQube scan"). The issues that are considered by the SonarQube analysis depends on the language and for C/C++ it relies on the sonar-cxx plugin. In addition to its own analysis the plugin allows to import issues addressed by external analysis tools.

In this section we run some of the external tools that are compatible to perform static (clang-tidy, cppcheck) and dynamic analysis (coverage with gcov, valgrind, drmemory).

Notice we already have performed the clang static analysis during the build step. There is no need to do something else, the clang reports generated with the "-plist" parameter are compatible with SonarQube.

SonarQube performs a static analysis when we invoke sonar-scanner. The issues that are considered by the SonarQube analysis depends on the language and for C/C++ it relies on the sonar-cxx plugin.

The sonar-scanner program is used for executing the SonarQube scan and put the results online i.e. upload to the server. The program takes some arguments such as the url of the server, credentials, the name we want to assign to our project analysis, paths to external analysis reports, etc.

See how to use it. Parameters can be given either with the -D argument of the program and/or defining them in a configuration file sonar-project.properties which must be located in the current path where sonar-scanner is invoked.

The same tools as for C program can be used. The only things to change are some parameters in the file sonar-project.properties.

• sonar.c properties should be replaced by sonar.cxx
• sonar.language should be set to "c++".
• to find compiler header files directories

echo | gcc -xc++ -E -Wp,-v - 2>&1 | grep "^ " | xargs | sed -e "s# #, #g"

Example with the Explauto library.

Install

git clone https://github.com/flowersteam/explauto.git
cd explauto
python3 -m pip install codecov pytest-cov setuptools
python3 -m pip install -r requirements.txt
python3 setup.py build
sudo python3 setup.py develop

Test + Coverage

pytest --cov=./ --junit-xml=test.xml --cov-report xml

Setup the SonarQube configuration

sonar.links.homepage=https://githutb.com/flowersteam/explauto
sonar.links.scm=https://github.com/flowersteam/explauto.git
sonar.projectKey=sedbso:explauto:fpruvost
sonar.projectDescription=An autonomous exploration library
sonar.projectVersion=1.4.0
sonar.scm.disabled=false
sonar.scm.provider=git
sonar.sourceEncoding=UTF-8
sonar.sources=.
sonar.exclusions=build
sonar.language=py
sonar.python.xunit.reportPath=test.xml
sonar.python.coverage.reportPaths=coverage.xml

Perform sonarqube analysis and publish the results

sonar-scanner -X -Dsonar.host.url=https://sonarqube.inria.fr/sonarqube -Dsonar.login=`cat ~/.sonarqubetoken` >sonar.log 2>&1

Notice SonarQube perfoms also a pylint analysis if it is installed on the system. In addition unitary tests and coverage are imported thanks to the xml files generated before with pytest.

See the results in SonarQube.

The C/C++ analyzer from SonarQube is a paid service, see SonarCFamily plugin. This plugin cannot be used in the community version we have instanciated at Inria. For C/C++ we use the community plugin sonar-cxx. With this plugin we get access to a few SonarQube issues (during the sonar-scanner analysis) and we can import many other reports coming from external analyzers. This is fine but we don't have access to the "official" SonarQube C/C++ analyzer which is surely interesting too.

The good news ? There exists a public SonarQube server, sonarcloud for opensource projects which gives access to the SonarCFamily plugin.

If you have an opensource project to analyze and you want to see what gives the sonar-scanner analysis with the SonarCFamily plugin please visit the sonarsource documentation.

See for example the SimGrid project.

Coverity is another analyzer that is normally a paid service but which gives access to a public server for opensource projects, see :

• Main page
• Dashboard

Download the cov-build program here.

COVERITY_TOKEN="XEJaJ1cAnqW-9M_zkmxd7w"
wget --no-check-certificate https://scan.coverity.com/download/linux64 --post-data "token=$COVERITY_TOKEN&project=Heat" -O coverity_tool.tgz
tar xvf coverity_tool.tgz
sudo ln -s -f $PWD/cov-analysis-linux64-*/bin/cov-build /usr/local/bin/cov-build

Build and analyze with the 'cov-build' command

git clone https://gitlab.inria.fr/sed-bso/heat.git
cd heat/
mkdir -p build
cd build
cmake ..
make clean
cov-build --dir cov-int make -j 4

Create a compress tar archive of the results

tar czvf heat.tgz cov-int

Submit compressed file created in previous step for analysis

curl --form token=$COVERITY_TOKEN --form email=florent.pruvost@inria.fr --form file=@heat.tgz --form version="`git rev-parse --short HEAD`" --form description="" https://scan.coverity.com/builds?project=Heat

See the results in the Coverity dashboard, follow the "View Defects" link.

Features

• Paid service if not opensource
• Languages: C/C++, C#, Java, Javascript, PHP, Python, .NET Core, ASP.NET, Objective-C, Go, JSP, Ruby, Swift, Fortran, Scala, VB.NET, iOS, TypeScript
• Well integrated in IDEs, supports most of compilers
• Fine detection of bugs (static analysis)
• Concurrency issues (concurrent data access, deadlocks, race conditions)
• Security issues, …

• chameleon - gitlab - sonarqube - sonar-scanner script
• hwloc - github - sonarqube - sonar-scanner script
• Mmg - github - sonarqube - sonar-scanner script
• Scotch - gitlab - sonarqube - sonar-scanner script
• Simgrid - github - sonarqube - sonar-project.properties
• StarPU - gforge - sonarqube

Now try it on your own code !

Please look at the Overview tab of your project.

One first thing you should think of is the question of the project visibility. Is your program free, opensource ? Do you want to share the SonarQube measures with your community ? If yes then go to the Administration -> Permissions tab and turn the project "Public". Hence the project will be examinable by anyone.

The main metrics are displayed such as the number of Bugs (reliability), Vulnerabilities (security aspects), Code Smells (encourage to follow standards), Unitary tests and Coverage, Duplications. Each measure is clickable, one can explore

• graphs of measures to get nice views of the least/most impacted files
  • circles at top-left : this is not good you should do something
  • circles at bottom-right : good job
• activity: historical trends in the course of several analysis
• lists of issues

The Quality Gate (QG), is always in a "Passed" (green) status because we have decided not to use it by default. One can use it to define a state change of the code quality. For example if we get new bugs between two analysis then the QG becomes red and developers should be notified and fix the bugs to turn the QG green.

There is also the notion of "New Bugs", code smells etc, since previous version. What does this mean ? It simply reports how things get worse since the previous version number, the number used for the sonar.projectVersion parameter. If you change the version from 1.0 to 2.0 subsequent analysis will report new issues since the last analysis of version 1 in the "Leak Period: since 1.0" yellow part.

By clicking on Issues you've got the list of all issues (bugs, vulnerabilities, code smells). The left part of the interface allows to filter the issues to improve the navigation and understanding.

This Issues tab is convenient to explore the results of static and dynamic analysis coming from the different tools (sonar-scanner, warning gcc, clang-sa, clang-tidy, cppcheck, valgrind, drmemory, etc) and to manage them. By manage we mean to decide whether or not the issues should be fixed or not. By default new issues have an Open status. The goal is to decrease the number of open issues. The first thing to do in the interface is to choose which issues should be fixed in a near future or not.

1. Issues to be fixed are tagged Confirmed. If in the next analysis issues are not in the reports anymore then SonarQube will turn them as Fixed and Closed.
2. Issues that are not fixed should be classified. Why not fixing these issues ? Is it because A. this is a false positive, and the issue should be tagged false positive B. the issue is known and we accept to keep it as it is for good reasons, the issue should be tagged won't fix

Remark: closed issues are kept for 30 days in the database.

There are many way to filter issues, see on the left side of the tab

• Type : bug, vulnerabilities, code smells
• Resolution and Status : Unresolved, fixed , Open, Closed, …
• Rule : the issue name
• Directory, File
• Assignee, Author
• Language

The Measures tab allows to get synthetic views with graphs, treemap, directory tree, about the bugs, vulnerabilities, code smells, coverage, unitary tests, duplications, size, complexity, issue kinds.

The Code tab allows to navigate in the code structure directory by directory and get the size (lines of code), the number of issues, the coverage percentage and the duplications.

The Activity tab informs about the history of analysis. One can find the dates and some graphs to follow the trend of some measures.

See https://docs.sonarqube.org/display/SONARQUBE67/Project+Administration+Guide.