CMake hands-on
Table of Contents
- 1. Introduction
- 2. Basics
- 3. Create a simple CMake project from scratch
- 4. Advanced usage
- 5. Good pratices
- 5.1. Variables scope
- 5.2. Custom commands (function/macro)
- 5.3. Generator expressions
- 5.4. Do not edit CMAKE_CXX_FLAGS in cmake project
- 5.5. Managing source file lists
- 5.6. Properties on targets vs. variables
- 5.7. Propagate properties on target or not, that is the question (PRIVATE|INTERFACE|PUBLIC)
- 5.8. Export your targets into a config file for your users
- 5.9. Use exported target when looking for dependencies
- 5.10. Ctest and cdash and how to use it with gcov and valgrind
- 5.11. Static analysis with CMake
- 6. Useful links
This CMake hands-on session is presented by the Inria's SED team (Service d'Expérimentation et Développement) of Inria Bordeaux Sud-Ouest.
This training session is intended for developers with little or no CMake experience. We will discuss the problems CMake addresses, and incrementally build around CMake a project representative of “real-world” software packages. During the exercises, we will tackle Unix environments (Linux and OS X) only and not Windows.
1 Introduction
1.1 Software building generalities
1.2 Requirements
1.2.1 Install compilers and CMake
The goal of the hands-on is to create a CMake project to compile C source code files and to generate a library and executables. Participants are expected to have a usable Unix installation, with some packages installed. The list of packages required is given for some common distributions in the following subsections.
Notice that CMake could also be installed from a binary distribution located here https://cmake.org/download/.
- Debian (apt-get)
sudo apt-get update sudo apt-get install -y build-essential libopenmpi-dev cmake cmake-data cmake-curses-gui cmake-qt-gui
- RedHat (yum)
yum update sudo yum -y groupinstall 'Development Tools' sudo yum -y install gcc-gfortran openmpi-devel cmake cmake-data cmake-gui export PATH=/usr/lib64/openmpi/bin:$PATH
- Fedora (dnf)
sudo dnf update sudo dnf -y groupinstall 'Development Tools' sudo dnf -y install gcc-gfortran openmpi-devel cmake cmake-gui export PATH=/usr/lib64/openmpi/bin:$PATH
- ArchLinux (pacman)
sudo pacman -Sy sudo pacman -S base-devel openmpi gcc-fortran cmake cmake-gui
- Mac OS X (port)
sudo port selfupdate sudo port install gcc49 cctools openmpi-devel cmake-devel +gui
- Mac OS X (brew)
brew update brew install gcc open-mpi make cmake
- Windows
Download and install with the Windows x64 Installer available here: https://cmake.org/download/.
1.2.2 Download the hands-on materials and setup the working directory
The hands-on session is organized around several exercises which tackle a precise topic.
Please choose now the directory where you want to work. The default choice here is $HOME/work but you are free to change this:
cd $HOME/work
Then download the materials required for the hands-on:
wget https://sed-bso.gitlabpages.inria.fr/cmake/cmake.tgz tar xf cmake.tgz cd cmake/ ls -l export CMAKE_HANDS_ON=$PWD
1.2.3 Explanations about the C program used for the exercises
- We want to solve the heat propagation equation
- Using the discretisations
\[\frac{\partial u}{\partial t}({\bf x},t_k) \approx \frac{u({\bf x}, t_k + dt) - u({\bf x}, t_k)}{dt},\] and
\begin{eqnarray*} \Delta u(x,y,t) & \approx & \frac{u(x-h_x, y,t) + u(x+ h_x, y,t) - 2 u(x,y,t)}{h_x^2} \\ & + & \frac{u(x, y-h_y,t) + u(x, y + h_y,t) - 2 u(x,y,t)}{h_y^2}, \end{eqnarray*}- It yields to the following matrix update:
\(\forall (i,j) \in \{1..n_x-1\}\times\{1..n_y-1\}\) with \[ w_x = \frac{dt}{h_x^2}, \quad , w_y = \frac{dt}{h_y^2} \mbox { and } d=1 - 2 w_x - 2 w_y \]
- The algorithm reduces to two iterations :
- an outer loop for time
- two nested inner loop for space
- The space inner loops consist in updating a matrix using a stencil
- Note that, we update only the "interior" of the matrix
- The cells on the boundary remain constants (ghosts)
2 Basics
2.1 What is CMake, why is it convenient (or not)?
CMake is a cross-platform (Linux, Mac OS X, Windows) open-source make
system. CMake is used to control the software compilation process
using simple platform-independent and compiler-independent
configuration files (cf. CMakeLists.txt
files).
Unlike many cross-platform systems, CMake is designed to be used in conjunction with the native build environment. Simple configuration files placed in each source directory (called CMakeLists.txt files) are used to generate standard build files (e.g., makefiles on Unix and projects/workspaces in Windows MSVC) which are used in the usual way.
2.1.1 Features summary
- Open-source: BSD 3-Clause License (BSD New)
- Cross-platform
- supports Linux, Darwin (OS X), Windows
- makefile generators: Borland, MSYS, MinGW, Ninja, NMake, Unix, Watcom
- generate project files for major IDEs: CodeBlocks, Eclipse, KDevelop, Sublime, Visual Studio, Xcode, see complete list
- Supported programming languages: C++, C, CUDA, Fortran, ASM
- Integrated tools suite for building software
- CMake: makefiles generator
- CTest: testing utility
- CDash: web interface to present a summary of builds, tests, coverage
- CPack: source/binary package builder
A more exhaustive list of cool features can also be explored here.
2.1.2 Pros and Cons
- Pros:
- full-featured
- cross-platform, supports well Windows unlike Autotools
- basic scripting language, fast learning curve compared to Autotools (which requires knowledge of at least make, m4, and shell scripting), cf. http://clubjuggler.livejournal.com/138364.html
- just one tool instead of automake+autoconf+libtool+m4
- requires few code for common tasks
- nice output: colors and progress status
- graphical user interfaces (both Curses and QT based)
- supports out-of-tree builds
- Cons:
- uses its own language, reinventing the wheel and not providing access to the extensibility and power of a real language, cf. https://github.com/SCons/scons/wiki/SconsVsOtherBuildTools
- it does not follow any well known standard or guidelines
- documentation is not so good and we often rely on non official webpages, the complete documentation is a charged book
- needs CMake to be installed
- no uninstall target by default
2.2 Install CMake
Binary packages can be easily installed, cf. requirements:
- Debian (apt-get):
cmake cmake-data cmake-curses-gui cmake-qt-gui
- CentOS (yum):
cmake cmake-data cmake-gui
- Fedora (dnf):
cmake cmake-gui
- ArchLinux (pacman):
cmake cmake-gui
- Mac OS X (port):
cmake-devel +gui
- Mac OS X (brew):
cmake
You can also get CMake pre-compiled binaries here or instructions to install from sources here.
2.3 Compile an existing program using CMake
Let's practice a little bit. We consider here that we have got the tarballs containing the material, cf. download material section.
Check the content of the first exercise directory
cd $CMAKE_HANDS_ON/exe1_usecmake/ ls -l
Please check that you have some C source code files and a CMakeLists.txt.
One interesting feature in CMake is that you can build "out-of-source" meaning that you can build in your prefered directory where you want on your system and it is not mandatory to build in the root directory of the project. Let's be honest, it is strongly recommended to build "out-of-source" in CMake. Why? Because there is no distclean rule that permit to remove all the files generated during the build phase. There is a clean target to remove files generated by the compiler and linker but there is no way for CMake to know in advance all the objects that user's scripts may create, cf. this link
Let's build our project in the sub-directory build/ to be fancy
mkdir -p build && cd build
The build process with CMake takes place in two stages:
- standard build files are created from configuration files using
the
cmake [options] <path-to-parent-CMakeLists.txt>
command, normally available in your environment, cf. requirements - then the platform's native build tools are used for the actual build
cmake .. make ./heat_seq 100 100 100 0
One can also drive the different steps, configure and build, without moving into build/ like this
cmake -S . -B build/ cmake --build build/ ./build/heat_seq 100 100 100 0
Remark that you can precise the compiler. There are different ways to give it, either using environments variables
cd build make clean CC=gcc cmake .. && make
or using a CMake variable CMAKE_<LANG>_COMPILER.
make clean
cmake -D CMAKE_C_COMPILER=clang .. && make
You may want to execute the generated driver, cf. the problem to solve.
./heat_seq # will print how to use the program ./heat_seq 100 100 100 0 # will compute the heat propagation on a 100x100 grid space and for 100 timesteps
2.3.1 Configuration through command line
The “cmake” executable is the CMake command-line interface. It may be used to configure projects in scripts.
Many options are availables, see
cmake --help
For example, to edit the build system generator use -G, e.g
cmake -G "Unix Makefiles" ..
Project configuration settings may be specified on the command line with the -D option. Different kind of options can be considered
- native CMake options, here a (non-exhaustive) list of
very useful ones
- CMAKE_<LANG>_COMPILER: compiler executable to use, <LANG> can be C, CXX, Fortran
- CMAKE_<LANG>_FLAGS: compiler flags for compiling <LANG> sources.
- CMAKE_EXE_LINKER_FLAGS, CMAKE_SHARED|STATIC_LINKER_FLAGS, CMAKE_MODULE_LINKER_FLAGS: linker flags
- CMAKE_BUILD_TYPE: to control the build type (Debug, Release, etc)
- BUILD_SHARED_LIBS: ON|OFF to enable or not generation of shared/dynamic libraries
- CMAKE_INSTALL_PREFIX: where the project will be installed through "make install"
- custom project options
- the name and the action of the option is related to the project, so something like: BAR_ENABLE_FEATURE, BAR_USE_LIBFOO, FOO_LIBDIR, etc
Try the following in $CMAKE_HANDS_ON/exe1_usecmake/build
:
cmake .. -DCMAKE_C_COMPILER=/usr/bin/gcc -DCMAKE_BUILD_TYPE=Debug -DCMAKE_C_FLAGS_DEBUG="-g -W -Wall" -DCMAKE_INSTALL_PREFIX=$PWD/install
It is possible to get detailed about build commands with the parameter VERBOSE=1 when invoking "make", e.g.
make -j4 VERBOSE=1
You should see that gcc is used and notice the extra compiler flags "-g -W -Wall".
When trying to build a new project whose internal custom options are unknown, it may be convenient to use "-L[A][H]" option in order to get the status of "CACHE" variables used by the project to tune the build. For example, in our simple project here there is an option to turn on the MPI version:
cmake .. -LH
The "H" parameter allows to display help for each variable, possible values may be given or guessed thanks to the type. Here our HEAT_USE_MPI option is a "BOOL" so that the feature is enabled|disabled with ON|OFF:
cmake .. -DHEAT_USE_MPI=ON && make
If CMake has properly automatically detected MPI in your environment you can try to execute the executable linked with MPI:
mpiexec -np 2 ./heat_par 100 100 100 1 2 0
2.3.2 Configuration through an interface
CMake provides optional tools such that Curses and Qt interfaces, cf. requirements.
To use the Curses interface, call the "ccmake" command
ccmake .. # or with make edit_cache
To use the Qt based interface, call the "cmake-gui" command
cmake-gui ..
It is convenient the first time we work on a project to discover main options but it becomes cumbersome when used often to configure because several steps are required to build the project:
- open the interface,
- change the options,
- clic on configure,
- clic on generate,
- then build,
while one command line that can be scripted is far more efficient.
3 Create a simple CMake project from scratch
Each build project contains a CMakeLists.txt file in every directory that controls the build process of files into the directory. The CMakeLists.txt file has one or more commands in the form COMMAND (args…), with COMMAND representing the name of each command and args the list of arguments, each separated by white space. While there are many built-in rules for compiling the software libraries (static and dynamic) and executables, there are also provisions for custom build rules. Some build dependencies can be determined automatically. Advanced users can also create and incorporate additional makefile generators to support their specific compiler and OS needs.
3.1 A few words about the syntax
CMake is a "scripting language"; you can process CMake script files with -P option
cmake -P script.cmake
- The names of commands are case insensitive, set() is equivalent to SET()
Variables always contain strings. Sometimes, we use the string to store a boolean, a path to a file, an integer, or a list. Variables are defined usually with set(var value)
set(VAR 0) # 0 is considered as a string and this is equivalent to set(VAR "0") set(VAR "hello") # "hello" is consider as a string and this is equivalent to set(VAR hello) # the name of variables is case-sensitive, here VAR contains 0, var 1 set(VAR 0) set(var 1)
To print message in the standard output message() is used
message("Hello") message(STATUS "It works fine for now") message(WARNING "Something is missing here") message(FATAL_ERROR "I can't build in this situation")
To access the value of a variable, you perform a substitution. To perform a substitution, use the syntax ${variablename}
set(FIBO "0 1 1 2 3 5") # FIBO contains the string "0 1 1 2 3 5" # you can also set some lists, structure on which you will be able to iterate over the elements set(FIBO 0 1 1 2 3 5) # FIBO contains the list of strings "0;1;1;2;3;5" # try to see it with message message(${FIBO}) # iterate on the list foreach(_val ${FIBO}) message("${_val}") endforeach()
- To make a variable undefined use unset(var) or set(var)
- Note that there exist a set of already existing, and potentialy defined, CMake variables, whose name usually begins with CMAKE_ see the documentation and some common useful cmake variables
- Each directory added with add_subdirectory or each funtion declared with function creates a new scope. The new child scope inherits all variable definitions from its parent scope. Variable assignments in the new child scope with the set command will only be visible in the child scope unless the PARENT_SCOPE option is used.
CMake has control statements: foreach, if, while (read the documentation)
# compare numbers set( number 4 ) # if ${number} is greater than 10 if( number GREATER 10 ) message( "The number ${number} is too large." ) endif() # test if a variable is defined (not empty) set(VAR "hello") if (VAR) message("Let's play with ${VAR}") else() message("Variable VAR whose value is ${VAR} is undefined") endif()
- Booleans: CMake if considers an empty string, "0", "FALSE", "OFF", "NO", or any string ending in "-NOTFOUND" to be false.
CMake provides a list command that performs basic operations with list
set(VAR "hello") list(APPEND VAR " world") # add an element to the list, last position list(GET VAR 0 VAR_0) # get an element at a specific location (first here, O-indexed) list(INSERT VAR 1 ${VAR}) # insert a new element in second list(REMOVE_DUPLICATES VAR) # remove duplicated entries
function and macro can be declared as an additional CMake command The main difference is that function creates a new scope while macro does not.
One can call macros and functions with additional arguments and there exist native keywords to manipulate them.
Try this in a cmake script
function(bar arg1 arg2) message(STATUS "Arguments are: ${arg1} and ${arg2}") message(STATUS "Number of arguments: ${ARGC}") message(STATUS "First argument: ${ARGV0}") message(STATUS "Second argument: ${ARGV1}") message(STATUS "All arguments: ${ARGV}") message(STATUS "All optional arguments: ${ARGN}") set(list_var "${ARGV}") foreach(_var IN LISTS list_var) message(STATUS "${_var}") endforeach() set(${arg1} "one") endfunction() message(STATUS "--bar--") bar(one two three four) message(STATUS "one: ${one}")
macro(bar arg1 arg2) message(STATUS "Arguments are: ${arg1} and ${arg2}") message(STATUS "Number of arguments: ${ARGC}") message(STATUS "First argument: ${ARGV0}") message(STATUS "Second argument: ${ARGV1}") message(STATUS "All arguments: ${ARGV}") message(STATUS "All optional arguments: ${ARGN}") set(list_var "${ARGV}") foreach(_var IN LISTS list_var) message(STATUS "${_var}") endforeach() set(${arg1} "one") endmacro() message(STATUS "--bar--") bar(one two three four) message(STATUS "one: ${one}")
Guideline: Create macros to wrap commands that have output parameters, otherwise create a function.
include can be used to import other CMake scripts (.cmake)
set( command message ) # we want t file( WRITE temp "${command}( hi )" ) # writes "message( hi )" to temp file (in the same directory than the current CMakeLists.txt) include( temp ) # include the cmake script containing the message command file( REMOVE temp ) # temp file can be removed
To get details about the CMake syntax the reader could refer to the Wiki page section "The CMake Language".
3.2 First main CMakeLists.txt, build an executable, link to an external library
A minimal CMake configuration requires a "CMakeLists.txt" file containing the list of source files and the name of the targets (here executables) to build. The command invocation to build an executable is add_executable
add_executable(hello_world hello_world.c)
Please move to $CMAKE_HANDS_ON/exe2_buildexe/
and build the project
cd $CMAKE_HANDS_ON/exe2_buildexe/ && mkdir -p build && cd build && cmake .. && make
Open the CMakeLists.txt file, it looks like
project (Heat) # this is optional add_executable(heat_seq heat_seq.c heat.c mat_utils.c heat.h mat_utils.h) target_link_libraries(heat_seq m)
Pay attention to the target_link_libraries here which indicates to link with the libm external library because the code contains some calls to mathematical functions. Note "libm" (.a, .lib, .so, .dyld, …) should be available in the environment in standard system paths.
The source files could also have been collected with the file command with the GLOB attribute
file(GLOB heat_sources *.c *.h) add_executable(heat_seq ${heat_sources})
but this is not recommended because CMake will lose its ability to regenerate when some specific source files are added or removed, see documentation.
Guideline: avoid using file(GLOB … to collect source files to build, prefer explicit lists e.g. set(SRC src1 src2).
3.3 Build a library and manage sub-directories
For a large project with many source files it is normal to organize files into different sub-directories. New CMakeLists.txt files could be created in order to define, following the structure of the project, the different objects to build.
Please move to $CMAKE_HANDS_ON/exe3_buildlib/
and build the project
cd $CMAKE_HANDS_ON/exe3_buildlib/ && mkdir -p build && cd build && cmake .. && make
Remark that we have moved the code related to the computation of one iteration of the problem in include/ and lib/. The idea is to create a library named "heat" containing this portion of the program.
To build a shared library, the variable BUILD_SHARED_LIBS should be used
cmake .. -DBUILD_SHARED_LIBS=ON && make
In CMake the library is defined with add_library. Here it is declared where the source file related to it states (this is not mandatory!), see the CMakeLists.txt file in lib/
add_library(heat heat.c)
If it is required to build "static" or "shared" object only, the keyword STATIC, SHARED may be used in add_library
add_library(heat STATIC heat.c) add_library(heat SHARED heat.c)
To tell CMake to process other CMake files in sub-directories, the sub-directories must be declared with add_subdirectory, e.g.
# libheat add_subdirectory(lib) # heat_seq exe add_executable(heat_seq heat_seq.c mat_utils.h mat_utils.c) # dependency to libheat target_link_libraries(heat_seq heat) # dependency to libm (math) find_library(M_LIBRARY m) mark_as_advanced(M_LIBRARY) if (M_LIBRARY) target_link_libraries(heat_seq ${M_LIBRARY}) endif()
# in dir lib/ add_library(heat heat.c) target_include_directories(heat PUBLIC ${PROJECT_SOURCE_DIR}/include)
In this example we also have used other useful commands:
- enable_language: to initialize some sets of CMake variables related to one language (C, CXX, Fortran, CUDA …)
- target_include_directories: this is equivalent to "-I…", this adds a directory to the list of preprocessor include file search directories
- find_library: to find a library already installed on the system
One could also use target_sources to add sources to a target. This could be used to add sources depending on the operating system for example
target_sources(heat_seq PRIVATE mat_utils.c)
Guideline: avoid to use cmake commands like include_directories, link_directories, link_libraries and add_compile_options, because they work at the directory level and apply to all entities defined in scope, meaning to all sub-directories as well. Prefere adding properties on specific targets.
3.4 Add options
It is often useful in a project to tune parameters of the build to enable/disable some features for instance or specify by hand the path to an external dependency. This can be done either by the command option (only ON|OFF) or by setting "CACHE" variables (of type FILEPATH, PATH, STRING, BOOL) which are by nature global variables known from everywhere in the project and persistent over subsequent cmake configurations.
Please move to $CMAKE_HANDS_ON/exe4_options/
and build the project
specifying HEAT_USE_MPI=ON
cd $CMAKE_HANDS_ON/exe4_options/ && mkdir -p build && cd build && cmake .. -DHEAT_USE_MPI=ON && make
Look into the CMakeLists.txt to see the option added
option(HEAT_USE_MPI "Build MPI executable" OFF)
Remark it is set to OFF by default. Note also that the following is equivalent
set(HEAT_USE_MPI ON CACHE BOOL "Build MPI executable")
The "CACHE" keyword allows to define a variable that:
- has a global scope (known from all the subtree)
- keeps the last defined value across multiple "configure" steps
- can be explicitly changed with -D to command line and interfaces
It may be used to let the users act on not boolean options, for example
set(HEAT_ORDER "2" CACHE STRING "Order of the numerical scheme: 1, 2 (default) or 3")
We can add constraints on a variable to limit possibilities
set( HEAT_ORDER_LIST "1;2;3") set( HEAT_ORDER "2" CACHE STRING "Choose the scheme order in ${HEAT_ORDER_LIST}") set_property(CACHE HEAT_ORDER PROPERTY STRINGS ${HEAT_ORDER_LIST})
3.5 Install/uninstall process
Many scientific applications are used to validate new numerical and algorithmic techniques and are often re-compiled to make experiments. But many programs also offer a set of services, features that can be used directly by users (executables e.g. gcc, scripts, …) or by upper-level programs (e.g. blas, boost). In the latter situation the program must be installed somewhere on the system after the build process so that only required files (executables, libraries, header files, etc) are copied for users. This can be done with the target install.
Please move to $CMAKE_HANDS_ON/exe5_install/
and build the project
cd $CMAKE_HANDS_ON/exe5_install/ && mkdir -p build && cd build && cmake .. -DHEAT_USE_MPI=ON && make
One can see in the root CMakeLists.txt the install, usage of this command is detailed in here, command used like this
install(TARGETS heat_seq DESTINATION bin) set_target_properties(heat PROPERTIES PUBLIC_HEADER "${PROJECT_SOURCE_DIR}/include/heat.h") install(TARGETS heat DESTINATION lib PUBLIC_HEADER DESTINATION include)
This tells CMake to copy the files related to the target in
CMAKE_INSTALL_PREFIX, a native CMake variable. Here the
header file heat.h, the executable heat_seq and the library heat
are copied in CMAKE_INSTALL_PREFIX/include
,
CMAKE_INSTALL_PREFIX/bin
, CMAKE_INSTALL_PREFIX/lib
respectively
when the user invokes make install
.
cmake .. -DCMAKE_INSTALL_PREFIX=$PWD/install
make install
ls -l install/
Check here that the files have been copied into the install directory.
In the case your executables are linked with shared libraries there is the possibility to add paths in the RPATH in order to set the directories where to find shared libraries.
set(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib")
Note that it is possible to install any file or directory and not only targets, e.g.
install(DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/doc DESTINATION share/foo/doc) install(FILES libfoo.pc DESTINATION lib/pkgconfig)
One limitation of CMake is that there is no uninstall target by default, cf. previous discussion. This problem can be bypassed by defining the uninstall target of the project.
Try the following
make uninstall
This method has been picked up from here and seems more to be a trick than a clean solution.
Notice the use of:
- add_custom_target: to define a target that executes a custom
command such as rm, here it executes our custom cmake script
cmake_uninstall.cmake
.
4 Advanced usage
4.1 Drive the build with well chosen compilers and flags
We have seen in the section use CMake that the user can define compilers, compiler flags and linker flags by hand through the command line or through the interfaces. The CMake variables are the following:
- CMAKE_<LANG>_COMPILER: compiler executable to use, <LANG> can be C, CXX, CUDA, Fortran, ASM
- CMAKE_<LANG>_FLAGS: compiler flags for compiling <LANG> sources.
- CMAKE_EXE_LINKER_FLAGS, CMAKE_SHARED|STATIC_LINKER_FLAGS, CMAKE_MODULE_LINKER_FLAGS: linker flags
These variables are internal to CMake and can be either set by the developer in the configuration CMakeLists.txt files in the project or, if nothing is specified in the project, set by default by CMake during the configuration step (compilers are detected automatically and no specific flags are chosen). Of course these variables can be set or changed by hand for specific needs as we have seen in the section use CMake.
Let's give some hints to force compilers, flags and definitions in configuration files for a project. Remember that to see the details about the build commands, the parameter VERBOSE=1 can be used when using "make" or alternatively the variable CMAKE_VERBOSE_MAKEFILE may be set to ON.
cmake .. -D CMAKE_VERBOSE_MAKEFILE=ON
Force a compiler, e.g.
cmake .. -D CMAKE_C_COMPILER="icc"
Add some compiler flags, e.g.
cmake .. -D CMAKE_C_FLAGS="-W -Wall"
Add some linker flags, e.g.
cmake .. -D CMAKE_SHARED_LINKER_FLAGS="-undefined dynamic_lookup"
The CMake variables we have seen are user's interface variables which are added to the one defined by developers in the CMakeLists.txt files. To set some compile properties directly in the CMakeLists.txt, for a long term usage, there exist some macros that set or update the options for targets and source files.
Here some command to set options globally for all sources and targets defined after the call
- add_definitions: adds -D define flags to the compilation of source files for all the targets
- include_directories: idem header files (-I)
- add_compile_options: idem compile flags
add_definitions(-DADD_) include_directories(${MPI_INCLUDE_PATH}) add_compile_options(-W;-Wall)
Guideline: avoid to set compilations flags globally with these commands, prefer setting flags on specific targets with the macros presented hereafter.
Specific flags, definitions or other properties can also be defined for some precise objects of the project (in a directory only, for a target or some source files).
- set_property: very high level macro to set some options, flags, definitions
- set_target_properties: set properties for targets
- set_source_files_properties: set properties for source files
- target_include_directories: add include directories to a target (include_directories command applies to all targets)
- target_compile_definitions: add compile definitions to a target (add_definitions command applies to all targets)
- target_compile_options: add compile options to a target (compile_options command applies to all targets)
- target_link_libraries: specify libraries or flags to use when linking a given target
- target_link_options: options to the link step for an executable, shared library or module library target
See the properties that can be tuned for each kind of object here.
set_property(DIRECTORY PROPERTY COMPILE_OPTIONS ${flags_to_add}) set_target_properties(${TARGET} PROPERTIES COMPILE_FLAGS ${flags_to_add}) set_source_files_properties(${SOURCE_FILES} PROPERTIES COMPILE_FLAGS ${flags_to_add}) target_include_directories(${TARGET} PUBLIC /path/to/some/headers;/other/path) target_compile_definitions(${TARGET} PUBLIC ADEF) target_compile_options(${TARGET} PUBLIC -std=c++11) target_link_libraries(${TARGET} m)
4.2 How to detect external libraries
Many projects rely on external libraries that are not necessarily vital for an operating system (e.g. MPI, CUDA, etc). This implies that the project should provide a clean way to detect the dependency it relies on and do not try to build if this dependency is not found on the system (and in the later case should give some hints to install it). This subject is presented in the Wiki here.
In the previous exercise, $CMAKE_HANDS_ON/exe5_install/
, you can
open the CMakeLists.txt file and check the line where we look for
the MPI library (to handle parallel tasks on distributed memory
architectures).
find_package(MPI)
This command will check if MPI can be found on the system, in some
standard paths or in the environment, and will set some variables
(e.g. MPI_C_LIBRARIES) with the required information to link
with/use the dependency. The command relies on some module files
that are either released with CMake,
e.g. /usr/share/cmake-3.10/Modules/FindMPI.cmake
, or providen from
outside. For example the project could provide some custom modules
with some specific rules to find the dependency. Note that
additional module directories must be given by setting the variable
CMAKE_MODULE_PATH, e.g.
# add this directory to look for cmake modules list(APPEND CMAKE_MODULE_PATH "${PROJECT_SOURCE_DIR}/cmake_modules/") # include cmake code of my custom modules MyCMakeModule.cmake include(MyCMakeModule)
The module files released in CMake (in
/usr/share/cmake-3.10/Modules
) are written by the community so that
there can be many differences between each and developers should
read the usage of the module (head of the file) carrefully to learn
how to use it and understand its behaviour. Have a look to
cmake --help-module-list
in order to list all module files available in your current environment. Usage can also be seen, e.g.
cmake --help-module FindMPI
The common API of the find_package command is the following
find_package(<package> [version] [EXACT] [QUIET] [MODULE] [REQUIRED] [[COMPONENTS] [components...]] [OPTIONAL_COMPONENTS components...] [NO_POLICY_SCOPE])
- version: to specify a version, e.g. "at least 1.0" or "exactly 1.2"
- QUIET: to disable module messages
- REQUIRED: to end in failure if the package is not found
- COMPONENTS: to specify "I want this feature in the package" or "I need this package to be linked with libfoo", this strongly depends on the module maintainers choice
This macro will certainly give variables as outputs with the
results of the detection, e.g. in MPI_C_LIBRARIES,
MPI_C_INCLUDE_DIRS. More recently "Imported targets" may be defined
and ease the linking process because all properties are given to
the dependent when using target_link_libraries
.
For example instead of using
target_include_directories(heat_par PRIVATE ${MPI_C_INCLUDE_DIRS}) target_link_libraries(heat_par ${MPI_C_LIBRARIES})
One can use directly
target_link_libraries(heat_par PRIVATE MPI::MPI_C) # here all important properties are given for users: compile flags and definitions, link flags, ...
If the detection module does not exist yet for the library you are looking at you can either write your own FindFOO.cmake (see next section) or use the following CMake macros:
- find_path: to find directories containing a specific file (useful for headers or fortran modules)
- find_library: to find libraries with absolute paths.
4.3 Write your own FindFOO.cmake
The modules existing in CMake concerns a set of libraries and tools for which the effort has been done by the community to help users to automatically detect the program. Of course not all libraries can be found that way and developers often have to write the ones missing for their specific project. Notice it is higly discouraged to write find modules for packages that themselves build with CMake. Instead it is better to provide a CMake package configuration file with the package itself like we will see in the next section.
The goal is to detect the set of files (e.g. headers) and objects (e.g. libraries) that an application will need to use the program. The CMake code should be written in module file whose name starts with Find and ends with the package name followed by .cmake.
Let's give some hints with an example, the hwloc library.
See for example the file
$CMAKE_HANDS_ON/exe6_findpackage/cmake_modules/FindHWLOC.cmake
and how
to use it in $CMAKE_HANDS_ON/exe6_findpackage/CMakeLists.txt
.
You can verify that the executable heat_seq is properly linked to hwloc and that it calls the API (see in the file heat_seq.c the main function)
cd $CMAKE_HANDS_ON/exe6_findpackage/ && mkdir -p build && cd build && cmake .. && make VERBOSE=1
Lets describe a bit what is written in the Find module.
First, it should start with a "Usage" section explaining how to use the module
### # # - Find HWLOC include dirs and libraries # Use this module by invoking find_package with the form: # find_package(HWLOC # [REQUIRED]) # Fail with error if hwloc is not found # # This module finds headers and hwloc library. # Results are reported in variables: # HWLOC_FOUND - True if headers and requested libraries were found # HWLOC_INCLUDE_DIRS - hwloc include directories # HWLOC_LIBRARY_DIRS - Link directories for hwloc libraries # HWLOC_LIBRARIES - hwloc component libraries to be linked # # The user can give specific paths where to find the libraries adding cmake # options at configure (ex: cmake path/to/project -DHWLOC_DIR=path/to/hwloc): # HWLOC_DIR - Where to find the base directory of hwloc # HWLOC_INCDIR - Where to find the header files # HWLOC_LIBDIR - Where to find the library files # The module can also look for the following environment variables if paths # are not given as cmake variable: HWLOC_DIR, HWLOC_INCDIR, HWLOC_LIBDIR
The module should provide a way to let the user give some exotic paths directly by hand through CMake variables or through environment variables
set(ENV_HWLOC_DIR "$ENV{HWLOC_DIR}") set(ENV_HWLOC_INCDIR "$ENV{HWLOC_INCDIR}") set(ENV_HWLOC_LIBDIR "$ENV{HWLOC_LIBDIR}") set(HWLOC_GIVEN_BY_USER "FALSE") if ( HWLOC_DIR OR ( HWLOC_INCDIR AND HWLOC_LIBDIR) OR ENV_HWLOC_DIR OR (ENV_HWLOC_INCDIR AND ENV_HWLOC_LIBDIR) ) set(HWLOC_GIVEN_BY_USER "TRUE") endif()
Some programs give directly the required information thanks to the pkg-config tool. When a program relies on pkg-config the module should try to use these information first if the paths are not given by the user.
# Optionally use pkg-config to detect include/library dirs (if pkg-config is available) # ------------------------------------------------------------------------------------- include(FindPkgConfig) find_package(PkgConfig QUIET) if( PKG_CONFIG_EXECUTABLE AND NOT HWLOC_GIVEN_BY_USER ) pkg_search_module(HWLOC hwloc) if (NOT HWLOC_FIND_QUIETLY) if (HWLOC_FOUND AND HWLOC_LIBRARIES) message(STATUS "Looking for HWLOC - found using PkgConfig") else() message("Looking for HWLOC - not found using PkgConfig." "Perhaps you should add the directory containing hwloc.pc to" "the PKG_CONFIG_PATH environment variable.") endif() endif() endif( PKG_CONFIG_EXECUTABLE AND NOT HWLOC_GIVEN_BY_USER )
If the package is not found we will try to find it on the system or in the paths given by the user. Note in the following the use of find_path to look for specific files and find_library to look for specific libraries. Note also that we search in priority into directories given by the user and then into default directories depending on the system.
# Looking for include # ------------------- # Add system include paths to search include # ------------------------------------------ unset(_inc_env) if(ENV_HWLOC_INCDIR) list(APPEND _inc_env "${ENV_HWLOC_INCDIR}") elseif(ENV_HWLOC_DIR) list(APPEND _inc_env "${ENV_HWLOC_DIR}") list(APPEND _inc_env "${ENV_HWLOC_DIR}/include") list(APPEND _inc_env "${ENV_HWLOC_DIR}/include/hwloc") else() if(WIN32) string(REPLACE ":" ";" _inc_env "$ENV{INCLUDE}") else() string(REPLACE ":" ";" _path_env "$ENV{INCLUDE}") list(APPEND _inc_env "${_path_env}") string(REPLACE ":" ";" _path_env "$ENV{C_INCLUDE_PATH}") list(APPEND _inc_env "${_path_env}") string(REPLACE ":" ";" _path_env "$ENV{CPATH}") list(APPEND _inc_env "${_path_env}") string(REPLACE ":" ";" _path_env "$ENV{INCLUDE_PATH}") list(APPEND _inc_env "${_path_env}") endif() endif() list(APPEND _inc_env "${CMAKE_C_IMPLICIT_INCLUDE_DIRECTORIES}") list(REMOVE_DUPLICATES _inc_env) # set paths where to look for set(PATH_TO_LOOK_FOR "${_inc_env}") # Try to find the hwloc header in the given paths # ------------------------------------------------- # call cmake macro to find the header path if(HWLOC_INCDIR) set(HWLOC_hwloc.h_DIRS "HWLOC_hwloc.h_DIRS-NOTFOUND") find_path(HWLOC_hwloc.h_DIRS NAMES hwloc.h HINTS ${HWLOC_INCDIR}) else() if(HWLOC_DIR) set(HWLOC_hwloc.h_DIRS "HWLOC_hwloc.h_DIRS-NOTFOUND") find_path(HWLOC_hwloc.h_DIRS NAMES hwloc.h HINTS ${HWLOC_DIR} PATH_SUFFIXES "include" "include/hwloc") else() set(HWLOC_hwloc.h_DIRS "HWLOC_hwloc.h_DIRS-NOTFOUND") find_path(HWLOC_hwloc.h_DIRS NAMES hwloc.h HINTS ${PATH_TO_LOOK_FOR} PATH_SUFFIXES "hwloc") endif() endif() mark_as_advanced(HWLOC_hwloc.h_DIRS) # Add path to cmake variable # ------------------------------------ if (HWLOC_hwloc.h_DIRS) set(HWLOC_INCLUDE_DIRS "${HWLOC_hwloc.h_DIRS}") else () set(HWLOC_INCLUDE_DIRS "HWLOC_INCLUDE_DIRS-NOTFOUND") if(NOT HWLOC_FIND_QUIETLY) message(STATUS "Looking for hwloc -- hwloc.h not found") endif() endif () # Looking for lib # --------------- # Add system library paths to search lib # -------------------------------------- unset(_lib_env) if(ENV_HWLOC_LIBDIR) list(APPEND _lib_env "${ENV_HWLOC_LIBDIR}") elseif(ENV_HWLOC_DIR) list(APPEND _lib_env "${ENV_HWLOC_DIR}") list(APPEND _lib_env "${ENV_HWLOC_DIR}/lib") else() if(WIN32) string(REPLACE ":" ";" _lib_env "$ENV{LIB}") else() if(APPLE) string(REPLACE ":" ";" _lib_env "$ENV{DYLD_LIBRARY_PATH}") else() string(REPLACE ":" ";" _lib_env "$ENV{LD_LIBRARY_PATH}") endif() list(APPEND _lib_env "${CMAKE_C_IMPLICIT_LINK_DIRECTORIES}") endif() endif() list(REMOVE_DUPLICATES _lib_env) # set paths where to look for set(PATH_TO_LOOK_FOR "${_lib_env}") # Try to find the hwloc lib in the given paths # ---------------------------------------------- # call cmake macro to find the lib path if(HWLOC_LIBDIR) set(HWLOC_hwloc_LIBRARY "HWLOC_hwloc_LIBRARY-NOTFOUND") find_library(HWLOC_hwloc_LIBRARY NAMES hwloc HINTS ${HWLOC_LIBDIR}) else() if(HWLOC_DIR) set(HWLOC_hwloc_LIBRARY "HWLOC_hwloc_LIBRARY-NOTFOUND") find_library(HWLOC_hwloc_LIBRARY NAMES hwloc HINTS ${HWLOC_DIR} PATH_SUFFIXES lib lib32 lib64) else() set(HWLOC_hwloc_LIBRARY "HWLOC_hwloc_LIBRARY-NOTFOUND") find_library(HWLOC_hwloc_LIBRARY NAMES hwloc HINTS ${PATH_TO_LOOK_FOR}) endif() endif() mark_as_advanced(HWLOC_hwloc_LIBRARY) # If found, add path to cmake variable # ------------------------------------ if (HWLOC_hwloc_LIBRARY) get_filename_component(hwloc_lib_path ${HWLOC_hwloc_LIBRARY} PATH) # set cmake variables set(HWLOC_LIBRARIES "${HWLOC_hwloc_LIBRARY}") set(HWLOC_LIBRARY_DIRS "${hwloc_lib_path}") else () set(HWLOC_LIBRARIES "HWLOC_LIBRARIES-NOTFOUND") set(HWLOC_LIBRARY_DIRS "HWLOC_LIBRARY_DIRS-NOTFOUND") if(NOT HWLOC_FIND_QUIETLY) message(STATUS "Looking for hwloc -- lib hwloc not found") endif() endif ()
After the files have been found, the information that will be transmitted to the user should be tested. Here we try to link and check that a basic symbol exist and can be called from a program. Notice the use of check_function_exists to check if a C function can be linked.
# check a function to validate the find if(HWLOC_LIBRARIES) # set required libraries for link set(CMAKE_REQUIRED_LIBRARIES "${HWLOC_LIBRARIES}") # test link unset(HWLOC_WORKS CACHE) include(CheckFunctionExists) check_function_exists(hwloc_topology_init HWLOC_WORKS) mark_as_advanced(HWLOC_WORKS) if(NOT HWLOC_WORKS) if(NOT HWLOC_FIND_QUIETLY) message(STATUS "Looking for hwloc : test of hwloc_topology_init with hwloc library fails") message(STATUS "Check in CMakeFiles/CMakeError.log to figure out why it fails") endif() endif() endif(HWLOC_LIBRARIES)
Create a custom target to ease libraries linking avoiding to use lists directly.
if(NOT TARGET hwloc::hwloc) # initialize imported target add_library(hwloc::hwloc INTERFACE IMPORTED) if (HWLOC_INCLUDE_DIRS) set_target_properties(hwloc::hwloc PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${HWLOC_INCLUDE_DIRS}") endif() if (HWLOC_LIBRARY_DIRS) set_target_properties(hwloc::hwloc PROPERTIES INTERFACE_LINK_DIRECTORIES "${HWLOC_LIBRARY_DIRS}") endif() if (HWLOC_LIBRARIES) set_target_properties(hwloc::hwloc PROPERTIES INTERFACE_LINK_LIBRARIES "${HWLOC_LIBRARIES}") endif() if (HWLOC_CFLAGS_OTHER) set_target_properties(hwloc::hwloc PROPERTIES INTERFACE_COMPILE_OPTIONS "${HWLOC_CFLAGS_OTHER}") endif() if (HWLOC_LDFLAGS_OTHER) set_target_properties(hwloc::hwloc PROPERTIES INTERFACE_LINK_OPTIONS "${HWLOC_LDFLAGS_OTHER}") endif() endif (NOT TARGET hwloc::hwloc)
Finally we validate the find process to set HWLOC_FOUND to true or false depending on the status of some variables. Here for example, if HWLOC_LIBRARIES or HWLOC_WORKS are not well defined then the package will be considered as not found.
# check that HWLOC has been found # ------------------------------- include(FindPackageHandleStandardArgs) find_package_handle_standard_args(HWLOC DEFAULT_MSG HWLOC_LIBRARIES HWLOC_WORKS)
Some references:
4.4 Provide package configuration
Writing a module to be used with find_package is useful especially when the dependency was not built with CMake. For dependencies that rely on CMake, configuration files may be providen and used very efficiently to propagate all required information about the build and installation.
In the exercise 7 (exe7_exportconf) we provide a configuration
file, $CMAKE_HANDS_ON/exe7_exportconf/lib/HeatConfig.cmake
, which
gives the definitions of targets exported for users.
cd $CMAKE_HANDS_ON/exe7_exportconf/ && mkdir -p build && cd build && cmake .. -DCMAKE_INSTALL_PREFIX=$PWD/install
This file is copied in lib/cmake/HeatConfig.cmake
in the
installation path after make install
, this to give the appropriate
information to users.
make install
This means that another cmake project can use these information to use/link to the Heat project
# heat_ext exe add_executable(heat_ext heat_ext.c) # find libheat find_package(Heat REQUIRED) # dependency to libheat target_link_libraries(heat_ext heat::heat)
and giving the path to the configuration file
cmake .. -DCMAKE_PREFIX_PATH=where/is/installed/heat/lib/cmake # or also cmake .. -DHeat_DIR=where/is/installed/heat/lib/cmake/heat
See in the lib/CMakeLists.txt
file how the "heat" target can be
exported for users
# define target information to export install(TARGETS heat EXPORT heatTargets DESTINATION lib PUBLIC_HEADER DESTINATION include) # rule to generate the file containing the targets information install(EXPORT heatTargets FILE HeatTargets.cmake NAMESPACE heat:: DESTINATION lib/cmake/heat) # install the config file containing all information (targets + some variables like the version) install(FILES "HeatConfig.cmake" DESTINATION lib/cmake/heat)
Some references:
- https://gitlab.kitware.com/cmake/community/wikis/doc/Tutorials#cmake-packages
- https://gitlab.kitware.com/cmake/community/wikis/doc/tutorials/How-to-create-a-ProjectConfig.cmake-file
See a more complex example:
4.5 Drive the build of external projects (ExternalProject_Add)
In some situations the detection of dependencies is not robust enough and developers prefer to drive the installation (a precise version, a custom configuration), of dependencies directly from their CMake project. This can be done in the CMakeLists.txt files thanks to the externalproject_add command. The possibilities are many, cf. documentation
- download step: download a tarball or pull from an SCM repository
- update step: svn up, apply patches
- configure step: drive the configuration with a command or with CMake variables
- build step: choose where to build and "make" parameters
- install step: choose where to install and "make install" parameters
- test step
- log, …
Example
ExternalProject_Add(foo SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/foo" CMAKE_ARGS ${foo_cmake_args} BUILD_COMMAND "make" "-j4" )
4.6 Test using CTest
CTest is a testing tool distributed as a part of CMake. It can be used to automate updating (using GIT for example), configuring, building, testing, performing memory checking, performing coverage, and submitting results to a CDash.
4.6.1 Add test
You need the CMake command enable_testing() to be able to define tests through the add_test command
add_test(NAME <name> COMMAND <command> [<arg>...] [CONFIGURATIONS <config>...] [WORKING_DIRECTORY <dir>])
Here a simple example with the NAME and COMMAND only, CTest checks that the program ends normally
enable_testing() add_test(heat_seq_usage ./heat_seq)
You can add conditions based on regular expressions to validate the test
set_tests_properties(test1 [test2...] PROPERTIES PASS_REGULAR_EXPRESSION "TestPassed;All ok") set_tests_properties(test1 [test2...] PROPERTIES FAIL_REGULAR_EXPRESSION "[^a-z]Error;ERROR;Failed")
A precise example
set_tests_properties(heat_seq_usage PROPERTIES PASS_REGULAR_EXPRESSION "Usage*")
Please move to $CMAKE_HANDS_ON/exe8_ctest/
and build the project
cd $CMAKE_HANDS_ON/exe8_ctest/ && mkdir -p build && cd build && cmake .. -DHEAT_USE_MPI=ON && make
After the project is built, one can execute the "make test" command to run the tests defined in CMakeLists.txt files
make test # or ctest
4.6.2 CTest options
We give here some useful options that can used with the ctest excutable
- ctest -V : verbose mode, see actual execution commands
- ctest -N : to list available tests without running it
- ctest -R expr : to execute tests containing “expr” in their name
- ctest -E expr : to omit tests containing “expr” in their name
- ctest -I … : to filter tests through a numbering
- ctest –timeout t : stop tests after t seconds
Complete list of options for "ctest" command available here.
The ability to filter tests by their name is important to be able to define different kind of tests, unit tests, profiling, benchmarking, integration tests, etc
ctest -R "usage" # runs all tests whose name contains "usage" ctest -R "par" # runs all tests whose name contains "par" ctest -R "usage|par" # runs all tests with "usage" or "par" in their name ctest -E "usage" # runs all tests except those with "usage" in their name
4.6.3 Submit tests to CDash
The execution of tests is a job which should be made automatic and scheduled through scripting. CTest+CDash tools help in managing this and allow to further published the tests online on a CDash server.
Main features of CTest+CDash:
- automatize the process "configure + build + test + publication"
letting the choice to users to execute all four phases each time
or some well chosen only
- the idea is to launch one benchmark every day (e.g. during the night)
- using a crontab for instance or configuring a job in your favourite continuous integration framework
- manage several configurations
- e.g. Debug/Release, with and without MPI
- Submit results to a CDash web server to conveniently analyze and share results
- follow regressions and fixes concerning
- compilation (check if success, if warning, with/without enabling some features)
- tests (check what succeeds/fails, code coverage)
- performances (time, memory footprint)
To be able to use CTest to submit results two things are required
- include(CTest) must be written in the project and
- a configuration file, CTestConfig.cmake should be given in the project in the same directory tests are defined
Check the CMakeLists.txt and CTestConfig.cmake of the exercise 6
cd $CMAKE_HANDS_ON/exe8_ctest/
In the CTestConfig.cmake we define the name of the CDash project, the server location, the build name collecting some information of the CMake configuration.
To execute the test chain, try the following
cd $CMAKE_HANDS_ON/exe8_ctest/ && mkdir -p build && cd build cmake .. -DHEAT_USE_MPI=ON -DCMAKE_C_FLAGS="-Wall -Wunused-parameter -Wundef -Wno-long-long -Wsign-compare -Wmissing-prototypes -Wstrict-prototypes -Wcomment -pedantic --coverage" -DCMAKE_EXE_LINKER_FLAGS="--coverage" ctest -D Experimental
The CMake configuration can also be controlled with CMake scripts, see files dash.cmake, dash_mpi.cmake and cdash_heat.sh.
See the results on the CDash server here.
4.7 Packaging using CPack
CPack is a cross-platform software packaging tool. The idea is to filter a subset of source files and objects to release in order to diffuse it as a tarball or a binary package, see wiki about CPack
For example to release a tarball
set(CPACK_SOURCE_GENERATOR "TGZ") set(CPACK_PACKAGE_NAME "heat") set(CPACK_PACKAGE_VERSION "1.0.0") set(CPACK_SOURCE_IGNORE_FILES "/build/;.gitignore") include(CPack)
or a Debian package
set(CPACK_GENERATOR "DEB") set(CPACK_PACKAGE_NAME "heat") set(CPACK_DEBIAN_PACKAGE_MAINTAINER "inria-bso-sed") set(CPACK_PACKAGE_VERSION "1.0.0") include(CPack)
The reader should refer to this link to get an exhaustive view of common CPack variables.
Visit the exercise 9 and try to package "Heat"
cd $CMAKE_HANDS_ON/exe9_cpack/ && mkdir -p build && cd build && cmake .. -DHEAT_USE_MPI=ON -DBUILD_SHARED_LIBS=ON make
Use
make package_source
to make an archive of source files (e.g. a .tar.gz file) and
make package
to create a binary package with the generator given in CPACK_GENERATOR
4.8 Misc
4.8.1 Call outside scripts/commands to generate files
4.8.2 Add dependencies between targets
Example, we need here that "foo" headers are generated in the build directory before generating the "foo" library
# Force generation of headers set(FOO_HDRS headr1.h headr2.h) add_custom_command( OUTPUT ${FOO_HDRS} COMMAND cmake -E touch ${FOO_HDRS} ) add_custom_target(foo_include ALL SOURCES ${FOO_HDRS}) add_dependencies(foo foo_include)
See documentation:
4.8.3 Test if a function exists, if we can link or build a specific source file
Check if the given struct or class has the specified member variable, e.g.
include(CheckStructHasMember) check_struct_has_member( "struct hwloc_obj" parent hwloc.h HAVE_HWLOC_PARENT_MEMBER )
Check that a symbol exists in a library, e.g.
include(CheckLibraryExists) check_library_exists(${HWLOC_LIBRARIES} hwloc_bitmap_free "" HAVE_HWLOC_BITMAP)
Test that you can call a specific function given the way to link, e.g.
include(CheckFunctionExists) set(CMAKE_REQUIRED_INCLUDES "${REQUIRED_INCDIRS}") set(CMAKE_REQUIRED_LIBRARIES "${REQUIRED_LIBS}") check_function_exists(hwloc_topology_init HWLOC_WORKS) mark_as_advanced(HWLOC_WORKS) if(NOT HWLOC_WORKS) message(WARNING "Looking for hwloc : test of hwloc_topology_init with hwloc library fails") endif()
Check if given C source compiles and links into an executable, e.g.
include(CheckCSourceCompiles) check_c_source_compiles( "#include <hwloc.h> int main(void) { hwloc_obj_t o; o->type = HWLOC_OBJ_PU; return 0;}" HAVE_HWLOC_OBJ_PU)
5 Good pratices
5.1 Variables scope
There are different variable types that can be defined in CMake.
Normal variable
set(<variable> <value>... [PARENT_SCOPE])
Cache variable
set(<variable> <value>... CACHE <type> <docstring> [FORCE])
meant to provide user-settable values.
Environment variable
set(ENV{<variable>} [<value>])
- Normal variables are only defined in the current scope
- Each new directory or function (not macro) creates a new scope
5.2 Custom commands (function/macro)
- do not hesitate to define functions and macro to factorize code
- function create new scope, text replacement in macro
- argc argv, argn, …
Function example
function(bar arg1 arg2) message(STATUS "Arguments are: ${arg1} and ${arg2}") message(STATUS "Number of arguments: ${ARGC}") message(STATUS "First argument: ${ARGV0}") message(STATUS "Second argument: ${ARGV1}") message(STATUS "All arguments: ${ARGV}") message(STATUS "All optional arguments: ${ARGN}") set(list_var "${ARGV}") foreach(_var IN LISTS list_var) message(STATUS "${_var}") endforeach() endfunction() message(STATUS "--bar--") bar(one two three four) message(STATUS "list_var: ${list_var}")
Macro example
macro(bar arg1 arg2) message(STATUS "Arguments are: ${arg1} and ${arg2}") message(STATUS "Number of arguments: ${ARGC}") message(STATUS "First argument: ${ARGV0}") message(STATUS "Second argument: ${ARGV1}") message(STATUS "All arguments: ${ARGV}") message(STATUS "All optional arguments: ${ARGN}") set(list_var "${ARGV}") foreach(_var IN LISTS list_var) message(STATUS "${_var}") endforeach() endmacro() message(STATUS "--bar--") bar(one two three four) message(STATUS "list_var: ${list_var}")
Use macros if you want to change parameters (output parameters), function otherwise.
5.3 Generator expressions
To improve compact writting when properties are added under some conditions see cmake generator expressions.
target_compile_options(MyTarget PRIVATE "$<$<CONFIG:Debug>:--my-flag>") target_compile_options(MyTarget PRIVATE "$<IF:$<CONFIG:Debug>,--my-flag,--my-other-flag>")
5.4 Do not edit CMAKE_CXX_FLAGS in cmake project
It's difficult to maintain due to the different compilers flags and each compiler also evolved.
Prefer using target_compile_options
and target_compile_features
,
optionnaly with cmake generator expressions.
target_compile_options(FOO PRIVATE -Werror) target_compile_features(FOO PRIVATE cxx_std_11)
5.5 Managing source file lists
Avoid using file(GLOB …)
file(GLOB SRC_FILES ${CMAKE_CURRENT_SOURCE_DIR}/*.cpp) add_executable(foo ${SRC_FILES})
because if no CMakeLists.txt file changes when a source is added or removed then the generated build system cannot know when to ask CMake to regenerate.
Prefer directly define your target with a precise list of sources. Additional files can be added later.
add_executable(foo foo1.cpp foo2.cpp) ... if (FOO_BOOST) target_sources(foo foo_boost.cpp) endif() if (WIN32) target_sources(foo foo_win.cpp) endif()
This allows to stick to requirements and is less prone to errors than playing with custom variables. Custom variables can contain errors, empty strings or typos. Giving such a bazar to cmake commands leads to undefined behaviors and is hard to debug.
5.6 Properties on targets vs. variables
- cmake <= 2.8.12: create custom variables, append list of sources, flags, link libraries and call a few time cmake commands add_executable, add_library, target_link_libraries, include_directories, add_compile_options …
- cmake >=3.0.0: targets and properties, call appropriate cmake commands to update properties of specific targets
Don't do that
find_package(Boost 1.55 COMPONENTS asio) list(APPEND INCLUDE_DIRS ${BOOST_INCLUDE_DIRS}) list(APPEND LIBRARIES ${BOOST_LIBRARIES}) include_directories(${INCLUDE_DIRS}) link_libraries(${LIBRARIES})
- avoid using command that operates at the directory level
- add_compile_options - include_directories - link_directories - link_libraries
- prefer
- target_compile_definitions - target_compile_features - target_compile_options - target_include_directories - target_link_libraries - target_sources - set_target_properties
Do that
find_package(Boost 1.55 COMPONENTS asio) target_include_directories(MyTarget PUBLIC ${BOOST_INCLUDE_DIRS}) target_link_libraries(MyTarget PUBLIC ${BOOST_LIBRARIES}) # or better target_link_libraries(MyTarget PUBLIC Boost::boost) # see also Boost::<component>
5.7 Propagate properties on target or not, that is the question (PRIVATE|INTERFACE|PUBLIC)
target_link_libraries
is used to add dependencies between
targets. This is not only dependency in term of library linking but
also in term of flags (compilation flags, definitions). Flags can be
propagated transitively from targets to targets that are linked
depending on the keyword used.
Keywords PRIVATE, INTERFACE, PUBLIC can be used in "target_" commands in order to inform cmake whether to propagate the given properties of a target (library) to its dependents or not.
- PRIVATE : target properties only required at the build level
- INTERFACE : only required when the target is used, when linked to it (header only library)
- PUBLIC : required in both situation, to build the target and when another target depends on it
target_link_libraries(foo PUBLIC bar PRIVATE cow)
- cow is used internally by foo but is not exposed in foo's API so that dependents do not need it to use foo
- bar is also used by foo and some bar's objects are used in foo's API so that we need to inform the dependent of foo that bar is also required to use foo
This can be used to transitively propagate required dependencies and allows to avoid having to manage different custom lists of dependencies by hand (cumbersome and difficult to maintain).
In the following example public headers are stored in include
while
some other private headers are located in src
target_include_directories(foo PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/include PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/src)
5.7.1 headers only library
add_library(fooho INTERFACE) target_compile_definitions(fooho INTERFACE FOO=1) target_include_directories(fooho INTERFACE include)
Users only have to use for example
target_link_libraries(bar PUBLIC fooho)
to get appropriate flags from fooho.
5.8 Export your targets into a config file for your users
# define target information to export install(TARGETS heat EXPORT heatTargets DESTINATION lib PUBLIC_HEADER DESTINATION include) # rule to generate the file containing the targets information install(EXPORT heatTargets FILE HeatTargets.cmake NAMESPACE heat:: DESTINATION lib/cmake/heat) # install the config file containing all information (targets + some variables like the version) install(FILES "HeatConfig.cmake" DESTINATION lib/cmake/heat)
Minimalist Config file
include("${CMAKE_CURRENT_LIST_DIR}/HeatTargets.cmake")
5.9 Use exported target when looking for dependencies
Find modules vs. config files exported
- find modules like FindFOO.cmake are full of guesses, it tries to determine where and how a dependeny has been installed
- Config files are better like FOOConfig.cmake because it contains the exact properties of the installed project
- and config files with targets and not just variables are the best (the list of targets exported should be documented somewhere)
In an utopic world users should just write that and inherit from the foo target "INTERFACE" properties
find_package(foo REQUIRED) ... target_link_libraries(bar Foo::foo ...)
5.10 Ctest and cdash and how to use it with gcov and valgrind
- ctest script outside of the project, this can be project agnostic
- filtering tests by name, naming convention with your project name as prefix (so that your project can be imported)
- –output-on-failure
# CTest script set(CTEST_SOURCE_DIRECTORY "/src") set(CTEST_BINARY_DIRECTORY "/build") # should ctest wipe the binary tree before running set(CTEST_START_WITH_EMPTY_BINARY_DIRECTORY TRUE) set(CTEST_CMAKE_GENERATOR "Unix Makefiles") find_program(COVERAGE_COMMAND gcov) find_program(MEMORYCHECK_COMMAND valgrind) set(CTEST_COVERAGE_COMMAND "${COVERAGE_COMMAND}") set(CTEST_MEMORYCHECK_COMMAND "${MEMORYCHECK_COMMAND}") # define ctest steps ctest_start("Experimental") ctest_configure(OPTIONS "-DCMAKE_C_FLAGS=-g -O0 -Wall --coverage -DCMAKE_EXE_LINKER_FLAGS=--coverage -DCMAKE_C_COMPILER=gcc") ctest_build() ctest_test() ctest_coverage() ctest_memcheck() ctest_submit()
ctest -S bench.cmake
5.11 Static analysis with CMake
Tools like cppcheck, clang-tidy, cpplint, include-what-you-use (iwyu) can be used natively when invoking make after cmake configure
find_program(CPPCHECK cppcheck) set(CMAKE_CXX_CPPCHECK "${CPPCHECK}" "--language=c" "--platform=unix64" "--enable=all" "--force" "--inline-suppr" ) ... set(CMAKE_CXX_CLANG_TIDY "${CLANG_TIDY}") set(CMAKE_CXX_CPPLINT "${CPPLINT}") set(CMAKE_CXX_INCLUDE_WHAT_YOU_USE "${IWYI}")
6 Useful links
- Official documentation: https://cmake.org/documentation/
- CMake Wiki (useful sandbox of links): https://gitlab.kitware.com/cmake/community/wikis/home
- CMake FAQ: https://gitlab.kitware.com/cmake/community/wikis/FAQ
- CMake scripting language: https://cmake.org/cmake/help/latest/manual/cmake-language.7.html, https://gitlab.kitware.com/cmake/community/wikis/doc/cmake/Language-Syntax
- CMake Useful variables: https://gitlab.kitware.com/cmake/community/wikis/doc/cmake/Useful-Variables
- Get CMake binaries or install from sources: https://cmake.org/download/ and https://cmake.org/install/