CMake hands-on

Please follow this link to get the slides. Another set of slides that are worth seing.

CMake is a cross-platform (Linux, Mac OS X, Windows) open-source make system. CMake is used to control the software compilation process using simple platform-independent and compiler-independent configuration files (cf. CMakeLists.txt files).

Unlike many cross-platform systems, CMake is designed to be used in conjunction with the native build environment. Simple configuration files placed in each source directory (called CMakeLists.txt files) are used to generate standard build files (e.g., makefiles on Unix and projects/workspaces in Windows MSVC) which are used in the usual way.

Binary packages can be easily installed, cf. requirements:

• Debian (apt-get): cmake cmake-data cmake-curses-gui cmake-qt-gui
• CentOS (yum): cmake cmake-data cmake-gui
• Fedora (dnf): cmake cmake-gui
• ArchLinux (pacman): cmake cmake-gui
• Mac OS X (port): cmake-devel +gui
• Mac OS X (brew): cmake

You can also get CMake pre-compiled binaries here or instructions to install from sources here.

Let's practice a little bit. We consider here that we have got the tarballs containing the material, cf. download material section.

Check the content of the first exercise directory

cd $CMAKE_HANDS_ON/exe1_usecmake/
ls -l

Please check that you have some C source code files and a CMakeLists.txt.

One interesting feature in CMake is that you can build "out-of-source" meaning that you can build in your prefered directory where you want on your system and it is not mandatory to build in the root directory of the project. Let's be honest, it is strongly recommended to build "out-of-source" in CMake. Why? Because there is no distclean rule that permit to remove all the files generated during the build phase. There is a clean target to remove files generated by the compiler and linker but there is no way for CMake to know in advance all the objects that user's scripts may create, cf. this link

Let's build our project in the sub-directory build/ to be fancy

mkdir -p build && cd build

The build process with CMake takes place in two stages:

1. standard build files are created from configuration files using the cmake [options] <path-to-parent-CMakeLists.txt> command, normally available in your environment, cf. requirements
2. then the platform's native build tools are used for the actual build

cmake ..
make
./heat_seq 100 100 100 0

One can also drive the different steps, configure and build, without moving into build/ like this

cmake -S . -B build/
cmake --build build/
./build/heat_seq 100 100 100 0

Remark that you can precise the compiler. There are different ways to give it, either using environments variables

cd build
make clean
CC=gcc cmake .. && make

or using a CMake variable CMAKE_<LANG>_COMPILER.

make clean
cmake -D CMAKE_C_COMPILER=clang .. && make

You may want to execute the generated driver, cf. the problem to solve.

./heat_seq # will print how to use the program
./heat_seq 100 100 100 0 # will compute the heat propagation on a 100x100 grid space and for 100 timesteps

• CMake is a "scripting language"; you can process CMake script files with -P option

  cmake -P script.cmake
  

• The names of commands are case insensitive, set() is equivalent to SET()

• Variables always contain strings. Sometimes, we use the string to store a boolean, a path to a file, an integer, or a list. Variables are defined usually with set(var value)

  set(VAR 0) # 0 is considered as a string and this is equivalent to
  set(VAR "0")
  set(VAR "hello") # "hello" is consider as a string and this is equivalent to
  set(VAR hello)
  # the name of variables is case-sensitive, here VAR contains 0, var 1
  set(VAR 0)
  set(var 1)
  

• To print message in the standard output message() is used

  message("Hello")
  message(STATUS "It works fine for now")
  message(WARNING "Something is missing here")
  message(FATAL_ERROR "I can't build in this situation")
  

• To access the value of a variable, you perform a substitution. To perform a substitution, use the syntax ${variablename}

  set(FIBO "0 1 1 2 3 5") # FIBO contains the string "0 1 1 2 3 5"
  # you can also set some lists, structure on which you will be able to iterate over the elements
  set(FIBO 0 1 1 2 3 5) # FIBO contains the list of strings "0;1;1;2;3;5"
  # try to see it with message
  message(${FIBO})
  # iterate on the list
  foreach(_val ${FIBO})
    message("${_val}")
  endforeach()
  

• To make a variable undefined use unset(var) or set(var)
• Note that there exist a set of already existing, and potentialy defined, CMake variables, whose name usually begins with CMAKE_ see the documentation and some common useful cmake variables
• Each directory added with add_subdirectory or each funtion declared with function creates a new scope. The new child scope inherits all variable definitions from its parent scope. Variable assignments in the new child scope with the set command will only be visible in the child scope unless the PARENT_SCOPE option is used.

• CMake has control statements: foreach, if, while (read the documentation)

  # compare numbers
  set( number 4 )
  # if ${number} is greater than 10
  if( number GREATER 10 )
    message( "The number ${number} is too large." )
  endif()
  
  # test if a variable is defined (not empty)
  set(VAR "hello")
  if (VAR)
    message("Let's play with ${VAR}")
  else()
    message("Variable VAR whose value is ${VAR} is undefined")
  endif()
  

• Booleans: CMake if considers an empty string, "0", "FALSE", "OFF", "NO", or any string ending in "-NOTFOUND" to be false.

• CMake provides a list command that performs basic operations with list

  set(VAR "hello")
  list(APPEND VAR " world") # add an element to the list, last position
  list(GET VAR 0 VAR_0) # get an element at a specific location (first here, O-indexed)
  list(INSERT VAR 1 ${VAR}) # insert a new element in second
  list(REMOVE_DUPLICATES VAR) # remove duplicated entries
  

• function and macro can be declared as an additional CMake command The main difference is that function creates a new scope while macro does not.

  One can call macros and functions with additional arguments and there exist native keywords to manipulate them.

  Try this in a cmake script

  function(bar arg1 arg2)
    message(STATUS "Arguments are: ${arg1} and ${arg2}")
    message(STATUS "Number of arguments: ${ARGC}")
    message(STATUS "First argument: ${ARGV0}")
    message(STATUS "Second argument: ${ARGV1}")
    message(STATUS "All arguments: ${ARGV}")
    message(STATUS "All optional arguments: ${ARGN}")
    set(list_var "${ARGV}")
    foreach(_var IN LISTS list_var)
      message(STATUS "${_var}")
    endforeach()
    set(${arg1} "one")
  endfunction()
  
  message(STATUS "--bar--")
  bar(one two three four)
  message(STATUS "one: ${one}")
  

  macro(bar arg1 arg2)
    message(STATUS "Arguments are: ${arg1} and ${arg2}")
    message(STATUS "Number of arguments: ${ARGC}")
    message(STATUS "First argument: ${ARGV0}")
    message(STATUS "Second argument: ${ARGV1}")
    message(STATUS "All arguments: ${ARGV}")
    message(STATUS "All optional arguments: ${ARGN}")
    set(list_var "${ARGV}")
    foreach(_var IN LISTS list_var)
      message(STATUS "${_var}")
    endforeach()
    set(${arg1} "one")
  endmacro()
  
  message(STATUS "--bar--")
  bar(one two three four)
  message(STATUS "one: ${one}")
  

  Guideline: Create macros to wrap commands that have output parameters, otherwise create a function.

• include can be used to import other CMake scripts (.cmake)

  set( command message ) # we want t
  file( WRITE temp "${command}( hi )" ) # writes "message( hi )" to temp file (in the same directory than the current CMakeLists.txt)
  include( temp )                       # include the cmake script containing the message command
  file( REMOVE temp )                   # temp file can be removed
  

To get details about the CMake syntax the reader could refer to the Wiki page section "The CMake Language".

A minimal CMake configuration requires a "CMakeLists.txt" file containing the list of source files and the name of the targets (here executables) to build. The command invocation to build an executable is add_executable

add_executable(hello_world hello_world.c)

Please move to $CMAKE_HANDS_ON/exe2_buildexe/ and build the project

cd $CMAKE_HANDS_ON/exe2_buildexe/ && mkdir -p build && cd build && cmake .. && make

Open the CMakeLists.txt file, it looks like

project (Heat) # this is optional
add_executable(heat_seq heat_seq.c heat.c mat_utils.c heat.h mat_utils.h)
target_link_libraries(heat_seq m)

Pay attention to the target_link_libraries here which indicates to link with the libm external library because the code contains some calls to mathematical functions. Note "libm" (.a, .lib, .so, .dyld, …) should be available in the environment in standard system paths.

The source files could also have been collected with the file command with the GLOB attribute

file(GLOB heat_sources *.c *.h)
add_executable(heat_seq ${heat_sources})

but this is not recommended because CMake will lose its ability to regenerate when some specific source files are added or removed, see documentation.

Guideline: avoid using file(GLOB … to collect source files to build, prefer explicit lists e.g. set(SRC src1 src2).

For a large project with many source files it is normal to organize files into different sub-directories. New CMakeLists.txt files could be created in order to define, following the structure of the project, the different objects to build.

Please move to $CMAKE_HANDS_ON/exe3_buildlib/ and build the project

cd $CMAKE_HANDS_ON/exe3_buildlib/ && mkdir -p build && cd build && cmake .. && make

Remark that we have moved the code related to the computation of one iteration of the problem in include/ and lib/. The idea is to create a library named "heat" containing this portion of the program.

To build a shared library, the variable BUILD_SHARED_LIBS should be used

cmake .. -DBUILD_SHARED_LIBS=ON && make

In CMake the library is defined with add_library. Here it is declared where the source file related to it states (this is not mandatory!), see the CMakeLists.txt file in lib/

add_library(heat heat.c)

If it is required to build "static" or "shared" object only, the keyword STATIC, SHARED may be used in add_library

add_library(heat STATIC heat.c)
add_library(heat SHARED heat.c)

To tell CMake to process other CMake files in sub-directories, the sub-directories must be declared with add_subdirectory, e.g.

# libheat
add_subdirectory(lib)

# heat_seq exe
add_executable(heat_seq heat_seq.c mat_utils.h mat_utils.c)

# dependency to libheat
target_link_libraries(heat_seq heat)

# dependency to libm (math)
find_library(M_LIBRARY m)
mark_as_advanced(M_LIBRARY)
if (M_LIBRARY)
  target_link_libraries(heat_seq ${M_LIBRARY})
endif()

# in dir lib/
add_library(heat heat.c)
target_include_directories(heat PUBLIC ${PROJECT_SOURCE_DIR}/include)

In this example we also have used other useful commands:

• enable_language: to initialize some sets of CMake variables related to one language (C, CXX, Fortran, CUDA …)
• target_include_directories: this is equivalent to "-I…", this adds a directory to the list of preprocessor include file search directories
• find_library: to find a library already installed on the system

One could also use target_sources to add sources to a target. This could be used to add sources depending on the operating system for example

target_sources(heat_seq PRIVATE mat_utils.c)

Guideline: avoid to use cmake commands like include_directories, link_directories, link_libraries and add_compile_options, because they work at the directory level and apply to all entities defined in scope, meaning to all sub-directories as well. Prefere adding properties on specific targets.

It is often useful in a project to tune parameters of the build to enable/disable some features for instance or specify by hand the path to an external dependency. This can be done either by the command option (only ON|OFF) or by setting "CACHE" variables (of type FILEPATH, PATH, STRING, BOOL) which are by nature global variables known from everywhere in the project and persistent over subsequent cmake configurations.

Please move to $CMAKE_HANDS_ON/exe4_options/ and build the project specifying HEAT_USE_MPI=ON

cd $CMAKE_HANDS_ON/exe4_options/ && mkdir -p build && cd build && cmake .. -DHEAT_USE_MPI=ON && make

Look into the CMakeLists.txt to see the option added

option(HEAT_USE_MPI "Build MPI executable" OFF)

Remark it is set to OFF by default. Note also that the following is equivalent

set(HEAT_USE_MPI ON CACHE BOOL "Build MPI executable")

The "CACHE" keyword allows to define a variable that:

• has a global scope (known from all the subtree)
• keeps the last defined value across multiple "configure" steps
• can be explicitly changed with -D to command line and interfaces

It may be used to let the users act on not boolean options, for example

set(HEAT_ORDER "2" CACHE STRING "Order of the numerical scheme: 1, 2 (default) or 3")

We can add constraints on a variable to limit possibilities

set( HEAT_ORDER_LIST "1;2;3")
set( HEAT_ORDER "2" CACHE STRING "Choose the scheme order in ${HEAT_ORDER_LIST}")
set_property(CACHE HEAT_ORDER PROPERTY STRINGS ${HEAT_ORDER_LIST})

Many scientific applications are used to validate new numerical and algorithmic techniques and are often re-compiled to make experiments. But many programs also offer a set of services, features that can be used directly by users (executables e.g. gcc, scripts, …) or by upper-level programs (e.g. blas, boost). In the latter situation the program must be installed somewhere on the system after the build process so that only required files (executables, libraries, header files, etc) are copied for users. This can be done with the target install.

Please move to $CMAKE_HANDS_ON/exe5_install/ and build the project

cd $CMAKE_HANDS_ON/exe5_install/ && mkdir -p build && cd build && cmake .. -DHEAT_USE_MPI=ON && make

One can see in the root CMakeLists.txt the install, usage of this command is detailed in here, command used like this

install(TARGETS heat_seq DESTINATION bin)
set_target_properties(heat PROPERTIES PUBLIC_HEADER "${PROJECT_SOURCE_DIR}/include/heat.h")
install(TARGETS heat DESTINATION lib PUBLIC_HEADER DESTINATION include)

This tells CMake to copy the files related to the target in CMAKE_INSTALL_PREFIX, a native CMake variable. Here the header file heat.h, the executable heat_seq and the library heat are copied in CMAKE_INSTALL_PREFIX/include, CMAKE_INSTALL_PREFIX/bin, CMAKE_INSTALL_PREFIX/lib respectively when the user invokes make install.

cmake .. -DCMAKE_INSTALL_PREFIX=$PWD/install
make install
ls -l install/

Check here that the files have been copied into the install directory.

In the case your executables are linked with shared libraries there is the possibility to add paths in the RPATH in order to set the directories where to find shared libraries.

set(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib")

Note that it is possible to install any file or directory and not only targets, e.g.

install(DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/doc DESTINATION share/foo/doc)
install(FILES libfoo.pc DESTINATION lib/pkgconfig)

One limitation of CMake is that there is no uninstall target by default, cf. previous discussion. This problem can be bypassed by defining the uninstall target of the project.

Try the following

make uninstall

This method has been picked up from here and seems more to be a trick than a clean solution.

Notice the use of:

• add_custom_target: to define a target that executes a custom command such as rm, here it executes our custom cmake script cmake_uninstall.cmake.

We have seen in the section use CMake that the user can define compilers, compiler flags and linker flags by hand through the command line or through the interfaces. The CMake variables are the following:

• CMAKE_<LANG>_COMPILER: compiler executable to use, <LANG> can be C, CXX, CUDA, Fortran, ASM
• CMAKE_<LANG>_FLAGS: compiler flags for compiling <LANG> sources.
• CMAKE_EXE_LINKER_FLAGS, CMAKE_SHARED|STATIC_LINKER_FLAGS, CMAKE_MODULE_LINKER_FLAGS: linker flags

These variables are internal to CMake and can be either set by the developer in the configuration CMakeLists.txt files in the project or, if nothing is specified in the project, set by default by CMake during the configuration step (compilers are detected automatically and no specific flags are chosen). Of course these variables can be set or changed by hand for specific needs as we have seen in the section use CMake.

Let's give some hints to force compilers, flags and definitions in configuration files for a project. Remember that to see the details about the build commands, the parameter VERBOSE=1 can be used when using "make" or alternatively the variable CMAKE_VERBOSE_MAKEFILE may be set to ON.

cmake .. -D CMAKE_VERBOSE_MAKEFILE=ON

Force a compiler, e.g.

cmake .. -D CMAKE_C_COMPILER="icc"

Add some compiler flags, e.g.

cmake .. -D CMAKE_C_FLAGS="-W -Wall"

Add some linker flags, e.g.

cmake .. -D CMAKE_SHARED_LINKER_FLAGS="-undefined dynamic_lookup"

The CMake variables we have seen are user's interface variables which are added to the one defined by developers in the CMakeLists.txt files. To set some compile properties directly in the CMakeLists.txt, for a long term usage, there exist some macros that set or update the options for targets and source files.

Here some command to set options globally for all sources and targets defined after the call

• add_definitions: adds -D define flags to the compilation of source files for all the targets
• include_directories: idem header files (-I)
• add_compile_options: idem compile flags

add_definitions(-DADD_)
include_directories(${MPI_INCLUDE_PATH})
add_compile_options(-W;-Wall)

Guideline: avoid to set compilations flags globally with these commands, prefer setting flags on specific targets with the macros presented hereafter.

Specific flags, definitions or other properties can also be defined for some precise objects of the project (in a directory only, for a target or some source files).

• set_property: very high level macro to set some options, flags, definitions
• set_target_properties: set properties for targets
• set_source_files_properties: set properties for source files
• target_include_directories: add include directories to a target (include_directories command applies to all targets)
• target_compile_definitions: add compile definitions to a target (add_definitions command applies to all targets)
• target_compile_options: add compile options to a target (compile_options command applies to all targets)
• target_link_libraries: specify libraries or flags to use when linking a given target
• target_link_options: options to the link step for an executable, shared library or module library target

See the properties that can be tuned for each kind of object here.

set_property(DIRECTORY PROPERTY COMPILE_OPTIONS ${flags_to_add})
set_target_properties(${TARGET} PROPERTIES COMPILE_FLAGS ${flags_to_add})
set_source_files_properties(${SOURCE_FILES} PROPERTIES COMPILE_FLAGS ${flags_to_add})
target_include_directories(${TARGET} PUBLIC /path/to/some/headers;/other/path)
target_compile_definitions(${TARGET} PUBLIC ADEF)
target_compile_options(${TARGET} PUBLIC -std=c++11)
target_link_libraries(${TARGET} m)

Many projects rely on external libraries that are not necessarily vital for an operating system (e.g. MPI, CUDA, etc). This implies that the project should provide a clean way to detect the dependency it relies on and do not try to build if this dependency is not found on the system (and in the later case should give some hints to install it). This subject is presented in the Wiki here.

In the previous exercise, $CMAKE_HANDS_ON/exe5_install/, you can open the CMakeLists.txt file and check the line where we look for the MPI library (to handle parallel tasks on distributed memory architectures).

find_package(MPI)

This command will check if MPI can be found on the system, in some standard paths or in the environment, and will set some variables (e.g. MPI_C_LIBRARIES) with the required information to link with/use the dependency. The command relies on some module files that are either released with CMake, e.g. /usr/share/cmake-3.10/Modules/FindMPI.cmake, or providen from outside. For example the project could provide some custom modules with some specific rules to find the dependency. Note that additional module directories must be given by setting the variable CMAKE_MODULE_PATH, e.g.

# add this directory to look for cmake modules
list(APPEND CMAKE_MODULE_PATH "${PROJECT_SOURCE_DIR}/cmake_modules/")
# include cmake code of my custom modules MyCMakeModule.cmake
include(MyCMakeModule)

The module files released in CMake (in /usr/share/cmake-3.10/Modules) are written by the community so that there can be many differences between each and developers should read the usage of the module (head of the file) carrefully to learn how to use it and understand its behaviour. Have a look to

cmake --help-module-list

in order to list all module files available in your current environment. Usage can also be seen, e.g.

cmake --help-module FindMPI

The common API of the find_package command is the following

find_package(<package> [version] [EXACT] [QUIET] [MODULE]
             [REQUIRED] [[COMPONENTS] [components...]]
             [OPTIONAL_COMPONENTS components...]
             [NO_POLICY_SCOPE])

• version: to specify a version, e.g. "at least 1.0" or "exactly 1.2"
• QUIET: to disable module messages
• REQUIRED: to end in failure if the package is not found
• COMPONENTS: to specify "I want this feature in the package" or "I need this package to be linked with libfoo", this strongly depends on the module maintainers choice

This macro will certainly give variables as outputs with the results of the detection, e.g. in MPI_C_LIBRARIES, MPI_C_INCLUDE_DIRS. More recently "Imported targets" may be defined and ease the linking process because all properties are given to the dependent when using target_link_libraries.

For example instead of using

target_include_directories(heat_par PRIVATE ${MPI_C_INCLUDE_DIRS})
target_link_libraries(heat_par ${MPI_C_LIBRARIES})

One can use directly

target_link_libraries(heat_par PRIVATE MPI::MPI_C)
# here all important properties are given for users: compile flags and definitions, link flags, ...

If the detection module does not exist yet for the library you are looking at you can either write your own FindFOO.cmake (see next section) or use the following CMake macros:

• find_path: to find directories containing a specific file (useful for headers or fortran modules)
• find_library: to find libraries with absolute paths.

The modules existing in CMake concerns a set of libraries and tools for which the effort has been done by the community to help users to automatically detect the program. Of course not all libraries can be found that way and developers often have to write the ones missing for their specific project. Notice it is higly discouraged to write find modules for packages that themselves build with CMake. Instead it is better to provide a CMake package configuration file with the package itself like we will see in the next section.

The goal is to detect the set of files (e.g. headers) and objects (e.g. libraries) that an application will need to use the program. The CMake code should be written in module file whose name starts with Find and ends with the package name followed by .cmake.

Let's give some hints with an example, the hwloc library.

See for example the file $CMAKE_HANDS_ON/exe6_findpackage/cmake_modules/FindHWLOC.cmake and how to use it in $CMAKE_HANDS_ON/exe6_findpackage/CMakeLists.txt.

You can verify that the executable heat_seq is properly linked to hwloc and that it calls the API (see in the file heat_seq.c the main function)

cd $CMAKE_HANDS_ON/exe6_findpackage/ && mkdir -p build && cd build && cmake .. && make VERBOSE=1

Lets describe a bit what is written in the Find module.

First, it should start with a "Usage" section explaining how to use the module

###
#
# - Find HWLOC include dirs and libraries
# Use this module by invoking find_package with the form:
#  find_package(HWLOC
#               [REQUIRED]) # Fail with error if hwloc is not found
#
# This module finds headers and hwloc library.
# Results are reported in variables:
#  HWLOC_FOUND           - True if headers and requested libraries were found
#  HWLOC_INCLUDE_DIRS    - hwloc include directories
#  HWLOC_LIBRARY_DIRS    - Link directories for hwloc libraries
#  HWLOC_LIBRARIES       - hwloc component libraries to be linked
#
# The user can give specific paths where to find the libraries adding cmake
# options at configure (ex: cmake path/to/project -DHWLOC_DIR=path/to/hwloc):
#  HWLOC_DIR             - Where to find the base directory of hwloc
#  HWLOC_INCDIR          - Where to find the header files
#  HWLOC_LIBDIR          - Where to find the library files
# The module can also look for the following environment variables if paths
# are not given as cmake variable: HWLOC_DIR, HWLOC_INCDIR, HWLOC_LIBDIR

The module should provide a way to let the user give some exotic paths directly by hand through CMake variables or through environment variables

set(ENV_HWLOC_DIR "$ENV{HWLOC_DIR}")
set(ENV_HWLOC_INCDIR "$ENV{HWLOC_INCDIR}")
set(ENV_HWLOC_LIBDIR "$ENV{HWLOC_LIBDIR}")
set(HWLOC_GIVEN_BY_USER "FALSE")
if ( HWLOC_DIR OR ( HWLOC_INCDIR AND HWLOC_LIBDIR) OR ENV_HWLOC_DIR OR (ENV_HWLOC_INCDIR AND ENV_HWLOC_LIBDIR) )
  set(HWLOC_GIVEN_BY_USER "TRUE")
endif()

Some programs give directly the required information thanks to the pkg-config tool. When a program relies on pkg-config the module should try to use these information first if the paths are not given by the user.

# Optionally use pkg-config to detect include/library dirs (if pkg-config is available)
# -------------------------------------------------------------------------------------
include(FindPkgConfig)
find_package(PkgConfig QUIET)
if( PKG_CONFIG_EXECUTABLE AND NOT HWLOC_GIVEN_BY_USER )
  pkg_search_module(HWLOC hwloc)
  if (NOT HWLOC_FIND_QUIETLY)
    if (HWLOC_FOUND AND HWLOC_LIBRARIES)
      message(STATUS "Looking for HWLOC - found using PkgConfig")
    else()
      message("Looking for HWLOC - not found using PkgConfig."
              "Perhaps you should add the directory containing hwloc.pc to"
              "the PKG_CONFIG_PATH environment variable.")
    endif()
  endif()
endif( PKG_CONFIG_EXECUTABLE AND NOT HWLOC_GIVEN_BY_USER )

If the package is not found we will try to find it on the system or in the paths given by the user. Note in the following the use of find_path to look for specific files and find_library to look for specific libraries. Note also that we search in priority into directories given by the user and then into default directories depending on the system.

# Looking for include
# -------------------

# Add system include paths to search include
# ------------------------------------------
unset(_inc_env)
if(ENV_HWLOC_INCDIR)
  list(APPEND _inc_env "${ENV_HWLOC_INCDIR}")
elseif(ENV_HWLOC_DIR)
  list(APPEND _inc_env "${ENV_HWLOC_DIR}")
  list(APPEND _inc_env "${ENV_HWLOC_DIR}/include")
  list(APPEND _inc_env "${ENV_HWLOC_DIR}/include/hwloc")
else()
  if(WIN32)
    string(REPLACE ":" ";" _inc_env "$ENV{INCLUDE}")
  else()
    string(REPLACE ":" ";" _path_env "$ENV{INCLUDE}")
    list(APPEND _inc_env "${_path_env}")
    string(REPLACE ":" ";" _path_env "$ENV{C_INCLUDE_PATH}")
    list(APPEND _inc_env "${_path_env}")
    string(REPLACE ":" ";" _path_env "$ENV{CPATH}")
    list(APPEND _inc_env "${_path_env}")
    string(REPLACE ":" ";" _path_env "$ENV{INCLUDE_PATH}")
    list(APPEND _inc_env "${_path_env}")
  endif()
endif()
list(APPEND _inc_env "${CMAKE_C_IMPLICIT_INCLUDE_DIRECTORIES}")
list(REMOVE_DUPLICATES _inc_env)

# set paths where to look for
set(PATH_TO_LOOK_FOR "${_inc_env}")

# Try to find the hwloc header in the given paths
# -------------------------------------------------
# call cmake macro to find the header path
if(HWLOC_INCDIR)
  set(HWLOC_hwloc.h_DIRS "HWLOC_hwloc.h_DIRS-NOTFOUND")
  find_path(HWLOC_hwloc.h_DIRS
    NAMES hwloc.h
    HINTS ${HWLOC_INCDIR})
else()
  if(HWLOC_DIR)
    set(HWLOC_hwloc.h_DIRS "HWLOC_hwloc.h_DIRS-NOTFOUND")
    find_path(HWLOC_hwloc.h_DIRS
      NAMES hwloc.h
      HINTS ${HWLOC_DIR}
      PATH_SUFFIXES "include" "include/hwloc")
  else()
    set(HWLOC_hwloc.h_DIRS "HWLOC_hwloc.h_DIRS-NOTFOUND")
    find_path(HWLOC_hwloc.h_DIRS
              NAMES hwloc.h
              HINTS ${PATH_TO_LOOK_FOR}
              PATH_SUFFIXES "hwloc")
  endif()
endif()
mark_as_advanced(HWLOC_hwloc.h_DIRS)

# Add path to cmake variable
# ------------------------------------
if (HWLOC_hwloc.h_DIRS)
  set(HWLOC_INCLUDE_DIRS "${HWLOC_hwloc.h_DIRS}")
else ()
  set(HWLOC_INCLUDE_DIRS "HWLOC_INCLUDE_DIRS-NOTFOUND")
  if(NOT HWLOC_FIND_QUIETLY)
      message(STATUS "Looking for hwloc -- hwloc.h not found")
  endif()
endif ()

# Looking for lib
# ---------------

# Add system library paths to search lib
# --------------------------------------
unset(_lib_env)
if(ENV_HWLOC_LIBDIR)
  list(APPEND _lib_env "${ENV_HWLOC_LIBDIR}")
elseif(ENV_HWLOC_DIR)
  list(APPEND _lib_env "${ENV_HWLOC_DIR}")
  list(APPEND _lib_env "${ENV_HWLOC_DIR}/lib")
else()
  if(WIN32)
    string(REPLACE ":" ";" _lib_env "$ENV{LIB}")
  else()
    if(APPLE)
        string(REPLACE ":" ";" _lib_env "$ENV{DYLD_LIBRARY_PATH}")
    else()
        string(REPLACE ":" ";" _lib_env "$ENV{LD_LIBRARY_PATH}")
    endif()
    list(APPEND _lib_env "${CMAKE_C_IMPLICIT_LINK_DIRECTORIES}")
  endif()
endif()
list(REMOVE_DUPLICATES _lib_env)

# set paths where to look for
set(PATH_TO_LOOK_FOR "${_lib_env}")

# Try to find the hwloc lib in the given paths
# ----------------------------------------------

# call cmake macro to find the lib path
if(HWLOC_LIBDIR)
  set(HWLOC_hwloc_LIBRARY "HWLOC_hwloc_LIBRARY-NOTFOUND")
  find_library(HWLOC_hwloc_LIBRARY
      NAMES hwloc
      HINTS ${HWLOC_LIBDIR})
else()
  if(HWLOC_DIR)
    set(HWLOC_hwloc_LIBRARY "HWLOC_hwloc_LIBRARY-NOTFOUND")
    find_library(HWLOC_hwloc_LIBRARY
        NAMES hwloc
        HINTS ${HWLOC_DIR}
        PATH_SUFFIXES lib lib32 lib64)
  else()
    set(HWLOC_hwloc_LIBRARY "HWLOC_hwloc_LIBRARY-NOTFOUND")
    find_library(HWLOC_hwloc_LIBRARY
                 NAMES hwloc
                 HINTS ${PATH_TO_LOOK_FOR})
  endif()
endif()
mark_as_advanced(HWLOC_hwloc_LIBRARY)

# If found, add path to cmake variable
# ------------------------------------
if (HWLOC_hwloc_LIBRARY)
  get_filename_component(hwloc_lib_path ${HWLOC_hwloc_LIBRARY} PATH)
  # set cmake variables
  set(HWLOC_LIBRARIES    "${HWLOC_hwloc_LIBRARY}")
  set(HWLOC_LIBRARY_DIRS "${hwloc_lib_path}")
else ()
  set(HWLOC_LIBRARIES    "HWLOC_LIBRARIES-NOTFOUND")
  set(HWLOC_LIBRARY_DIRS "HWLOC_LIBRARY_DIRS-NOTFOUND")
  if(NOT HWLOC_FIND_QUIETLY)
      message(STATUS "Looking for hwloc -- lib hwloc not found")
  endif()
endif ()

After the files have been found, the information that will be transmitted to the user should be tested. Here we try to link and check that a basic symbol exist and can be called from a program. Notice the use of check_function_exists to check if a C function can be linked.

# check a function to validate the find
if(HWLOC_LIBRARIES)
  # set required libraries for link
  set(CMAKE_REQUIRED_LIBRARIES "${HWLOC_LIBRARIES}")

  # test link
  unset(HWLOC_WORKS CACHE)
  include(CheckFunctionExists)
  check_function_exists(hwloc_topology_init HWLOC_WORKS)
  mark_as_advanced(HWLOC_WORKS)

  if(NOT HWLOC_WORKS)
    if(NOT HWLOC_FIND_QUIETLY)
        message(STATUS "Looking for hwloc : test of hwloc_topology_init with hwloc library fails")
        message(STATUS "Check in CMakeFiles/CMakeError.log to figure out why it fails")
    endif()
  endif()
endif(HWLOC_LIBRARIES)

Create a custom target to ease libraries linking avoiding to use lists directly.

if(NOT TARGET hwloc::hwloc)

    # initialize imported target
    add_library(hwloc::hwloc INTERFACE IMPORTED)

    if (HWLOC_INCLUDE_DIRS)
        set_target_properties(hwloc::hwloc PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${HWLOC_INCLUDE_DIRS}")
    endif()
    if (HWLOC_LIBRARY_DIRS)
        set_target_properties(hwloc::hwloc PROPERTIES INTERFACE_LINK_DIRECTORIES "${HWLOC_LIBRARY_DIRS}")
    endif()
    if (HWLOC_LIBRARIES)
        set_target_properties(hwloc::hwloc PROPERTIES INTERFACE_LINK_LIBRARIES "${HWLOC_LIBRARIES}")
    endif()
    if (HWLOC_CFLAGS_OTHER)
        set_target_properties(hwloc::hwloc PROPERTIES INTERFACE_COMPILE_OPTIONS "${HWLOC_CFLAGS_OTHER}")
    endif()
    if (HWLOC_LDFLAGS_OTHER)
        set_target_properties(hwloc::hwloc PROPERTIES INTERFACE_LINK_OPTIONS "${HWLOC_LDFLAGS_OTHER}")
    endif()

endif (NOT TARGET hwloc::hwloc)

Finally we validate the find process to set HWLOC_FOUND to true or false depending on the status of some variables. Here for example, if HWLOC_LIBRARIES or HWLOC_WORKS are not well defined then the package will be considered as not found.

# check that HWLOC has been found
# -------------------------------
include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(HWLOC DEFAULT_MSG
                                  HWLOC_LIBRARIES
                                  HWLOC_WORKS)

Some references:

• https://gitlab.kitware.com/cmake/community/wikis/doc/tutorials/How-To-Find-Libraries
• https://gitlab.kitware.com/cmake/community/wikis/doc/cmake/dev/Module-Maintainers

Writing a module to be used with find_package is useful especially when the dependency was not built with CMake. For dependencies that rely on CMake, configuration files may be providen and used very efficiently to propagate all required information about the build and installation.

In the exercise 7 (exe7_exportconf) we provide a configuration file, $CMAKE_HANDS_ON/exe7_exportconf/lib/HeatConfig.cmake, which gives the definitions of targets exported for users.

cd $CMAKE_HANDS_ON/exe7_exportconf/ && mkdir -p build && cd build && cmake .. -DCMAKE_INSTALL_PREFIX=$PWD/install

This file is copied in lib/cmake/HeatConfig.cmake in the installation path after make install, this to give the appropriate information to users.

make install

This means that another cmake project can use these information to use/link to the Heat project

# heat_ext exe
add_executable(heat_ext heat_ext.c)

# find libheat
find_package(Heat REQUIRED)

# dependency to libheat
target_link_libraries(heat_ext heat::heat)

and giving the path to the configuration file

cmake .. -DCMAKE_PREFIX_PATH=where/is/installed/heat/lib/cmake # or also
cmake .. -DHeat_DIR=where/is/installed/heat/lib/cmake/heat

See in the lib/CMakeLists.txt file how the "heat" target can be exported for users

# define target information to export
install(TARGETS heat EXPORT heatTargets
        DESTINATION lib
        PUBLIC_HEADER DESTINATION include)
# rule to generate the file containing the targets information
install(EXPORT heatTargets
        FILE HeatTargets.cmake
        NAMESPACE heat::
        DESTINATION lib/cmake/heat)

# install the config file containing all information (targets + some variables like the version)
install(FILES "HeatConfig.cmake"
        DESTINATION lib/cmake/heat)

Some references:

• https://gitlab.kitware.com/cmake/community/wikis/doc/Tutorials#cmake-packages
• https://gitlab.kitware.com/cmake/community/wikis/doc/tutorials/How-to-create-a-ProjectConfig.cmake-file

See a more complex example:

• https://gitlab.inria.fr/solverstack/spm/-/blob/master/src/CMakeLists.txt
• https://gitlab.inria.fr/solverstack/spm/-/blob/master/cmake_modules/SPMConfig.cmake.in
• https://gitlab.inria.fr/solverstack/spm/-/blob/master/CMakeLists.txt

In some situations the detection of dependencies is not robust enough and developers prefer to drive the installation (a precise version, a custom configuration), of dependencies directly from their CMake project. This can be done in the CMakeLists.txt files thanks to the externalproject_add command. The possibilities are many, cf. documentation

• download step: download a tarball or pull from an SCM repository
• update step: svn up, apply patches
• configure step: drive the configuration with a command or with CMake variables
• build step: choose where to build and "make" parameters
• install step: choose where to install and "make install" parameters
• test step
• log, …

Example

ExternalProject_Add(foo
  SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/foo"
  CMAKE_ARGS ${foo_cmake_args}
  BUILD_COMMAND "make" "-j4"
  )

CTest is a testing tool distributed as a part of CMake. It can be used to automate updating (using GIT for example), configuring, building, testing, performing memory checking, performing coverage, and submitting results to a CDash.

CPack is a cross-platform software packaging tool. The idea is to filter a subset of source files and objects to release in order to diffuse it as a tarball or a binary package, see wiki about CPack

For example to release a tarball

set(CPACK_SOURCE_GENERATOR "TGZ")
set(CPACK_PACKAGE_NAME "heat")
set(CPACK_PACKAGE_VERSION "1.0.0")
set(CPACK_SOURCE_IGNORE_FILES "/build/;.gitignore")
include(CPack)

or a Debian package

set(CPACK_GENERATOR "DEB")
set(CPACK_PACKAGE_NAME "heat")
set(CPACK_DEBIAN_PACKAGE_MAINTAINER "inria-bso-sed")
set(CPACK_PACKAGE_VERSION "1.0.0")
include(CPack)

The reader should refer to this link to get an exhaustive view of common CPack variables.

Visit the exercise 9 and try to package "Heat"

cd $CMAKE_HANDS_ON/exe9_cpack/ && mkdir -p build && cd build && cmake .. -DHEAT_USE_MPI=ON -DBUILD_SHARED_LIBS=ON
make

Use

make package_source

to make an archive of source files (e.g. a .tar.gz file) and

make package

to create a binary package with the generator given in CPACK_GENERATOR

There are different variable types that can be defined in CMake.

Normal variable

set(<variable> <value>... [PARENT_SCOPE])

Cache variable

set(<variable> <value>... CACHE <type> <docstring> [FORCE])

meant to provide user-settable values.

Environment variable

set(ENV{<variable>} [<value>])

• Normal variables are only defined in the current scope
• Each new directory or function (not macro) creates a new scope

• do not hesitate to define functions and macro to factorize code
• function create new scope, text replacement in macro
• argc argv, argn, …

Function example

function(bar arg1 arg2)
  message(STATUS "Arguments are: ${arg1} and ${arg2}")
  message(STATUS "Number of arguments: ${ARGC}")
  message(STATUS "First argument: ${ARGV0}")
  message(STATUS "Second argument: ${ARGV1}")
  message(STATUS "All arguments: ${ARGV}")
  message(STATUS "All optional arguments: ${ARGN}")
  set(list_var "${ARGV}")
  foreach(_var IN LISTS list_var)
    message(STATUS "${_var}")
  endforeach()
endfunction()

message(STATUS "--bar--")
bar(one two three four)
message(STATUS "list_var: ${list_var}")

Macro example

macro(bar arg1 arg2)
  message(STATUS "Arguments are: ${arg1} and ${arg2}")
  message(STATUS "Number of arguments: ${ARGC}")
  message(STATUS "First argument: ${ARGV0}")
  message(STATUS "Second argument: ${ARGV1}")
  message(STATUS "All arguments: ${ARGV}")
  message(STATUS "All optional arguments: ${ARGN}")
  set(list_var "${ARGV}")
  foreach(_var IN LISTS list_var)
    message(STATUS "${_var}")
  endforeach()
endmacro()

message(STATUS "--bar--")
bar(one two three four)
message(STATUS "list_var: ${list_var}")

Use macros if you want to change parameters (output parameters), function otherwise.

To improve compact writting when properties are added under some conditions see cmake generator expressions.

target_compile_options(MyTarget PRIVATE "$<$<CONFIG:Debug>:--my-flag>")
target_compile_options(MyTarget PRIVATE "$<IF:$<CONFIG:Debug>,--my-flag,--my-other-flag>")

It's difficult to maintain due to the different compilers flags and each compiler also evolved.

Prefer using target_compile_options and target_compile_features, optionnaly with cmake generator expressions.

target_compile_options(FOO PRIVATE -Werror)
target_compile_features(FOO PRIVATE cxx_std_11)

Avoid using file(GLOB …)

file(GLOB SRC_FILES ${CMAKE_CURRENT_SOURCE_DIR}/*.cpp)
add_executable(foo ${SRC_FILES})

because if no CMakeLists.txt file changes when a source is added or removed then the generated build system cannot know when to ask CMake to regenerate.

Prefer directly define your target with a precise list of sources. Additional files can be added later.

add_executable(foo
  foo1.cpp
  foo2.cpp)
...
if (FOO_BOOST)
  target_sources(foo foo_boost.cpp)
endif()
if (WIN32)
  target_sources(foo foo_win.cpp)
endif()

This allows to stick to requirements and is less prone to errors than playing with custom variables. Custom variables can contain errors, empty strings or typos. Giving such a bazar to cmake commands leads to undefined behaviors and is hard to debug.

• cmake <= 2.8.12: create custom variables, append list of sources, flags, link libraries and call a few time cmake commands add_executable, add_library, target_link_libraries, include_directories, add_compile_options …
• cmake >=3.0.0: targets and properties, call appropriate cmake commands to update properties of specific targets

Don't do that

find_package(Boost 1.55 COMPONENTS asio)
list(APPEND INCLUDE_DIRS ${BOOST_INCLUDE_DIRS})
list(APPEND LIBRARIES ${BOOST_LIBRARIES})

include_directories(${INCLUDE_DIRS})
link_libraries(${LIBRARIES})

• avoid using command that operates at the directory level

- add_compile_options
- include_directories
- link_directories
- link_libraries

• prefer

- target_compile_definitions
- target_compile_features
- target_compile_options
- target_include_directories
- target_link_libraries
- target_sources
- set_target_properties

Do that

find_package(Boost 1.55 COMPONENTS asio)
target_include_directories(MyTarget PUBLIC ${BOOST_INCLUDE_DIRS})
target_link_libraries(MyTarget PUBLIC ${BOOST_LIBRARIES})

# or better
target_link_libraries(MyTarget PUBLIC Boost::boost)
# see also Boost::<component>

target_link_libraries is used to add dependencies between targets. This is not only dependency in term of library linking but also in term of flags (compilation flags, definitions). Flags can be propagated transitively from targets to targets that are linked depending on the keyword used.

Keywords PRIVATE, INTERFACE, PUBLIC can be used in "target_" commands in order to inform cmake whether to propagate the given properties of a target (library) to its dependents or not.

• PRIVATE : target properties only required at the build level
• INTERFACE : only required when the target is used, when linked to it (header only library)
• PUBLIC : required in both situation, to build the target and when another target depends on it

target_link_libraries(foo
  PUBLIC  bar
  PRIVATE cow)

• cow is used internally by foo but is not exposed in foo's API so that dependents do not need it to use foo
• bar is also used by foo and some bar's objects are used in foo's API so that we need to inform the dependent of foo that bar is also required to use foo

This can be used to transitively propagate required dependencies and allows to avoid having to manage different custom lists of dependencies by hand (cumbersome and difficult to maintain).

In the following example public headers are stored in include while some other private headers are located in src

target_include_directories(foo
  PUBLIC  ${CMAKE_CURRENT_SOURCE_DIR}/include
  PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/src)

# define target information to export
install(TARGETS heat EXPORT heatTargets
        DESTINATION lib
        PUBLIC_HEADER DESTINATION include)
# rule to generate the file containing the targets information
install(EXPORT heatTargets
        FILE HeatTargets.cmake
        NAMESPACE heat::
        DESTINATION lib/cmake/heat)

# install the config file containing all information (targets + some variables like the version)
install(FILES "HeatConfig.cmake"
        DESTINATION lib/cmake/heat)

Minimalist Config file

include("${CMAKE_CURRENT_LIST_DIR}/HeatTargets.cmake")

Find modules vs. config files exported

• find modules like FindFOO.cmake are full of guesses, it tries to determine where and how a dependeny has been installed
• Config files are better like FOOConfig.cmake because it contains the exact properties of the installed project
• and config files with targets and not just variables are the best (the list of targets exported should be documented somewhere)

In an utopic world users should just write that and inherit from the foo target "INTERFACE" properties

find_package(foo REQUIRED)
...
target_link_libraries(bar Foo::foo ...)

• ctest script outside of the project, this can be project agnostic
• filtering tests by name, naming convention with your project name as prefix (so that your project can be imported)
• –output-on-failure

# CTest script

set(CTEST_SOURCE_DIRECTORY "/src")
set(CTEST_BINARY_DIRECTORY "/build")

# should ctest wipe the binary tree before running
set(CTEST_START_WITH_EMPTY_BINARY_DIRECTORY TRUE)

set(CTEST_CMAKE_GENERATOR "Unix Makefiles")

find_program(COVERAGE_COMMAND gcov)
find_program(MEMORYCHECK_COMMAND valgrind)
set(CTEST_COVERAGE_COMMAND "${COVERAGE_COMMAND}")
set(CTEST_MEMORYCHECK_COMMAND "${MEMORYCHECK_COMMAND}")

# define ctest steps
ctest_start("Experimental")
ctest_configure(OPTIONS "-DCMAKE_C_FLAGS=-g -O0 -Wall --coverage -DCMAKE_EXE_LINKER_FLAGS=--coverage -DCMAKE_C_COMPILER=gcc")
ctest_build()
ctest_test()
ctest_coverage()
ctest_memcheck()
ctest_submit()

ctest -S bench.cmake

Tools like cppcheck, clang-tidy, cpplint, include-what-you-use (iwyu) can be used natively when invoking make after cmake configure

find_program(CPPCHECK cppcheck)
set(CMAKE_CXX_CPPCHECK "${CPPCHECK}"
  "--language=c"
  "--platform=unix64"
  "--enable=all"
  "--force"
  "--inline-suppr"
  )
...
set(CMAKE_CXX_CLANG_TIDY "${CLANG_TIDY}")
set(CMAKE_CXX_CPPLINT "${CPPLINT}")
set(CMAKE_CXX_INCLUDE_WHAT_YOU_USE "${IWYI}")