Getting started with GNU Guix

Table of Contents

1 software deployment in HPC

1.1 module

1.2 Spack, EasyBuild, CONDA, etc.

1.3 Docker, Singularity, Shifter, etc.

1.4 Guix! https://guix-hpc.bordeaux.inria.fr/

  • reproducibility: same code on laptop & cluster
  • versatility: “package management”, container provisioning, etc.
  • transactional upgrades and rollback
  • PlaFRIM support! \o/

2 package management (think apt-get, yum, etc.)

documentation

2.1 installation, removal

guix package --list-installed
guix package -i hwloc gcc-toolchain
which lstopo

ls ~/.guix-profile/bin
cat ~/.guix-profile/etc/profile

guix package -i python
python3 --version

guix package -r python -i ruby
guix package --list-generations
guix package --roll-back

2.2 using separate profiles

guix package -p old-times -i hwloc@1 gcc-toolchain@5.5
./old-times/bin/lstopo --version

eval `guix package -p ~/old-times --search-paths`

2.3 update and upgrade

guix pull            # updates Guix and the list of available packages
guix package -u      # *upgrades* previuosly installed packages

2.4 reproduce

laptop$ guix --version
guix (GNU Guix) cabba9e
...

plafrim$ guix pull --commit=cabba9e   # get the exact same Guix as on my laptop

2.5 declarative approach

Assume ~/my-profile.scm: 

(specifications->manifest '("hwloc" "emacs"))


cat ~/my-profile.scm
guix package -m ~/my-profile.scm

3 one-off development environments with guix environment

documentation 

guix environment petsc          # start a devel env. for petsc


guix environment --ad-hoc python3 python-numpy python-scikit-learn -- python3

# and then: "import numpy" and "import sklearn"

4 customized packages on the command line

documentation

4.1 changing the source

# Download the old tarball, then use it as the source of the existing
# 'hwloc' recipe (needs network access!).
guix build hwloc --with-source=https://www.open-mpi.org/software/hwloc/v1.11/downloads/hwloc-1.11.0.tar.gz

# Likewise, but with a local file.
guix build hwloc --with-source=./hwloc-2.0.1rc42.tar.gz

4.2 changing the dependencies

# Install MUMPS with pt-scotch instead of scotch.
guix package -i mumps --with-input=scotch=pt-scotch 

# Build SLURM with hwloc 2.x instead of 1.x
guix build slurm --with-input=hwloc@1=hwloc@2

5 extra packages through $GUIX_PACKAGE_PATH

documentation

5.1 StarPU, SimGrid, PaSTiX, Chameleon, oh my!

git clone https://gitlab.inria.fr/guix-hpc/guix-hpc.git
export GUIX_PACKAGE_PATH="$PWD/guix-hpc/modules"

Add your own!

6 creating Docker images with guix pack

documentation story “Using Guix without Being root”

6.1 creating an hwloc bundle

guix pack hwloc -S /bin=bin -f docker

And on the Docker side: 

docker load -i hwloc-pack.tar.gz

# Assuming $IMAGE is the hash returned by 'docker load' above.
docker run $IMAGE /bin/lstopo

6.2 creating a devel bundle

guix pack -S /bin=bin -S /etc=etc -f docker gcc-toolchain openmpi

And on the Docker side: 

docker load -i devel-pack.tar.gz
docker run $IMAGE /bin/sh # will source /etc/profile and set up env vars

7 defining new packages

documentation

7.1 importing from external repos with guix import

8 links

9 Join us now! :-)

Created: 2019-10-02 az. 11:43

Validate