This article describes a technique to build Apptainer containers using Milan compute nodes, via a Slurm job. You can also build Singularity container using this technique.
Building container via Slurm
The new Milan compute nodes can be used to build Apptainer containers using the fakeroot feature. This functionality is only available on these nodes at the moment due to their operating system version.
To illustrate this functionality, create an example container definition file
my_container.def from a shell session on NeSI as follows:
cat << EOF > my_container.def BootStrap: docker From: ubuntu:20.04 %post apt-get -y update apt-get install -y wget EOF
Then submit the following Slurm job submission script to build the container:
#!/bin/bash -e #SBATCH --job-name=apptainer_build #SBATCH --partition=milan #SBATCH --time=0-00:30:00 #SBATCH --mem=4GB #SBATCH --cpus-per-task=2
# load environment module
module purge module load Apptainer/1.1.2
# create a build and cache directory on nobackup storage
export APPTAINER_CACHEDIR="/nesi/nobackup/$SLURM_JOB_ACCOUNT/$USER/apptainer_cache" export APPTAINER_TMPDIR="/nesi/nobackup/$SLURM_JOB_ACCOUNT/$USER/apptainer_tmpdir" mkdir -p $APPTAINER_CACHEDIR $APPTAINER_TMPDIR setfacl -b $APPTAINER_TMPDIR
# build the container apptainer build --fakeroot my_container.sif my_container.def
Note this script will start an Slurm job for 30 minutes using 2 cores and 4 GB of memory to build the image. Make sure to set these resources correctly, some containers can take hours to build and require tens of GB of memory.
More information about how to submit a Slurm job is available in the Submitting your first job support page.
Build environment variables
To build containers, you need to ensure that Apptainer has enough storage space to create intermediate files. It also requires a cache folder to save images pulled from a different location (e.g. DockerHub). By default both of these locations are set to
/tmpwhich has limited space, large builds may exceed this limitation causing the builder to crash. The environment variables
APPTAINER_CACHEDIRare used to overwrite the default location of these directories.
In this example, the Slurm job submission script creates these folders using your project
If your container uses RPM to install packages, i.e. is based on CentOS or Rocky Linux, you need to disable the
APPTAINER_TMPDIR environment variable (use
unset APPTAINER_TMPDIR) and request more memory for your Slurm job. Otherwise, RPM will crash due to an incompatibility with the
If you encounter the following error when using a base Docker image in your Apptainer definition file
While making image from oci registry: error fetching image to cache: while building SIF from layers: conveyor failed to get: unsupported image-specific operation on artifact with type "application/vnd.docker.container.image.v1+json"
it is likely due to an upstream issue (e.g. bad image on Dockerhub). In this case, try an older image version or a different base image.
This method, using fakeroot, is known to not work for all types of Apptainer/Singularity containers.
If you encounter an issue, please contact us at email@example.com.