Service Status
Mahuika's new nodes are in an Early Access Programme (EAP) phase and not fully in production.
See Mahuika Extension Onboarding for more information about it.
This article describes a technique to build Apptainer containers using Mahuika Extension nodes, via a Slurm job. You can also build Singularity container using this technique.
Build Environment Variables
To build containers, you need to ensure that Apptainer has enough storage space to create intermediate files. It also requires a cache folder to save images pulled from a different location (e.g. DockerHub). By default both of these locations are set to /tmp
which has limited space, large builds may exceed this limitation causing the builder to crash.
The environment variables APPTAINER_TMPDIR
and APPTAINER_CACHEDIR
environment can be used to overwrite the default location of these directories.
export APPTAINER_CACHEDIR=/nesi/nobackup/PROJECTID/apptainer_cache
export APPTAINER_TMPDIR=/nesi/nobackup/PROJECTID/apptainer_tmpdir
mkdir -p $APPTAINER_CACHEDIR $APPTAINER_TMPDIR
setfacl -b $APPTAINER_TMPDIR
where PROJECTID
is your NeSI project ID.
Building container via Slurm
The new Mahuika Extension nodes can be used to build Apptainer containers using the fakeroot feature. This functionality is only available on these nodes at the moment due to their operating system version.
To illustrate this functionality, create an example container definition file my_container.def
from a shell session on NeSI as follows:
cat << EOF > my_container.def
BootStrap: docker
From: ubuntu:20.04
%post
apt-get -y update
apt-get install -y wget
EOF
Then, from a Mahuika login node, you can build the container using the srun
command as follows:
module purge
module load Apptainer/1.1.2
srun -p milan --time 0-00:30:00 --mem 4GB --cpus-per-task 2 apptainer build --fakeroot my_container.sif my_container.def
This command will start an interactive Slurm job for 30 minutes using 2 cores and 4 GB of memory to build the image. Make sure to set these resources correctly, some containers can take hours to build and require tens of GB of memory.
If you need more resources to build your container, please consider submitting your job using a Slurm submission script, for example:
#!/bin/bash -e
#SBATCH --job-name=apptainer_build
#SBATCH --partition=milan
#SBATCH --time=0-00:30:00
#SBATCH --mem=4GB
#SBATCH --cpus-per-task=2
module purge
module load Apptainer/1.1.2
apptainer build --fakeroot my_container.sif my_container.def
More information about how to submit a Slurm job is available in the Submitting your first job support page.
Known limitations
If your container uses RPM to install packages, i.e. is based on CentOS or Rocky Linux, you need to disable the APPTAINER_TMPDIR
environment variable (use unset APPTAINER_TMPDIR
) and request more memory for your Slurm job. Otherwise, RPM will crash due to an incompatibility with the nobackup
filesystem.
If you encounter the following error when using a base Docker image in your Apptainer definition file
While making image from oci registry: error fetching image to cache: while building SIF from layers: conveyor failed to get: unsupported image-specific operation on artifact with type "application/vnd.docker.container.image.v1+json"
it is likely due to an upstream issue (e.g. bad image on Dockerhub). In this case, try an older image version or a different base image.
Other limitations
This method, using fakeroot, is known to not work for all types of Apptainer/Singularity containers.
If you encounter an issue, please contact us at support@nesi.org.nz.