NeSI Access to Māui (XC50) and Māui_Ancil (CS500)
Partitions on these systems that may be used for NeSI workloads carry the prefix "nesi_".
Definitions
CPU - Logical Core, also known as a hardware thread. Referred to as a "CPU" in the Slurm documentation. Since Hyperthreading is enabled, there are two CPUs per physical core.
Māui (XC50) Slurm Partitions
Nodes are not shared between jobs on Māui, so the minimum charging unit is node-hours, where 1 node-hour is 40 core-hours, or 80 Slurm CPU-hours.
There is only one partition available to NeSI jobs:
|
Name |
Nodes |
Max Walltime |
Avail / Node |
Max / Account |
Description |
|
nesi_research |
264 |
24 hours |
80 CPUs 90 or 180 GB RAM |
240 nodes 1200 node-hours running |
Standard partition for all NeSI jobs. |
Limits
As a consequence of the above limit on the node-hours reserved by your running jobs (GrpTRESRunMins in Slurm documentation, shown in squeue output when you hit it as the reason "AssocGrpCPURunMinutes" ) you can occupy more nodes simultaneously if your jobs request a shorter time limit:
| nodes | hours | node-hours | limits reached |
| 1 | 24 | 24 | 24 hours |
| 50 | 24 | 1200 | 1200 node-hours, 24 hours |
| 100 | 12 | 1200 | 1200 node-hours |
| 240 | 5 | 1200 | 1200 node-hours, 240 nodes |
| 240 | 1 | 240 | 240 nodes |
Most of the time job priority will be the most important influence on how long your jobs have to wait - the above limits are just backstops to ensure that Maui's resources are not all committed too far into the future, so that debug and other higher-priority jobs can start reasonably quickly.
Debug QoS
Each job has a "QoS", with the default QoS for a job being determined by the allocation class of its project. Specifying --qos=debug will override that and give the job very high priority, but is subject to strict limits: 15 minutes per job, and only 1 job at a time per user. Debug jobs are limited to 2 nodes.
maui_ancil Slurm Partitions
|
Partition |
Nodes |
Max Walltime |
Avail / Node |
Max / Job |
Max / User |
Description |
|
nesi_prepost |
4 |
24 hours |
80 CPUs 720 GB RAM |
20 CPUs 700 GB RAM |
80 CPUs 700 GB RAM |
Pre and post processing tasks. |
|
nesi_gpu |
4 to 5 |
72 hours |
4 CPUs 12 GB RAM 1 GPU |
4 CPUs 12 GB RAM 1 GPU |
4 CPUs 12 GB RAM 1 GPU |
GPU jobs and visualisation. |
|
nesi_igpu |
0 to 1 |
2 hours |
4 CPUs 12 GB RAM 1 GPU |
4 CPUs 12 GB RAM 1 GPU |
4 CPUs 12 GB RAM 1 GPU |
Interactive GPU access 7am - 8pm. |