Māui Slurm Partitions

NeSI Access to Māui (XC50) and Māui_Ancil (CS500)

Partitions on these systems that may be used for NeSI workloads carry the prefix "nesi_". 

Definitions

CPU - Logical Core, also known as a hardware thread. Referred to as a "CPU" in the Slurm documentation.  Since Hyperthreading is enabled, there are two CPUs per physical core.

Māui (XC50) Slurm Partitions

Nodes are not shared between jobs on Māui, so the minimum charging unit is node-hours, where 1 node-hour is 40 core-hours, or 80 Slurm CPU-hours.

There is only one partition available to NeSI jobs:

 Name 

Nodes

Max Walltime

Avail / Node

Max / Account

Description

nesi_research

264

24 hours

80 CPUs

80 or 160 GB RAM

240 nodes

1200 node-hours running

Standard partition for all NeSI jobs.

Limits

As a consequence of the above limit on the node-hours reserved by your running jobs (GrpTRESRunMins in Slurm documentation, shown in squeue output when you hit it as the reason "AssocGrpCPURunMinutes" ) you can occupy more nodes simultaneously if your jobs request a shorter time limit:

nodes hours node-hours limits reached
1 24 24 24 hours
50 24 1200 1200 node-hours, 24 hours
100 12 1200 1200 node-hours
240 5 1200 1200 node-hours, 240 nodes
240 1 240 240 nodes 

Most of the time job priority will be the most important influence on how long your jobs have to wait - the above limits are just backstops to ensure that Maui's resources are not all committed too far into the future, so that debug and other higher-priority jobs can start reasonably quickly.

Quality of Service: nesi_debug

Each job has a "QoS", with the default QoS for a job being determined by the allocation class of its project. Specifying --qos=nesi_debug will override that and give the job very high priority, but is subject to strict limits: 20 minutes per job, and only 1 job at a time per user. Debug jobs are limited to 8 nodes.

maui_ancil Slurm Partitions

Partition  

Nodes

Max Walltime

Avail / Node

Max / Job

Max / User

Description

nesi_prepost

4

24 hours

80 CPUs

720 GB RAM

20 CPUs

700 GB RAM

80 CPUs

700 GB RAM

Pre and post processing tasks.

nesi_gpu

4 to 5

72 hours

4 CPUs

12 GB RAM

1 GPU

4 CPUs

12 GB RAM

1 GPU

4 CPUs

12 GB RAM

1 GPU

GPU jobs and visualisation. 

nesi_igpu

0 to 1

2 hours

4 CPUs

12 GB RAM

1 GPU

4 CPUs

12 GB RAM

1 GPU

4 CPUs

12 GB RAM

1 GPU

Interactive GPU access 7am - 8pm.

 

 

Labels: maui slurm
Was this article helpful?
0 out of 0 found this helpful