Entitlements to NeSI's Computing Capacity

What entitlements are there to NeSI's computing capacity and why?

NeSI is set up under a Crown Funding Agreement. That agreement governs how NeSI HPC Compute and Analytics Services are to be allocated across the Collaborating Institutions (University of Auckland, University of Otago, Manaaki Whenua Landcare Research and the National Institute of Water and Atmospheric Research) and other users of this Service as specified below.  

Allocation Class Entitlement to consumed compute capacity (by dollar value)
Collaborating Institutions At most 60%
Subscriptions (provides access to Non-Collaborators on terms similar to Collaborators) and Commercial contracts At least 20%
Merit (which includes Postgraduate and Proposal Development access) At least 20%

The partitioning is on the basis of dollar value because the cost of producing a core hour differs between Mahuika and Māui.

How do we manage the Allocation Classes?

We manage entitlements in two ways. The first is by managing, as a function of Allocation Class, the amount of computing capacity allocated to individual projects.

  • We limit the total amount of capacity awarded as Proposal Development allocations by restricting such allocations to new projects only and keeping them small and short.
  • We use quarterly calls to manage our commitments in the Merit and Postgraduate classes. Postgraduate allocations will only be made if there is capacity left over in the Government-funded pool after all suitable Merit allocations have been awarded.
  • Collaborating and Subscribing institutions are responsible for managing their own allocations up to their entitlement, but are subject to our overall fair share governance (see below).

The second way is by using Slurm's Fair Share mechanism to prioritise individual jobs. Slurm has been configured so that it balances Allocation Classes against each other according to the entitlements shown above, as well as balancing projects against each other within their respective Allocation Classes. Fair Share aims to manage each project's rate of usage of our core hour resource, rather than the overall amount of computing resource used by the project.

We do not reject or postpone jobs in Allocations Classes that are consuming more of the consumed capacity than they are entitled to if by doing so we would leave parts of our facilities idle.

Labels: author
Was this article helpful?
0 out of 0 found this helpful