ABAQUS

Licences

A list of commands can be found with:

abaqus help

Tips

Required ABAQUS licences can be determined by this simple and intuitive formula ⌊ 5 x N0.422 where N is number of CPUs.

Solver Compatibility

Not all solvers are compatible with all types of parallelisation.

  mp_mode=threads mp_mode=mpi
Element operations
Iterative solver
Direct solver
Lanczos solver

Note

If your input files were created using an older version of ABAQUS you will need to update them using the command,

abaqus -upgrade -job new_job_name -odb old.odb

or

abaqus -upgrade -job new_job_name -inp old.inp

Example scripts

Serial


For when only one CPU is required, generally as part of an job array.

 

#!/bin/bash -e

#SBATCH --job-name      ABAQUS-Shared
#SBATCH --time 00:05:00 # Walltime #SBATCH --cpus-per-task 1 #SBATCH --mem 512MB # total mem #SBATCH --hint nomultithread # Hyperthreading disabled module load ABAQUS/2018
abaqus job="propeller_s4rs_c3d8r" verbose=2 interactive

Shared Memory


mp_mode=threads

Uses a nodes shared memory for communication. 

May have a small speedup compared to MPI when using a low number of CPUs, scales poorly. Needs significantly less memory than MPI.

Hyperthreading may be enabled if using shared memory but it is not recommended.
#!/bin/bash -e

#SBATCH --job-name      ABAQUS-Shared
#SBATCH --time 00:05:00 # Walltime #SBATCH --cpus-per-task 4 #SBATCH --mem 2048MB # total mem #SBATCH --hint nomultithread # Hyperthreading disabled module load ABAQUS/2018
abaqus job="propeller_s4rs_c3d8r" verbose=2 interactive \
cpus=${SLURM_CPUS_PER_TASK} mp_mode=threads

Shared Memory + UDF


Shared memory run with user defined function (fortran or C). 

user=<name_of_function> 

Function will be compiled at start of run. 

You may need to chance the function suffix if you usually compile on windows.

#!/bin/bash -e

#SBATCH --job-name      ABAQUS-Shared
#SBATCH --time 00:05:00 # Walltime #SBATCH --cpus-per-task 4 #SBATCH --mem 2048MB # total mem #SBATCH --hint nomultithread # Hyperthreading disabled
module load gimkl module load ABAQUS/2018
abaqus job="propeller_s4rs_c3d8r" user=my_udf.f90 verbose=2 interactive \
cpus=${SLURM_CPUS_PER_TASK} mp_mode=threads

Distributed Memory


mp_mode=mpi

Multiple processes each with a single thread.

Not limited to one node.
Model will be segmented into -np pieces which should be equal to --ntasks.

Each task could be running on a different node leading to increased communication overhead
.Jobs can be limited to a single node by adding  --nodes=1 however this will increase your time in the queue as contiguous cpu's are harder to schedule.

This is the default method if mp_mode is left unspecified.

#!/bin/bash -e

#SBATCH --job-name      ABAQUS-Distributed 
#SBATCH --time 00:05:00 # Walltime #SBATCH --ntasks 8 #SBATCH --mem-per-cpu 512MB # Each CPU needs it's own. #SBATCH --hint nomultithread # Hyperthreading disabled
#SBATCH --nodes 1 module load ABAQUS/2018
abaqus job="propeller_s4rs_c3d8r" verbose=2 interactive \
cpus=${SLURM_NTASKS} mp_mode=mpi

Shared Memory + GPUs


Must run on a GPU node.

The GPU nodes are limited to 16 CPUs

In order for the GPUs to be worthwhile, you should see a speedup equivalent to 56 CPU's per GPU used.

#!/bin/bash -e

#SBATCH --job-name      ABAQUS-Shared
#SBATCH --time 00:05:00 # Walltime #SBATCH --cpus-per-task 4 #SBATCH --mem 2048MB # total mem #SBATCH --hint nomultithread # Hyperthreading disabled
#SBATCH --partition gpu
#SBATCH --gres gpu:2 module load ABAQUS/2018
abaqus job="propeller_s4rs_c3d8r" verbose=2 interactive \
cpus=${SLURM_CPUS_PER_TASK} gpus=2 mp_mode=threads

User Defined Functions 

User defined functions (UDFs) can be included on the command line with the argument user=<filename> where <filename> is the C or fortran source code.

Make sure you have loaded the correct compiler with module load (for example module load gimkl/2020a). Extra compiler options can be set in your local abaqus_v6.env file. 

Environment file

The ABAQUS environment file contains a number of parameters that define how the your job will run, some of these you may with to change.

These parameters are read, 

../ABAQUS/SMA/site/abaqus_v6.env Set by NeSI and cannot be changed.

~/abaqus_v6.env (your home directory) If exists, will be used in all jobs submitted by you.

<working directory>/abaqus_v6.env If exists, will used in this job only.

 

Useful Links

 

ABAQUS_speedup_SharedVMPI.png

 

Note: Hyperthreading off, testing done on small mechanical FEA model. Results highly model dependant. Do your own tests.

Labels: mahuika application tier2 engineering
Was this article helpful?
1 out of 2 found this helpful
a.homepage:before