AMD GPU Nodes

We currently host only few AMD GPUs that are available for benchmarking.

Type	Spec	Node	Remarks
MI100	Milan 7703 processor with 1x MI100 GPU	ga004	Integrated into COSMA. Partition cosma8-shm2. Access via SLURM only.
MI200	AMD EPYC 7713 with 2x MI200	ga005, ga006	Integrated into COSMA. Partition cosma8-shm2. Access via SLURM only.
MI50	AMD EPYC 7282 with 6x AMD MI50	ga003	Integrated into COSMA. Interactive access only.

Cosma – gn003

These nodes are available for interactive access only. Log into a COSMA login node and ssh straightforwardly into the GPU nodes that you want to experiment with.

Cosma – ga004-ga006

Please log into COSMA8 login nodes and submit batch jobs from there via sbatch or salloc.

Batch jobs

The nodes are available within the cosma8-shm2 partition and have to be selected specifically within your SLURM script:

#SBATCH -p cosma8-shm2
#SBATCH -w ga004

Alternatively, you can use the –include or –exclude settings to pick the exact node.

Environment

ROCm toolchain

To be written.

Intel toolchain

The Intel toolchain on the AMD nodes works and yields good performance. However, the compiler has to be told about the architecture explicitly:

-O3 -fomit-frame-pointer -fstrict-aliasing -ffast-math -funroll-loops -axCOMMON-AVX512 -march=x86-64-v4 -mavx512vbmi

Funding and acknowledgements

The AMD test nodes have been installed in collaboration and as addendum to DiRAC@Durham facility managed by the Institute for Computational Cosmology on behalf of the STFC DiRAC HPC Facility (www.dirac.ac.uk). DiRAC equipment was funded by BEIS capital funding via STFC capital grants ST/P002293/1, ST/R002371/1 and ST/S002502/1, Durham University and STFC operations grant ST/R000832/1. DiRAC is part of the National e-Infrastructure.