XDG Nodes¶
We have 2 Dell PowerEdge XE8640 nodes with Nvidia Hopper H100 80GB cards for GPU jobs.
Accessing the XDG nodes
The XDG nodes are restricted to users within the Science and Engineering (S&E) faculty only.
GPU high memory
The XDG nodes have more RAM available than standard GPU nodes, supporting
up to 240G RAM per GPU (12 cores). Authorised users may use the
-l gpuhighmem submission with up to 20G per core (-l h_vmem=20G) to
request larger amounts of RAM on these nodes.
Requesting 12 cores per GPU
By default, the XDG nodes will only accept jobs requesting 8 cores per GPU.
To request 12 cores per GPU, ensure the -l node_type=xdg parameter is
requested, otherwise your job will be rejected upon submission. The
-l gpuhighmem option may be used as an alternative for node_type, if
submitting to either the RDG or XDG S&E nodes.
| XDG | Dell PowerEdge XE8640 |
|---|---|
| Processor | 2 x 32 Core Intel Xeon Platinum 8462Y+ (Sapphire Rapids) |
| Cores/Node | 64 |
| RAM | 1TB |
| Accessible RAM | ~988GB |
| TMP Size | 3TB |
| Interconnect | 25Gb Ethernet |
| GPU | 4 x NVIDIA Hopper H100 |
| GPU architecture | Hopper |
| Form Factor | HBM3 (High-bandwidth memory) |
| Tensor Cores | 528 4th Generation |
| CUDA Cores | 14,592 |
| GPU Memory | 80GiB per GPU |
| CUDA Compute | 9.0 (CUDA version 11.8 or greater required) |