Skip to content

Nodes Overview

internals

Apocrita runs a variety of different job types, and the cluster is comprised of different node types and queues to accommodate this. Unless you have a specific technical reason, you should avoid selecting a specific node type to run your job on. Some nodes are restricted depending on the source of funding.

Hyperthreading is not enabled on most compute nodes. See the "Threads" column within the below tables for details on the number of available cores and threads. Nodes with Hyperthreading enabled will show double the number of logical cores (threads) compared to the physical cores (cores) column.

The following nodes are open access, allowing jobs from any user.

Open Access Required variables Count Cores Threads RAM Accessible RAM Arch
ddy 56 48 48 384GB ~362GB Intel
nxv 5 32 32 512GB ~491GB Intel
nxg (volta) gpu 2 32 32 512GB ~491GB Intel
sbg (ampere) gpu 1 48 48 384GB ~362GB Intel
sbg (volta) gpu 2 32 64 384GB ~362GB Intel
sdx 52 36 36 384GB ~362GB Intel
smf highmem 8 24 24 768GB ~741GB Intel
srm highmem 2 36 36 768GB ~741GB Intel
Total 128 5160 5224 53.9TB

The following nodes are restricted to specific groups controlled by the purchasers of the nodes. Some are bulk purchases on behalf of a faculty (e.g Medicine & Dentistry and Science & Engineering), and jobs requested by members of the faculty will automatically be eligible for these nodes. Other nodes have been purchased by smaller research groups and requests for access to these nodes will need to be confirmed by the owner of the node (typically the P.I.) before access will be granted.

Restricted Access Required variables Count Cores Threads RAM Accessible RAM Arch
ddy 129 48 48 384GB ~362GB Intel
nxv 2 32 32 512GB ~491GB Intel
rdg (ampere) gpu 12 48 48 1024GB ~988GB Intel
rdg (hopper) gpu 1 32 64 512GB ~491GB Intel
sbg (ampere) gpu 2 32 64 384GB ~362GB Intel
sbg (ampere) gpu 1 48 48 384GB ~362GB Intel
sbg (ampere) gpu, cluster=andrena 16 48 48 384GB ~362GB Intel
sdv serial 4 24 24 192GB ~177GB Intel
sdv parallel infiniband=sdv-ii 8 24 24 192GB ~177GB Intel
srm highmem 1 48 48 1536GB ~1483GB Intel
xdg (hopper) gpu 2 64 64 1024GB ~988GB Intel
Total 178 8208 8304 76.5TB

Owned nodes complex

Users with access to owned nodes can ensure jobs run on an owned node by requesting -l owned in the job script or on the command line e.g. qlogin -l owned

The following table lists the CPU instruction sets supported by Apocrita Nodes. Note that AVX-512 is a family of extensions and an entry in this table may not mean that all AVX-512 extensions are supported on the node. To see whether a particular AVX-512 extension is supported you can check the CPU flags on a specific node.

Node Type SSE SSE2 SSSE3 SSE4 AVX AVX2 AVX-512
ddy
nxv
nxg
rdg
sbg
sdv
smf
srm
sdx
xdg