Nodes Overview

Apocrita runs a variety of different job types, and the cluster is comprised of different node types and queues to accommodate this. Unless you have a specific technical reason, you should avoid selecting a specific node type to run your job on. Some nodes are restricted depending on the source of funding.

The following nodes are open access, allowing jobs from any user.

Open Access	Required variables	Count	Cores	RAM	Accessible RAM	Arch
ddy		56	48	384GB	~362GB	Intel
emf	highmem	14	24	768GB	~741GB	Intel
nxv		5	32	512GB	~491GB	Intel
nxg (volta)	gpu	2	32	512GB	~491GB	Intel
sbg (ampere)	gpu	1	48	384GB	~365GB	Intel
sbg (volta)	gpu	2	32	384GB	~365GB	Intel
sdv	infiniband=sdv-i	31	24	192GB	~176GB	Intel
sdx		52	36	384GB	~362GB	Intel
srm	highmem	2	36	768GB	~741GB	Intel
Total		165	6048	64.4TB

The following nodes are restricted to specific groups controlled by the purchasers of the nodes. Some are bulk purchases on behalf of a faculty (e.g Medicine & Dentistry and Science & Engineering), and jobs requested by members of the faculty will automatically be eligible for these nodes. Other nodes have been purchased by smaller research groups and requests for access to these nodes will need to be confirmed by the owner of the node (typically the P.I.) before access will be granted.

Restricted Access	Required variables	Count	Cores	RAM	Accessible RAM	Arch
ddy		131	48	384GB	~362GB	Intel
nxv		2	32	512GB	~491GB	Intel
rdg (hopper)	gpu	1	32	512GB	~491GB	Intel
sbg (ampere)	gpu	3	48	384GB	~362GB	Intel
sbg (ampere)	gpu, cluster=andrena	16	48	384GB	~362GB	Intel
sdv serial		10	24	192GB	~177GB	Intel
sdv parallel	infiniband=sdv-ii	15	24	192GB	~177GB	Intel
srm	highmem	1	48	1536GB	~1483GB	Intel
Total		179	7944	64.5TB

Owned nodes complex

Users with access to owned nodes can ensure jobs run on an owned node by requesting -l owned in the job script or on the command line e.g. qlogin -l owned

The following table lists the CPU instruction sets supported by Apocrita Nodes. Note that AVX-512 is a family of extensions and an entry in this table may not mean that all AVX-512 extensions are supported on the node. To see whether a particular AVX-512 extension is supported you can check the CPU flags on a specific node.

Node Type	SSE	SSE2	SSSE3	SSE4	AVX	AVX2	AVX-512
ddy
emf
nxv
nxg
sdv
srm
sbg
sdx

Hyperthreading is not enabled on compute nodes. Jobs are therefore allocated real cores.