STAR-CCM+

STAR-CCM+ is a complete multi-disciplinary platform for the simulation of products and designs operating under real-world conditions.

STAR-CCM+ is available as a module on Apocrita.

Usage

To run the default version of STAR-CCM+, simply load the starccm module:

module load starccm

For usage documentation, run starccm+ -help

Example jobs

When running jobs, replace SIM-FILE in the examples below with the name of your simulation file.

CPU jobs

Serial job

Here is an example job running on 4 cores and 16G of ram on a single node:

#!/bin/bash
#$ -cwd
#$ -j y
#$ -pe smp 4
#$ -l h_rt=1:0:0
#$ -l h_vmem=4G

module load starccm
starccm+ -batchsystem sge -batch SIM-FILE

Parallel job

Here is an example of a multi-node job running on 96 cores and utilising the ddy-i infiniband island:

#!/bin/bash
#$ -cwd
#$ -j y
#$ -pe parallel 96
#$ -l infiniband=ddy-i
#$ -l h_rt=240:0:0

module load starccm
starccm+ -batchsystem sge -mpi intel -batch SIM-FILE

GPU job

Serial GPU job

Do not use NVIDIA MPS

STAR-CCM+ integrates the NVIDIA Multi-Process Service (MPS) but this should not be used on Apocrita as it interferes with correct GPU allocation and prevents other users on the same node from using their assigned GPU(s). Please ensure you add -gpgpu auto:nomps to all GPU jobs to ensure that MPS is disabled (see below example).

Here is an example job running on 8 CPU cores, 11G of ram per core and 1 GPU on a single node:

#!/bin/bash
#$ -cwd
#$ -j y
#$ -pe smp 8
#$ -l h_rt=1:0:0
#$ -l h_vmem=11G
#$ -l gpu=1

module load starccm
starccm+ -batchsystem sge -mpi intel -batch SIM-FILE -np ${NSLOTS} -gpgpu auto:nomps