Skip to content

TensorFlow

TensorFlow is an open source library for machine learning.

Versions

CPU and GPU versions of the TensorFlow python library are available on Apocrita, and require different methods to run.

GPU version is recommended

TensorFlow programs typically run much faster on a GPU.

CPU version

The CPU-only version of TensorFlow can be installed using pip in a virtualenv.

module load python/3.6.3
virtualenv --include-lib tensorcpu
source tensorcpu/bin/activate
pip install tensorflow

To obtain a specific TensorFlow version, a link to the specific python package can be obtained from the TensorFlow website. For example, the TensorFlow 1.4.1 package for python 3.6 can be installed with:

pip install \
https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.4.1-cp36-cp36m-linux_x86_64.whl

GPU version

GPU node access

Researchers need to request permission to be added to the list of GPU node users.

The GPU version of TensorFlow is provided as a Singularity container.

A list of available versions can be found in the /data/containers/tensorflow directory on Apocrita.

Example jobs

CPU-only example

The installation can be verified with a simple job:

#!/bin/bash
#$ -cwd
#$ -pe smp 1
#$ -l h_rt=1:0:0
#$ -l h_vmem=1G

module load python
source ~/tensorcpu/bin/activate
python -c 'import tensorflow as tf; print(tf.__version__)'

Submit the script to the job scheduler and the TensorFlow version number will be recorded in the job output file.

Warnings received when running CPU TensorFlow

You may receive a warning RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6. This has been acknowledged by TensorFlow developers and can be disregarded as it does not affect functionality.

Simple GPU job

#!/bin/bash
#$ -cwd
#$ -j y
#$ -pe smp 16
#$ -l h_rt=1:0:0
#$ -l gpu=1

# Assign the correct GPU card (see https://docs.hpc.qmul.ac.uk/using/usingGPU )
export CUDA_VISIBLE_DEVICES=${SGE_HGR_gpu// /,}
module load singularity
singularity exec --nv \
/data/containers/tensorflow/tensorflow-1.3-anaconda-python3-ubuntu-16.04.img \
python -c 'import tensorflow as tf; print(tf.__version__)'

Singularity GPU support

The --nv flag is required for GPU support and passes through the appropriate GPU drivers and libraries from the host to the container.

GPU Machine learning example

This example makes use of the TensorFlow tutorial obtained by performing a git clone https://github.com/tensorflow/models and uses 1 GPU on a node.

Using Multiple GPUs

Specifying 2 GPUs does not automatically mean that both will be used by the application. See the TensorFlow documentation for more information about constructing multi-GPU models.

#!/bin/bash
#$ -cwd
#$ -j y
#$ -pe smp 16
#$ -l h_rt=1:0:0
#$ -l gpu=1

export CUDA_VISIBLE_DEVICES=${SGE_HGR_gpu// /,}
module load singularity
singularity exec --nv \
/data/containers/tensorflow/tensorflow-1.3-anaconda-python3-ubuntu-16.04.img \
python ./models/tutorials/image/mnist/convolutional.py

References