Python

Notitle

Python is an interpreted, interactive, object-oriented programming language. It incorporates modules, exceptions, dynamic typing, very high level dynamic data types, and classes. Python combines remarkable power with very clear syntax. It has interfaces to many system calls and libraries, as well as to various window systems, and is extensible in C or C++.

Python is available as a module on Apocrita.

Python Versions

There are multiple versions of Python available as modules including Python 2 and Python 3, Python 2 is still very common but is now in legacy mode, Python 3 is under active development and has a large number of improvements.

While some code will work in both versions there are a number of incompatibilities so the version of Python you need may depend on the code you are running. For new code bases Python 3 is strongly recommended.

Usage

To run the latest installed version of Python (the latest Python 3), simply load the Python module:

module load python

then run Python with a script file:

python example.py

Python Command

We recommend using the Python command for the specific Python version you require (e.g. python2.7) rather than using the default python, as the default may change.

Example jobs

Serial job

Here is an example job running on 1 cores.

#!/bin/sh
#$ -cwd
#$ -j y
#$ -pe smp 1
#$ -l h_rt=4:0:0
#$ -l h_vmem=2G

module load python
python example.py

Serial job - Virtualenv

To use a Python virtualenv in a job script you need to activate the virtualenv:

#!/bin/sh
#$ -cwd
#$ -j y
#$ -pe smp 1
#$ -l h_rt=4:0:0
#$ -l h_vmem=2G

# Activate virtualenv
source <envname>/bin/activate

# Run Python script
python example.py

Serial job - Cutadapt

This is an example of using cutadapt installed in a virtualenv.

#!/bin/sh
#$ -cwd
#$ -j y
#$ -pe smp 1
#$ -l h_rt=1:0:0
#$ -l h_vmem=1G

# Activate virtualenv
source cutadapt/bin/activate

# Run cutadapt
cutadapt raw_data.fq.gz

Installing Python Packages

Whilst packages can be installed locally using pip and easy_install we recommend using virtualenvs to ensure clean environments.

Virtualenv

You can use virtualenv to set up your own virtual Python environment over which you have full control. This allows you to use a specific Python version and its own set of packages. Once the virtual environment is set up, you need activate it when you log in and then you can use python, pip and easy_install.

# virtualenv is installed as part of the python module
$ module load python
# Set up an environment called <envname>
$ virtualenv <envname>
# Activate the environment
$ . <envname>/bin/activate
# Use Python / pip etc. in the environment
(<envname>)$ pip install <module>
# Run Code
(<envname>)$ python example.py
# Stop using the environment
$ deactivate

To upgrade an installed version of a module run:

$ source <envname>/bin/activate
(<envname>)$ pip install <module> --upgrade

Setting up numpy

Using virtualenv, it is straight-forward to install a personal copy of numpy:

$ module load python
$ virtualenv numpy
$ source numpy/bin/activate
(numpy)$ pip install numpy

Setting up cutadapt

To install Cutadapt using a virtualenv run the following commands:

$ module load python
$ virtualenv cutadapt
$ source cutadapt/bin/activate
(cutadapt)$ pip install cutadapt
(cutadapt)$ cutadapt raw_data.fq.gz

References