Python is an interpreted, interactive, object-oriented programming language. It incorporates modules, exceptions, dynamic typing, very high level dynamic data types, and classes. Python combines remarkable power with very clear syntax. It has interfaces to many system calls and libraries, as well as to various window systems, and is extensible in C or C++.
Python is available as a module on Apocrita.
There are multiple versions of Python available as modules including Python 2 and Python 3, Python 2 is still very common but is now in legacy mode, Python 3 is under active development and has a large number of improvements.
While some code will work in both versions there are a number of incompatibilities so the version of Python you need may depend on the code you are running. For new code bases Python 3 is strongly recommended.
To run the default installed version of Python (the latest Python 3), simply load the Python module:
module load python
then run Python with a script file:
We recommend using the Python command for the specific Python version you
python2.7) rather than using the default
python, as the
default may change.
Here is an example job running on 1 cores.
#!/bin/sh #$ -cwd #$ -j y #$ -pe smp 1 #$ -l h_rt=4:0:0 #$ -l h_vmem=2G module load python python example.py
Serial job - virtualenv¶
To use a Python virtualenv in a job script you need to activate the virtualenv:
#!/bin/sh #$ -cwd #$ -j y #$ -pe smp 1 #$ -l h_rt=4:0:0 #$ -l h_vmem=2G # Activate virtualenv source <envname>/bin/activate # Run Python script python example.py
Serial job - Cutadapt¶
This is an example of using cutadapt installed in a virtualenv.
#!/bin/sh #$ -cwd #$ -j y #$ -pe smp 1 #$ -l h_rt=1:0:0 #$ -l h_vmem=1G # Activate virtualenv source cutadapt/bin/activate # Run cutadapt cutadapt raw_data.fq.gz
Installing Python packages¶
Whilst packages can be installed locally using
recommend using virtualenvs to ensure clean environments.
You can use virtualenv to set up your own virtual Python environment over which you have full control. This allows you to use a specific Python version and its own set of packages. Once the virtual environment is set up, you need activate it when you log in and then you can use python, pip and easy_install.
# virtualenv is installed as part of the python module $ module load python # Set up an environment called <envname> $ virtualenv --include-lib <envname> # Activate the environment $ source <envname>/bin/activate # Use Python / pip etc. in the environment (<envname>)$ pip install <module> # Run Code (<envname>)$ python example.py # Stop using the environment $ deactivate
To install a specific version of a module run:
$ source <envname>/bin/activate (<envname>)$ pip install <module>==<version>
To upgrade an installed version of a module run:
$ source <envname>/bin/activate (<envname>)$ pip install <module> --upgrade
The version of virtualenv on Apocrita has been patched to include shared
libraries inside the virtualenv, in order to use this functionality you
must provide the
--include-lib option when creating the virtualenv.
Without the shared libraries virtualenvs will not work unless the correct version of python has been loaded as a module before activating the virtualenv.
Setting up numpy¶
Using virtualenv, it is straight-forward to install a personal copy of numpy:
$ module load python $ virtualenv --include-lib numpy $ source numpy/bin/activate (numpy)$ pip install numpy
Setting up cutadapt¶
To install Cutadapt using a virtualenv run the following commands:
$ module load python $ virtualenv --include-lib cutadapt $ source cutadapt/bin/activate (cutadapt)$ pip install cutadapt (cutadapt)$ cutadapt raw_data.fq.gz
Setting up matplotlib¶
To install Matplotlib using a virtualenv, run the following commands:
$ module load python $ virtualenv --include-lib matplotlib $ source matplotlib/bin/activate (matplotlib)$ pip install matplotlib
Selecting the matplotlib backend
As python modules on Apocrita do not include Tkinter support, matplotlib requires a non-interactive backend to write output to a file rather than rendering results in a graphical window.
The following commands instruct matplotlib to write output to a file using the
Agg backend (after creating the virtualenv as instructed above):
(matplotlib)$ python >>> import matplotlib >>> matplotlib.use('Agg') >>> import matplotlib.pyplot as plt
For more information regarding matplotlib backends, please see: here.