Python is an interpreted, interactive, object-oriented programming language. It incorporates modules, exceptions, dynamic typing, very high level dynamic data types, and classes. Python combines remarkable power with very clear syntax. It has interfaces to many system calls and libraries, as well as to various window systems, and is extensible in C or C++.
Python is available as a module on Apocrita.
There are multiple versions of Python available as modules including Python 2 and Python 3, Python 2 is still very common but is now in legacy mode, Python 3 is under active development and has a large number of improvements.
While some code will work in both versions there are a number of incompatibilities so the version of Python you need may depend on the code you are running. For new code bases Python 3 is strongly recommended.
To run the latest installed version of Python (the latest Python 3), simply load the Python module:
module load python
then run Python with a script file:
We recommend using the Python command for the specific Python version you
python2.7) rather than using the default
python, as the
default may change.
Here is an example job running on 1 cores.
#!/bin/sh #$ -cwd #$ -j y #$ -pe smp 1 #$ -l h_rt=4:0:0 #$ -l h_vmem=2G module load python python example.py
Serial job - Virtualenv¶
To use a Python virtualenv in a job script you need to activate the virtualenv:
#!/bin/sh #$ -cwd #$ -j y #$ -pe smp 1 #$ -l h_rt=4:0:0 #$ -l h_vmem=2G # Activate virtualenv source <envname>/bin/activate # Run Python script python example.py
Serial job - Cutadapt¶
This is an example of using cutadapt installed in a virtualenv.
#!/bin/sh #$ -cwd #$ -j y #$ -pe smp 1 #$ -l h_rt=1:0:0 #$ -l h_vmem=1G # Activate virtualenv source cutadapt/bin/activate # Run cutadapt cutadapt raw_data.fq.gz
Installing Python Packages¶
Whilst packages can be installed locally using
recommend using virtualenvs to ensure clean environments.
You can use virtualenv to set up your own virtual Python environment over which you have full control. This allows you to use a specific Python version and its own set of packages. Once the virtual environment is set up, you need activate it when you log in and then you can use python, pip and easy_install.
# virtualenv is installed as part of the python module $ module load python # Set up an environment called <envname> $ virtualenv --include-lib <envname> # Activate the environment $ . <envname>/bin/activate # Use Python / pip etc. in the environment (<envname>)$ pip install <module> # Run Code (<envname>)$ python example.py # Stop using the environment $ deactivate
To upgrade an installed version of a module run:
$ source <envname>/bin/activate (<envname>)$ pip install <module> --upgrade
The version of virtualenv on Apocrita has been patched to include shared
libraries inside the virtualenv, in order to use this functionality you
must provide the
--include-lib option when creating the virtualenv.
Without the shared libraries virtualenvs will not work unless the correct version of python has been loaded as a module before activating the virtualenv.
Setting up numpy¶
Using virtualenv, it is straight-forward to install a personal copy of numpy:
$ module load python $ virtualenv --include-lib numpy $ source numpy/bin/activate (numpy)$ pip install numpy
Setting up cutadapt¶
To install Cutadapt using a virtualenv run the following commands:
$ module load python $ virtualenv --include-lib cutadapt $ source cutadapt/bin/activate (cutadapt)$ pip install cutadapt (cutadapt)$ cutadapt raw_data.fq.gz