Skip to content

Anaconda

Anaconda is a free and open source distribution of the Python and R programming languages. It contains a number of useful packages which make it popular in fields like data science, machine learning and scientific computing.

Anaconda is available as a module on Apocrita.

Anaconda Versions

There are multiple versions of Anaconda available as modules including support for Python 2 and 3.

While some code will work in both versions there are a number of incompatibilities so the version of Python you need may depend on the code you are running. For new code bases Python 3 is strongly recommended.

Usage

Loading the module

To run the latest installed version of Anaconda (Python 3), simply load the anaconda3 module:

module load anaconda3

You can then check your python version:

$ which python
/share/apps/centos7/anaconda3/5.2.0/bin/python
$ python -V
Python 3.6.5 :: Anaconda, Inc.

Conda package manager

In addition to the Conda package manager, Anaconda provides a number of package management tools including the familiar Python ones: pip and EasyInstall.

Environments

When working with Anaconda you'll probably want to update installed packages as well as installing some new ones. Since Anaconda is installed to shared storage and you don't have write access there, you'll need to create Anaconda environments somewhere where you do have write access (for example home directories or lab shares).

Python virtualenv environments can also be created using Anaconda because it also provides the Python programming language as well as Conda environments.

Listing environments

You can list existing Conda environments as follows:

$ conda env list
# conda environments:
#
base                  *  /share/apps/centos7/anaconda3/5.2.0

Let's set up a new location for environments, we do this by editing .condarc which is in your home directory:

$ cat ~/.condarc
channels:
  - defaults
ssl_verify: true
envs_dirs:
  - /data/example/anaconda

Here we have specified /data/example/anaconda as a location to install to.

Creating a new environment

Let's create a new environment:

$ conda create --quiet --yes --name myenv
Solving environment: ...working... done

## Package Plan ##

  environment location: /data/home/abc123/.conda/envs/myenv

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done

Here we have specified that we want a new environment called "myenv". We've not requested any packages. Suppose we wanted a specific point release of python installed. We could do:

$ conda create --quiet --yes --name myenv python=3.7
Solving environment: ...working... done

## Package Plan ##

  environment location: /data/home/abc123/.conda/envs//myenv

  added / updated specs:
    - python=3.7

The following packages will be downloaded:

(package list redacted)

The following NEW packages will be INSTALLED:

(package list redacted)

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done

Let's see which version of python we have in our $PATH:

$ which python
/share/apps/centos7/anaconda3/5.2.0/bin/python
$ python -V
Python 3.6.5 :: Anaconda, Inc.

Let's activate the new environment. The asterisk denotes the currently active environment:

$ source activate myenv
$ conda env list
# conda environments:
#
myenv                 *  /data/home/abc123/.conda/envs/myenv
base                     /share/apps/centos7/anaconda3/5.2.0

And let's check python again:

$ which python
~/.conda/envs/myenv/bin/python
$ python -V
Python 3.7.0 :: Anaconda, Inc.

Installing packages using conda

You can install other packages from Conda into your environment.

Packages must be installed into activated environments

Packages must be installed into your personal environments. If package installs are attempted without first activating an environment, a permission error will be shown.

The follow example demonstrates installation of the scipy package:

$ conda activate myenv
(myenv) $ conda install --quiet --yes scipy
Solving environment: ...working... done

## Package Plan ##

  environment location: /data/home/abc123/.conda/envs/myenv

  added / updated specs:
    - scipy

The following NEW packages will be INSTALLED:

(package list redacted for brevity)

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done

$ python -c "import scipy; print(scipy.__version__)"
1.1.0

The conda list command will show all of the packages installed into your environment.

Example job

Serial job

Here is an example job running on 1 core:

#!/bin/bash
#$ -cwd
#$ -j y
#$ -pe smp 1
#$ -l h_rt=1:0:0
#$ -l h_vmem=1G

module load anaconda3
python example.py

Serial job demonstrating environment switching

This example shows that you can switch Conda environments in a job script:

#!/bin/bash
#$ -cwd
#$ -j y
#$ -pe smp 1
#$ -l h_rt=1:0:0
#$ -l h_vmem=1G

module load anaconda3
conda env list
source activate myenv
conda env list

If we view the job output:

# conda environments:
#
myenv                    /data/home/abc123/.conda/envs/myenv
base                  *  /share/apps/centos7/anaconda3/5.2.0

# conda environments:
#
myenv                 *  /data/home/abc123/.conda/envs/myenv
base                     /share/apps/centos7/anaconda3/5.2.0

Here we can see from the output of the conda env list commands, that we are successfully switching environments inside a job script.

References