Using Jupyter via OnDemand¶
Jupyter is an interactive web-based environment for creating and using Jupyter notebooks, documents containing live code and visualisations. While many languages are supported by Jupyter, Python notebooks are very commonly used. Please refer to the overview section for instructions on how to login to OnDemand.
Starting a Jupyter session¶
If your code does not specifically use GPU acceleration and/or you do not have access to GPU-enabled nodes, then choose "Jupyter Notebook CPU" from the Interactive Apps list.
Here you can choose the number of cores and quantity of RAM required.
Request 1 core for most Jupyter jobs
Many Jupyter notebooks will only use 1 core, therefore any additional cores requested will be allocated but not utilised.
Choosing a 1 hour maximum running time is the best option for getting a session quickly, unless you have access to owned nodes which may also provide sessions immediately for up to 240hrs if resources are available.
Once clicking Launch, the request will be queued, and when resources have been allocated, you will be presented with the option to open the session.
Here you can see the compute host that the session has been scheduled on, and time remaining for the session.
Clicking Connect to Jupyter will open the familiar Jupyter session window, from which you can open notebooks.
And in action:
The Anaconda suite, which we are using, provides a lot of popular Python packages as standard. However, if you need you use modules outside of the default list provided by Anaconda, you will need to switch environments from within the Jupyter notebook.
We will demonstrate this by attempting to use the
emoji module to display
emojis. A default Anaconda environment does not contain this package.
Listing installed packages
You can inspect the list of packages provided by the default Anaconda
installation from within a terminal session on Apocrita by loading the
module load anaconda3 followed by the
conda list command
In a Jupyter notebook, attempting to load the
emoji module will fail.
We will need to create a new conda environment containing our desired packages.
emoji package into an activated environment will allow us to
use it on the command line interface, but to also make it visible to Jupyter we
ipykernel package, and a command to activate it.
We can perform this on Apocrita as follows:
module load anaconda3 # use -y to accept the y/n question, -q to reduce verbosity conda create -n emoji_env -y -q conda activate emoji_env # add the desired packages to the env # we need to specify the conda-forge channel to install the emoji package conda install -c conda-forge emoji ipykernel -y -q # make the env visible to Jupyter python -m ipykernel install --user --name emoji_env # deactivate your env conda deactivate
Note that the
ipykernel package and subsequent command are only required to
make the environment visible to Jupyter.
If you already have an existing conda environment
myenv but you wish to make
it visible to Jupyter, simply activate your environment:
conda activate myenv
conda install ipykernel -y -q python -m ipykernel install --user --name myenv
Now if you open a new Jupyter session from OnDemand, you will notice that the
new environment is available under the Kernel -> Change Kernel menu, with the
name we provided with the
--name option. Similarly, it will be available as
an option under the File -> New notebook menu in addition to the default
Python 3 environment.
Now if you open a new notebook, you can now use the
emoji_env we created.
Note that the currently active environment is displayed in the top right
Using this method, if you have notebooks requiring packages in addition to the defaults, you can now maintain different environments to support use of those notebooks.
Installing the R kernel¶
do not use
install.packages inside an R kernel
Due to the way the R kernel works within Anaconda, attempting to use the
install.packages method inside an R kernel will install R packages
outside of the Anaconda environment, which may cause unforeseen conflicts
when running the native R application, via a module.
To install the R kernel, run the following commands as a one-off setup:
qlogin -l h_vmem=10G module load anaconda3 conda create -n R -y -q conda activate R conda install r-recommended r-irkernel -y R -e 'IRkernel::installspec()'
The R kernel will now be available as an option within Jupyter Notebook when
starting a new notebook, or under the
Kernel -> Change kernel menu:
To uninstall the R kernel, simply run
conda env remove -n R (or delete the
environment install directory) and also delete the kernel specification
rm -r ~/.local/share/jupyter/kernels/ir/.