Python

The Python programming language is provided on all nodes of the Lovelace. Multiple versions of the Python Interpreter are provided. In fact, for a specific version of Python (e.g. Python 3.11), multiple versions of the same Interpreter are provided. This document details how users should load or install packages on the cluster.

Setting up a Python Environment with Modules

A selection of Python Packages are available through the modules framework described in ../user-guide/modules. To see a selection of the available Python packages, run:

module avail

Many Python packages are hidden as they are usually not directly loaded by users but rather as dependencies of other Python packages. It is possible show these packages but this view is not reccomended for users due to the length of the output. To see all available packages, run the command below.

module --show_hidden avail

Modules for Python packages are named with the prefix py-. Such modules support a specific version of Python. You can tell which packages a module supports by looking at its suffix. If there is no suffix, the module supports the recommended version of Python selected by the HPC Admin team, currently Python 3.11. If there is a suffix, such as -py3.13, the module supports only the version indicated in the suffix. Some examples of the naming convention are given below:

• py-numpy provides the NumPy Python Package for the recommended version of Python (currently Python 3.11)

• py-pip provides the Pip Python Package for the recommended version of Python (currently Python 3.11)

• py-pip-py3.13 provides the Pip Python Package for Python 3.13

To load only the recommended version of python itself, simply run:

module load python

Using Packages Provided in Modules with The Recommended Python

Note the modules you would like to use. For instance, say that we would like to use Numpy and Scipy. We see that py-numpy and py-scipy are available from the output of the command above. Also, we additionally assume that we would like to use the recommended version of Python, 3.11. In this case, it is sufficient to load these modules as follows:

module load py-numpy py-scipy

Then we can use numpy within python. For example:

[apraveen@uop-hpc ~]$ module load py-numpy py-scipy
[apraveen@uop-hpc ~]$ python3
Python 3.11.9 (main, Feb  3 2025, 14:12:39) [GCC 13.2.1 20231205 (Red Hat 13.2.1-6)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.eye(3)
array([[1., 0., 0.],
       [0., 1., 0.],
       [0., 0., 1.]])

Note that Python 3.11 and many other modules are implicitly loaded as dependencies. To see the loaded modules, you may use the command below:

module list

Using Both Packages Provided in Modules and Those Available Externally with The Recommended Python

However, if we want to use Numpy, Scipy and a module not currently provided on Lovelace such as Pygame, additionally load the virtualenv and pip and install Pygame within a virtual environment.

[apraveen@uop-hpc ~]$ module load py-numpy py-scipy py-virtualenv
[apraveen@uop-hpc ~]$ virtualenv pygame
created virtual environment CPython3.11.9.final.0-64 in 297ms
  creator CPython3Posix(dest=/users/apraveen/pygame, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/users/apraveen/.local/share/virtualenv)
    added seed packages: pip==24.3.1, setuptools==75.7.0, wheel==0.45.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
[apraveen@uop-hpc ~]$ . pygame/bin/activate
(pygame) [apraveen@uop-hpc ~]$ pip install pygame
Collecting pygame
  Downloading pygame-2.6.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (12 kB)
Downloading pygame-2.6.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.0 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.0/14.0 MB 126.8 MB/s eta 0:00:00
Installing collected packages: pygame
Successfully installed pygame-2.6.1

[notice] A new release of pip is available: 24.3.1 -> 25.0
[notice] To update, run: pip install --upgrade pip
(pygame) [apraveen@uop-hpc ~]$ python3
Python 3.11.9 (main, Feb  3 2025, 14:12:39) [GCC 13.2.1 20231205 (Red Hat 13.2.1-6)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pygame
>>> ...

Using Packages Available Externally with a Python other than the Recommended Version

Few modules are provided for Python versions other than the reccomnded one. As such, packages should generally be installed through Pip when using such a Python version. For instance, to use Numpy with Python 3.13:

[apraveen@uop-hpc ~]$ module load py-virtualenv-py3.13
[apraveen@uop-hpc ~]$ virtualenv numpy
created virtual environment CPython3.13.0.final.0-64 in 565ms
  creator CPython3Posix(dest=/users/apraveen/numpy, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, via=copy, app_data_dir=/users/apraveen/.local/share/virtualenv)
    added seed packages: pip==24.2
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
[apraveen@uop-hpc ~]$ . numpy/bin/activate
(numpy) [apraveen@uop-hpc ~]$ pip install numpy
Collecting numpy
   Downloading numpy-2.2.2-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (62 kB)
Downloading numpy-2.2.2-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.1/16.1 MB 128.8 MB/s eta 0:00:00
Installing collected packages: numpy
Successfully installed numpy-2.2.2

[notice] A new release of pip is available: 24.2 -> 25.0
[notice] To update, run: pip install --upgrade pip
(numpy) [apraveen@uop-hpc ~]$ python3
Python 3.13.0 (main, Feb  3 2025, 14:13:03) [GCC 13.2.1 20231205 (Red Hat 13.2.1-6)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.eye(3)
array([[1., 0., 0.],
       [0., 1., 0.],
       [0., 0., 1.]])

Setting up a Python Environment without Modules

It is also possible to set up a python environment without modules. The following versions of Python are provided outsite of modules:

• Python 3.12

• Python 3.11

• Python 3.9 (not reccomended)

• Python 3.6 (not recommended)

Users can also create Virtual Environments with these Python Versions. For example to set up SciPy in a Virtual Environment without using Modules with Python 3.12:

[apraveen@uop-hpc ~]$ python3.12 -m venv scipy
[apraveen@uop-hpc ~]$ . scipy/bin/activate
(scipy) [apraveen@uop-hpc ~]$ pip install scipy
Collecting scipy
  Obtaining dependency information for scipy from https://files.pythonhosted.org/packages/b0/3c/0de11ca154e24a57b579fb648151d901326d3102115bc4f9a7a86526ce54/scipy-1.15.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading scipy-1.15.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.0/62.0 kB 3.8 MB/s eta 0:00:00
Collecting numpy<2.5,>=1.23.5 (from scipy)
   Obtaining dependency information for numpy<2.5,>=1.23.5 from https://files.pythonhosted.org/packages/5b/73/65d2f0b698df1731e851e3295eb29a5ab8aa06f763f7e4188647a809578d/numpy-2.2.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading numpy-2.2.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (62 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.0/62.0 kB 11.3 MB/s eta 0:00:00
Downloading scipy-1.15.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (40.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.2/40.2 MB 121.8 MB/s eta 0:00:00
Downloading numpy-2.2.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.1/16.1 MB 157.2 MB/s eta 0:00:00
Installing collected packages: numpy, scipy
Successfully installed numpy-2.2.2 scipy-1.15.1

[notice] A new release of pip is available: 23.2.1 -> 25.0
[notice] To update, run: pip install --upgrade pip
(scipy) [apraveen@uop-hpc ~]$ python3
Python 3.12.1 (main, Feb 21 2024, 10:25:11) [GCC 8.5.0 20210514 (Red Hat 8.5.0-21)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import scipy.sparse
>>> scipy.sparse.eye(3)
<DIAgonal sparse matrix of dtype 'float64'
        with 3 stored elements (1 diagonals) and shape (3, 3)>

Conda

A conda module is also provided. To use it, simply load the module:

module load conda