numba¶

Numba is an Open Source NumPy-aware optimizing compiler for Python sponsored by Continuum Analytics, Inc. It uses the remarkable LLVM compiler infrastructure to compile Python syntax to machine code.

Policy¶

numba is freely available to users at HPC2N.

Overview¶

Numba translates Python functions to optimized machine code at runtime using the industry-standard LLVM compiler library. Numba-compiled numerical algorithms in Python can approach the speeds of C or FORTRAN.

You don’t need to replace the Python interpreter, run a separate compilation step, or even have a C/C++ compiler installed. Just apply one of the Numba decorators to your Python function, and Numba does the rest.

Numba is designed to be used with NumPy arrays and functions. Numba generates specialized code for different array data types and layouts to optimize performance. Special decorators can create universal functions that broadcast over NumPy arrays just like NumPy functions do.

Numba also works great with Jupyter notebooks for interactive computing, and with distributed execution frameworks, like Dask and Spark.

Numba offers a range of options for parallelizing your code for CPUs and GPUs, often with only minor code changes.

Usage at HPC2N¶

On HPC2N we have numba available as a module.

Loading¶

To use the numba module, add it to your environment. You can find versions with

module spider numba

and you can then find how to load a specific version (including prerequisites), with

module spider numba/<version> 

Running¶

In order to use numba you need to have a Python script that uses numba, and then a batch script to submit the job to the batch system or run it through an interactive job on GPUs.

Batch example¶

This is an example batch script. You can test it with this Python example: add-list.py.

#!/bin/bash
# Remember to change this to your own project ID! 
#SBATCH -A hpc2nXXXX-YYY     # HPC2N ID - change to your own
# We are asking for 5 minutes - change to what you need 
#SBATCH --time=00:05:00
# Asking for one L40s GPU - change if you want a different GPU
#SBATCH --gpus=1
#SBATCH -C l40s

# Remove any loaded modules and load the ones we need, here for numba 0.58.1 
# In this example also the SciPy-bundle is loaded (numpy, scipy, etc.) 
module purge  > /dev/null 2>&1
module load GCC/12.3.0 Python/3.11.3 OpenMPI/4.1.5 SciPy-bundle/2023.07 CUDA/12.1.1
module load numba/0.58.1 

# Run your Python script
python add-list.py

Interactive example¶

To run the same example Python script interactively, you could start an interactive job with

salloc -A hpc2nXXXX-YYY --time=00:30:00 -n 1 --gpus=1 -C l40s

In this example, you need to change the project id to your own, change the time (here 30 min) and also how many GPU cards and which types. Here one L40s card.

You can read more about which types of GPUs you can pick and how many cards they have in the section about selecting GPUs.

1. To run the same example Python script add-list.py interactively, you could start an interactive job with

   salloc -A hpc2nXXXX-YYY --time=00:30:00 -n 1 --gpus=1 -C l40s
   

2. The load the needed modules, here for numba 0.58.1 and also SciPy-bundle since the example needs numpy.

   module load GCC/12.3.0 Python/3.11.3 OpenMPI/4.1.5 SciPy-bundle/2023.07
   module load CUDA/12.1.1
   module load numba/0.58.1
   

3. Now run the example. NOTE that since the interactiveness at HPC2N is not “true” interactive, you are still located on the login node and needs to preface the executable with srun to actually run it on the allocated compute node(s):

   srun python add-list.py
   

Additional info¶

More information can be found on

• The official website for numba
• The official Numba documentation