Libraries¶

Build environment

Using the MPI and MATH libraries that are available through a compiler toolchain by itself is possible but requires a fair bit of manual work, figuring out which paths to add to -I or -L for include files and libraries, and similar.

To make life as a software builder easier there is a special module available, buildenv, that can be loaded on top of any toolchain. If it is missing for some toolchain, send a mail to support@hpc2n.umu.se and let us know.

This module defines a large number of environment variables with the relevant settings for the used toolchain. Among other things it sets CC, CXX, F90, FC, MPICC, MPICXX, MPIF90, CFLAGS, FFLAGS, and much more.

To see all of them, load a toolchain and do

ml show buildenv

There is some more information about buildenv in the Build environment section under compilers.

Linking with FlexiBLAS

FlexiBLAS - A BLAS and LAPACK wrapper library with runtime exchangable backends.

You can load it as a module after first loading a suitable GCC module (ml spider FlexiBLAS for more information). It is also available as part of several compiler toolchain versions. Remember, you can always see which other modules are included in a toolchain with ml show TOOLCHAIN/VERSION.

Or, after loading the buildenv module, use the environment variable to link with: $LIBBLAS.

Linking with OpenBLAS

OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version.

Load it either as a module after first loading a suitable GCC module (ml spider OpenBLAS for more information), or as part of several compiler toolchain versions. Remember, you can always see which other modules are included in a toolchain with ml show TOOLCHAIN/VERSION.

Or, after loading the buildenv module, use the environment variable to link with: $LIBBLAS.

Using Eigen

First load the “Eigen” module. To see which versions of Eigen are available use ml spider eigen. Remember to also load the needed prerequisites for the version (listed when you do ml spider Eigen/VERSION for the version you want).

You can find the Eigen library files under the $EBROOTEIGEN/lib directory after the module has been loaded.

There is a getting started guide and other documentation on the Eigen homepage.

Linking with FFTW

To use FFTW, you should load it as part of a compiler toolchain. The available modules are foss and fosscuda. Do ml av to see which versions you can load.

Use these commands to compile and link with FFTW3

Or use $LIBFFT -lm to link with ($LIBFFT_MT -lm for threaded). This requires you to load the buildenv module after loading the compiler toolchain.

Linking with FlexiBLAS

FlexiBLAS is available as part of several compiler toolchains. Remember, you can always see which other modules are included in a toolchain with ml show TOOLCHAIN/VERSION.

You can also load it as a module after first loading a suitable GCC module (ml spider FlexiBLAS for more information)

Or, after loading the buildenv module, use the environment variable to link with: $LIBBLAS.

Using GSL

First load the module. To see which versions of GSL are available use ml spider GSL. Then do ml spider GSL/VERSION for the VERSION you would like to load, in order to see the prerequisites that are needed to load the GSL module.

The GSL libraries can be found in $EBROOTGSL/lib after the module has been loaded, if you need to update yourself on their names.

After loading, you can get some information about GSL from the command man gsl.

Linking with MKL libraries

To use the MKL libraries, first load one of the following compiler toolchain modules:

• gomkl: GCC, OpenMPI, IntelMKL
• imkl: icc, ifort, IntelMKL
• intel: icc, ifort, IntelMPI, IntelMKL
• intelcuda: intel, CUDA

in a suitable version (check with ml spider for the relevant compiler toolchain).

To correctly use MKL it is vital to have read the documentation. To find the correct way of linking, take a look at the offical Intel MKL documentation.

Using the buildenv module, the common blas/lapack/scalapack/fftw libraries are available in the following environment variables, just like when using a non-MKL capable toolchain:

• LIBBLAS
• LIBLAPACK
• LIBSCALAPACK
• LIBFFT

And threaded versions are available from the corresponding environment variable appended with “_MT”

Read the section about buildenv for more information.

There are too many libraries in MKL to show a complete list of combinations. We refer you to the official MKL documentation for examples and support@hpc2n.umu.se for help.

Linking with LAPACK (and BLAS)

The Fortran based lapack library is included with the BLAS modules. To use it you must load the BLAS module you want, as well as its prerequisite compiler (toolchain).

Load a suitable version of a toolchain containing LAPACK/BLAS. See compiler toolchains for this.

Use the following commands to compile and link.

OpenBLAS

FlexiBLAS

Or use the environment variable $LIBLAPACK to link with. This requires you to load the buildenv module after loading the compiler toolchain.

Using Libint

To see which versions of Libint are available, and how to load it and any dependencies, use ml spider libint and then ml spider libint/VERSION for the specfic VERSION you are interested in.

When the module and its prerequisites have been loaded, you can use the environment variable $EBROOTLIBINT to find the binaries and libraries for Libint.

There is some information about Libint and how to use it on the Libint Homepage. There is a brief Libint Programmers Manual here.

Using Libxc

To see which versions of Libxc are available, and how to load it and any dependencies, use ml spider libxc and then use ml spider to check a specific version to see how to load it and the prerequisites.

When the module has been loaded, you can use the environment variable $EBROOTLIBXC to find the binaries and libraries for Libxc.

There is a Libxc manual here.

Using Libxsmm

To see which versions of Libxsmm are available, and how to load it and any dependencies, use ml spider libxsmm and then use ml spider libxsmm/VERSION to check a specific VERSION to see how to load it.

When the module has been loaded, you can use the environment variable $EBROOTLIBXSMM to find the binaries and libraries for Libxsmm.

There is some Libxsmm documentation here.

Using MAGMA

To access MAGMA, you first need to load the module and its prerequisites. Do ml spider magma to see the available modules and then do ml spider magma/VERSION for the specific version you are interested in to see which prerequisites you need to load first.

When you have loaded a MAGMA module, you can see the available libraries by looking where the environment variable $EBROOTMAGMA points.

There are some examples of using magma here:

• MAGMA: MAtrix Algebra on GPU and Multicore Architecture (PDF)
• MAGMA User’s guide

Linking with METIS

First load the METIS module. Do ml spider METIS to see versions. Remember to load the prerequisite compiler suite or toolchain!

Use these commands to compile and link with METIS

Using MPFR

To see which versions of MPFR are available, and how to load it and any dependencies, use ml spider mpfr and then do ml spider on a specific version to see how to load it.

When the module has been loaded, you can use the environment variable $EBROOTMPFR to find the binaries and libraries for MPFR.

The MPFR Reference Guide is here (for the newest version).

Using OpenBLAS

Load it either as a module after first loading a suitable GCC module (ml spider OpenBLAS for more information), or as part of several compiler toolchain versions. Remember, you can always see which other modules are included in a toolchain with ml show TOOLCHAIN/VERSION.

Or, after loading the buildenv module, use the environment variable to link with: $LIBBLAS.

Using ParMETIS

To see which versions of ParMETIS are available, and how to load it and any dependencies, use ml spider parmetis and then use ml spider on a specific version to see how to load it.

When the module has been loaded, you can use the environment variable $EBROOTPARMETIS to find the binaries and libraries for ParMETIS.

Linking with ScaLAPACK, BLAS, and LAPACK

You can load one of either the foss or the fosscuda toolchains. See the section compiler toolchains for this. .

In addition, you can use Intel MKL if you are using the Intel compilers (though there is the gomkl toolchain with GCC, OpenMPI, and Intel MKL).

After loading the suitable compiler toolchain module, use the following command to compile and link with ScaLAPACK

Toolchain versions with OpenBLAS

Toolchain versions with FlexiBLAS

Or use the environment variable, $LIBSCALAPACK to link with. This requires you to load the buildenv module after loading the compiler toolchain.

Using SCOTCH

To see which versions of SCOTCH are available, and how to load it and its dependencies, use ml spider scotch and then ml spider SCOTCH/VERSION for a specific VERSION to see how it is loaded (which prerequisites needs to be loaded first).

When the module has been loaded, you can use the environment variable $EBROOTSCOTCH to find the binaries and libraries for SCOTCH.

The text files for the user manual on how to use SCOTCH can be found in $EBROOTSCOTCH/man/man1/ when the module is loaded. You can also copy the content of that directory and then run “make” to generate a full user manual.

Note

If you do not find what you are looking for here, also take a look at the Python modules section.

Using cuTENSOR

You first need to load the module. To see the versions available, do ml spider cuTENSOR. The module can then be loaded directly.

You can see the libraries from $EBROOTCUTENSOR/lib/<VERSION>.

There is a user guide at the offical cuTENSOR page.

Using Horovod

You first need to load a Horovod module in order to use it. You can see the available versions with the command ml spider Horovod. Then you do ml spider module spider Horovod/<VERSION> to see which prerequisites to load before loading the Horovod module.

When the module is loaded, you can see the binaries and libraries from $EBROOTHOROVOD.

You do ´´horovodrun –help`` for some options.

Note

If you do not find what you are looking for here, also take a look at the Python modules section.

Using GeoPandas

First you need to load the GeoPandas module. To see which versions exists, do ml spider geopandas and then ml spider geopandas/<VERSION> to see which prerequisites needs to be loaded first.

You can use $EBROOTGEOPANDAS/lib to see the available libraries.

Using HDF5

You need to first load the HDF5 module. To see the available versions, do ml spider HDF5 and then ml spider HDF5/<VERSION> to see how to load the specific version of the module (which prerequisites to load first).

When the module is loaded, you can use $EBROOTHDF5 to find the binaries and libraries available.

Using netCDF

To see which versions of NetCDF are available, and how to load it and any dependencies, use ml spider netcdf and then do ml spider on a specific version to see how to load it.

When the module has been loaded, you can use the environment variable $EBROOTNETCDF to find the binaries and libraries for NetCDF.

There is some information about NetCDF and how to use it on the NetCDF documentation page.

There is also a Parallel netCDF available, for parallel I/O access. The module is called PnetCDF.

Using CUDA

CUDA can be loaded on its own, and it is also part of some compiler toolchains

After you have loaded CUDA (on its own or as part of a compiler toolchain) module, you compile and link with CUDA like this:

You can add other flags, like for instance -o my-binary to name the output differently than the standard a.out.

NOTE: CUDA functions can be called directly from Fortran programs:

1. First use the nvcc compiler to create an object file from the .cu file.
2. Then compile the Fortran code together with the object file from the .cu file.

Using numactl

You first need to load the module. Use ml spider numactl to see available versions, then use ml spider numactl/<VERSION> to see how to load a specific version (what prerequisited to load first).

When the module is loaded, you can find the binaries and libraries from $EBROOTNUMACTL.

With a numactl module loaded, you can also get help with

• man numactl
• numactl --help or just numactl with no options

Using rgdal

You need to load the rgdal module before using it. First do ml spider rgdal to see which versions exist, then do ml spider rgdal/<VERSION> for the version you are interested in, in order to see which prerequisites to load before loading the rgdal module.

When you have loaded the module, you can find docs, help, libs, etc. from $EBROOTRGDAL/rgdal

Using SIONlib

To see which versions of SIONlib are available, and how to load it and any dependencies, use ml spider sionlib and then do ml spider on a specific version to see how to load it.

When the module has been loaded, you can use the environment variable $EBROOTSIONLIB to find the binaries and libraries for SIONlib.

There is some documentation for SIONlib here.

Using StarPU

In order to use StarPU, you first need to load the module. Do ml spider starpu to see which versions are available and then do ml spider starpu/<VERSION> in order to see which prerequisites you need to load first.

When the module has been loaded you can use $EBROOTSTARPU to find binaries and libraries.

We have versions available for normal compute nodes and for GPUs. The tests and examples have not been built.

There is a manual for using StarPU here: StarPU Handbook

In the directory $EBROOTSTARPU/easybuild/ you will find the .eb file that was used to build the module. In there, under 'configopts' you will find the options the module was built with.

They can both be loaded with or without GPU capability. It is easiest to load the corresponding intel version instead of the separate icc, ifort, and impi modules.

Additional info:

• We have some information from one of our (older) courses (Task-based parallelism in scientific computing) which contains an overview and example for StarPU:
  • StarPU part 1 presentations and recording on YouTube
  • StarPU part 2 presentations and recording on YouTube
  • Example for LU factorization: presentation and recording on YouTube