Since October 2016 we are using a set of tools called EasyBuild and Lmod to install and manage software packages and their accompanying modules.

This results in a different naming of the modules themselves and also a completely different module layout, compared to how it was done with Tcl modules.

The modules are installed hierarchically in such a way that not all modules are visible initially.

The EasyBuild Hierarchial Module Naming System (HMNS for short) ensures that the correct libraries are available for a specific application while keeping the output of module avail to a minimum. See the section on using Lmod further down on this page, for more information.

In short, modules are

  • used to setup your environment (paths to executables, libraries, etc.) for using a particular (set of) software package(s)
  • a tool to help users manage their Unix/Linux shell environment, allowing groups of related environment-variable settings to be made or removed dynamically
  • a way to allow having multiple versions of a program or package available by just loading the proper module
  • installed in a hierarchial layout. This means that some modules are only available after loading a specific compiler and/or MPI version.

Important

Take care not to use the system installed version of e.g. gcc! Load the suitable module for the compilers etc. that you need!

Note

On this page, a dollar-sign ($) prefacing a line is sometimes used to illustrate the bash-shell prompt on the terminal. If you are using a shell from the c-shell family, the symbol would instead be a percentage-symbol (%).

Do NOT copy the prompt if you are copying a command to use it.

Verify that your environment works

Before trying anything else please check if your environment is correctly setup to work on our systems. Version number changes as we make updates.

Check the output of this command:

module --version 2>&1 | head -n 2

On Kebnekaise it should show something like:

Modules based on Lua: Version 8.6.2  2021-12-21 19:39 -06:00

if so, then your enviroment is working. If not, send a mail to support@hpc2n.umu.se asking for help. Please make sure to state your username.

Compilers and compiler toolchains

The module hierarchy is based on compiler toolchains.

A compiler toolchain is a bundling of software that makes up a complete environment for compiling software (or using a specific prebuilt software), usually including a compiler suite, an MPI version, BLAS, LAPACK, ScaLapack, and FFTW versions.

It may also be just a compiler, like GCC alone.

An example of a compiler toolchain is foss, which is based on the GCC compiler.

Example

$ module add foss/2019b
$ module list

Currently Loaded Modules:
  1) snicenvironment (S)   8) XZ/5.2.4           15) OpenMPI/3.1.4
  2) systemdefault   (S)   9) libxml2/2.9.9      16) OpenBLAS/0.3.7
  3) GCCcore/8.3.0        10) libpciaccess/0.14  17) FFTW/3.3.8
  4) zlib/1.2.11          11) hwloc/1.11.12      18) ScaLAPACK/2.0.2
  5) binutils/2.32        12) libevent/2.1.11    19) foss/2019b
  6) GCC/8.3.0            13) PMIx/3.1.4
  7) numactl/2.0.12       14) UCX/1.6.1

  Where:
   S:  Module is Sticky, requires --force to unload or purge

There are multiple toolchains, each one based on one of the compiler suites, GCC, icc/ifort, or others. You can always find an updated list with module avail

Available toolchains include:

  • GCC: GCC only
  • foss: GCC, OpenMPI, OpenBLAS/LAPACK, FFTW, ScaLAPACK
  • fosscuda: gompic, OpenBLAS/LAPACK, FFTW, ScaLAPACK
  • gcccuda: GCC and CUDA
  • gomkl: GCC, OpenMPI, IntelMKL
  • gompi: GCC, OpenMPI
  • gompic: GCC, OpenMPI, CUDA
  • iccifort: icc, ifort
  • iccifortcuda: icc, ifort, CUDA
  • iimpi: icc, ifort, IntelMPI
  • imkl: icc, ifort, IntelMKL
  • intel: icc, ifort, IntelMPI, IntelMKL
  • intelcuda: intel, CUDA
  • iompi: icc, ifort, OpenMPI

The main toolchains to use are foss, intel, fosscuda, intelcuda. More toolchains may be available in time.

By doing module spider TOOLCHAIN for a specific TOOLCHAIN, you will get a list of available versions.

How does the Hierarchial Module Naming System work

Initially you only see a smaller set of modules when doing module avail, these modules only depend on the libraries installed with the operating system and those modules which can be loaded without prerequisite modules.

When loading one of the compiler toolchains you get access to modules built with that specific toolchain.

What happens is that the toolchain module adds another module path which contains some modules built with that specific toolchain.

How to find a specific software

Since not every module is visible initially you can not use the module avail command to find software like GROMACS, NAMD, VASP, etc.

To find software hidden by the hierarchy you should use the module spider command.

Example

This is shortened, as it is a very long list. 

$ module spider

----------------------------------------------------------------------------------------
The following is a list of the modules and extensions currently available:
----------------------------------------------------------------------------------------
  ABySS: ABySS/2.2.5
    Assembly By Short Sequences - a de novo, parallel, paired-end sequence assembler

  ACTC: ACTC/1.1
    ACTC converts independent triangles into triangle strips or fans.

  ADGofTest: ADGofTest/0.3 (E)

  AICcmodavg: AICcmodavg/2.3-0 (E), AICcmodavg/2.3-1 (E)

  ALDEx2: ALDEx2/1.20.0 (E), ALDEx2/1.26.0 (E)

  ALL: ALL/1.30.0 (E), ALL/1.36.0 (E)

  AMGX: AMGX/2.2.0-CUDA-11.3.1
    Distributed multigrid linear solver library on GPU

...

Setting up the environment using Lmod

To set up your environment for using a particular (set of) software package(s), you can use the modules that are provided centrally.

Interacting with the modules is done via Lmod, using the module command or the handy shortcut command ml.

Module command cheatsheet

  • ml lists the currently loaded modules, and is equivalent with module list
  • ml spider SOFTWARE searches (case-insensitive) for SOFTWARE in the whole module hierarchy (same as module spider SOFTWARE)
  • ml spider SOFTWARE/VERSION shows information about the module SOFTWARE/VERSION including prerequisite modules
  • ml SOFTWARE/VERSION loads the SOFTWARE/VERSION module, and is equivalent with module load SOFTWARE/VERSION
  • ml -SOFTWARE unloads the currently loaded SOFTWARE module, and is equivalent with module unload SOFTWARE
  • ml av SOFTWARE prints the currently available modules that match SOFTWARE (case-insensitively), and is equivalent with module avail SOFTWARE
  • ml show SOFTWARE/VERSION prints more information about the SOFTWARE/VERSION module (shows how loading it will change the environment), and is equivalent with module show SOFTWARE/VERSION
  • ** ml save mycollection** stores the currently loaded modules to a collection named “mycollection”
  • ml restore mycollection restores a previously stored collection of modules
  • ml key search-term lists packages containing the “search-term” in its description
  • ml purge Unload all modules except the “sticky” modules (snicenvironment and systemdefault modules which defines some important environment variables). Equivalent to module purge

Examples

Checking available versions of fosscuda 

$ module spider fosscuda

----------------------------------------------------------------------------
fosscuda:
----------------------------------------------------------------------------
  Description:
    GCC based compiler toolchain __with CUDA support__, and including OpenMPI for MPI support, OpenBLAS (BLAS and LAPACK support), FFTW and ScaLAPACK.

  Versions:
     fosscuda/2019a
     fosscuda/2019b
     fosscuda/2020a
     fosscuda/2020b

----------------------------------------------------------------------------
  For detailed information about a specific "fosscuda" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider fosscuda/2020b
----------------------------------------------------------------------------

Checking how fosscuda/2019b is loaded 

$ module spider fosscuda/2019b

----------------------------------------------------------------------------
  fosscuda: fosscuda/2019b
----------------------------------------------------------------------------
    Description:
      GCC based compiler toolchain __with CUDA support__, and including OpenMPI for MPI support, OpenBLAS (BLAS and LAPACK support), FFTW and ScaLAPACK.


    This module can be loaded directly: module load fosscuda/2019b

    Help:
      Description
      ===========
      GCC based compiler toolchain __with CUDA support__, and including OpenMPI for MPI support, OpenBLAS (BLAS and LAPACK support), FFTW and ScaLAPACK.


      More information
      ================
       - Homepage: (none)

Loading the module “fosscuda” version 2019b 

$ module load fosscuda/2019b
$

Listing the loaded modules after loading fosscuda/2019b 

$ module list

Currently Loaded Modules:
  1) snicenvironment (S)   8) numactl/2.0.12     15) UCX/1.6.1-CUDA-10.1.243
  2) systemdefault   (S)   9) XZ/5.2.4           16) OpenMPI/3.1.4
  3) GCCcore/8.3.0        10) libxml2/2.9.9      17) OpenBLAS/0.3.7
  4) zlib/1.2.11          11) libpciaccess/0.14  18) FFTW/3.3.8
  5) binutils/2.32        12) hwloc/1.11.12      19) ScaLAPACK/2.0.2
  6) GCC/8.3.0            13) libevent/2.1.11    20) fosscuda/2019b
  7) CUDA/10.1.243        14) PMIx/3.1.4

  Where:
   S:  Module is Sticky, requires --force to unload or purge

Note that even though we only loaded a single module, the output of ml shows that a whole bunch of modules were loaded, which are required dependencies for fosscuda.

Checking how to load Python/3.9.5 

$ module spider Python/3.9.5

----------------------------------------------------------------------------------------------------------------------------------------------------------
  Python: Python/3.9.5
----------------------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      Python is a programming language that lets you work more quickly and integrate your systems more effectively.


    You will need to load all module(s) on any one of the lines below before the "Python/3.9.5" module is available to load.

      GCCcore/10.3.0

    This module provides the following extensions:

       alabaster/0.7.12 (E), appdirs/1.4.4 (E), asn1crypto/1.4.0 (E), atomicwrites/1.4.0 (E), attrs/21.2.0 (E), 
       Babel/2.9.1 (E), bcrypt/3.2.0 (E), bitstring/3.1.7 (E), blist/1.3.6 (E), CacheControl/0.12.6 (E), 
       cachy/0.3.0 (E), certifi/2020.12.5 (E), cffi/1.14.5 (E), chardet/4.0.0 (E), cleo/0.8.1 (E), click/7.1.2 (E), 
       clikit/0.6.2 (E), colorama/0.4.4 (E), crashtest/0.3.1 (E), cryptography/3.4.7 (E), Cython/0.29.23 (E), 
       decorator/5.0.7 (E), distlib/0.3.1 (E), docopt/0.6.2 (E), docutils/0.17.1 (E), ecdsa/0.16.1 (E), 
       filelock/3.0.12 (E), flit-core/3.2.0 (E), flit/3.2.0 (E), fsspec/2021.4.0 (E), future/0.18.2 (E), 
       html5lib/1.1 (E), idna/2.10 (E), imagesize/1.2.0 (E), importlib_metadata/4.0.1 (E), iniconfig/1.1.1 (E), 
       intervaltree/3.1.0 (E), intreehooks/1.0 (E), ipaddress/1.0.23 (E), jeepney/0.6.0 (E), Jinja2/2.11.3 (E), 
       joblib/1.0.1 (E), jsonschema/3.2.0 (E), keyring/21.8.0 (E), keyrings.alt/4.0.2 (E), liac-arff/2.5.0 (E), 
       lockfile/0.12.2 (E), MarkupSafe/1.1.1 (E), mock/4.0.3 (E), more-itertools/8.7.0 (E), msgpack/1.0.2 (E), 
       netaddr/0.8.0 (E), netifaces/0.10.9 (E), nose/1.3.7 (E), packaging/20.9 (E), paramiko/2.7.2 (E), 
       pastel/0.2.1 (E), pathlib2/2.3.5 (E), paycheck/1.0.2 (E), pbr/5.6.0 (E), pexpect/4.8.0 (E), pip/21.1.1 (E), 
       pkginfo/1.7.0 (E), pluggy/0.13.1 (E), poetry-core/1.0.3 (E), poetry/1.1.6 (E), psutil/5.8.0 (E), 
       ptyprocess/0.7.0 (E), py/1.10.0 (E), py_expression_eval/0.3.13 (E), pyasn1/0.4.8 (E), pycparser/2.20 (E), 
       pycrypto/2.6.1 (E), Pygments/2.9.0 (E), pylev/1.3.0 (E), PyNaCl/1.4.0 (E), pyparsing/2.4.7 (E), 
       pyrsistent/0.17.3 (E), pytest/6.2.4 (E), python-dateutil/2.8.1 (E), pytoml/0.1.21 (E), pytz/2021.1 (E), 
       regex/2021.4.4 (E), requests-toolbelt/0.9.1 (E), requests/2.25.1 (E), scandir/1.10.0 (E), SecretStorage/3.3.1 (E), 

...

Loading the module Python/3.9.5 and its prerequisite module(s) 

$ ml GCCcore/10.3.0 Python/3.9.5
$

Showing the available module versions for GROMACS (2024-04-24) 

$ ml spider gromacs

----------------------------------------------------------------------------------------------------------------------------------------------------------
  GROMACS:
----------------------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      GROMACS is a versatile package to perform molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles.

     Versions:
        GROMACS/3.3.3-cphmd-1.3
        GROMACS/3.3.3-cphmd-1.3-1lpernode
        GROMACS/3.3.3-cphmd-1.4
        GROMACS/4.0.7-ionstr
        GROMACS/5.1.4
        GROMACS/2016.x-drude-20180214-g3f7439a
        GROMACS/2016.x-drude-20220117-g78fe3d1e
        GROMACS/2016.x-drude-20220120-ge35ae4e2
        GROMACS/2016.4
        GROMACS/2018.8
        GROMACS/2019
        GROMACS/2019.4-PLUMED-2.5.4
        GROMACS/2021
        GROMACS/2022.4

----------------------------------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "GROMACS" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider GROMACS/2022.4
----------------------------------------------------------------------------------------------------------------------------------------------------------

Reminder

  • Using ml spider on a software name gives you a list of versions of that software that is installed (if any)

  • If you use ml spider on a full module name (in this case spider is case sensitive) like GROMACS/2019 it will tell you what you need to load to access it.

  • Using just ml spider without arguments, gives you a complete list of modules in the hierarchy.

  • Listing all modules available using the currently loaded toolchain (if any): module avail (or ml av)

  • Sometimes even though we only load a single module, the output of module list shows that a whole bunch of modules were loaded. These are required dependencies.

Advanced information about modules

In the current module naming scheme, each module name consists of two parts:

  • the part before the first /, corresponding to the software name; and
  • the remainder, corresponding to the software version and a possible version suffix

For example, the module name iccifort/2017.0.098-GCC-5.4.0-2.26 would set up the environment for using icc and ifort version 2017.0.098, with the GCC/5.4.0-2.26 compiler toolchain.

The (D) indicates that this particular version of the module is the default, but we strongly recommend to not rely on this as the default can change at any point. Usually, the default will point to the latest version available.

Note

  • Available versions for a particular software package under the currently loaded toolchain (if any): module avail MODULENAME (or ml av MODULENAME)

  • To check which versions of a particular software package that are available, you can provide the software name to ml av.

  • For example, to check which versions of gompi are available: ml av gompi

  • Note that the specified software name is treated case-insensitively.

  • Lmod does a partial match on the module name, so sometimes you need to use / to indicate the end of the software name you are interested in: ml av gcc/

Conflicting modules

It is important to note that only modules that are compatible with each other can be loaded together. In particular, modules must be installed either with the same toolchain as the modules that are already loaded, or with a compatible (sub)toolchain.

For example, once you have loaded one or more modules that were installed with the foss/2019b toolchain, all other modules that you load should have been installed with the same toolchain. Due to the nature of the hierarchial module naming system, only modules built with that same toolchain are actually visible, thus reducing the probablility of making a mistake.

In addition, only one single version of each software package can be loaded at a particular time. For example, once you have loaded the module Python/3.9.5, you can not load a different version of Python in the same session/job script; neither directly, nor indirectly as a dependency of another module you want to load.

Module collections

Note

Key commands here are: ml save, ml restore

If you have a set of modules that you need to load often, you can save these in a collection.

Example

  • Load all the modules you need, for example:

$ ml GCC/11.2.0  OpenMPI/4.1.1 SciPy-bundle/2021.10 matplotlib/3.4.3

  • Now store them in a collection using ml save:

$ ml save my-collection
Saved current collection of modules to: my-collection

  • Later, for example in a job script, you can reload all these modules with ml restore:

$ ml restore my-collection
Restoring modules to user's my-collection

  • With ml savelist you can get a list of all saved collections:

$ ml savelist
Named collection list :
 1) my-collection

  • To inspect a collection, use ml describe.

  • To remove a module collection, remove the corresponding entry in $HOME/.lmod.d.

Lmod vs Tcl-based environment modules

In October 2016, we switched to Lmod as a modules tool, a modern alternative to the outdated & no longer actively maintained Tcl-based environment modules tool.

Consult the Lmod documentation web site for more information.

Benefits

  • significantly more responsive module commands, in particular module avail
  • a better and easier to use interface (e.g. case-insensitive avail, the ml command, etc.)
  • additional useful features, like defining & restoring module collections
  • drop-in replacement for Tcl-based environment modules (existing Tcl module files do not need to be modified to work)
  • module files can be written in either Tcl or Lua syntax (and both types of modules can be mixed together)

Key differences

The switch to Lmod (or more specifically the use of EasyBuild for building and installing software) means that we have a completely different module naming system.

There are a couple of minor differences between Lmod & the old modules tool you should be aware of, if you are used to Tcl-based modules:

  • module conflicts are strictly enforced, meaning conflicting modules cannot be loaded together
  • module purge does not unload the snicenvironment and systemdefault modules

Using module purge effectively resets your environment to a pristine working state, i.e. the snicenvironment module stays loaded after the purge.

(This text is in large parts copied from https://www.vscentrum.be/cluster-doc/software/modules-lmod with their consent)