Skip to main content

Environment Modules on the Peregrine System

Peregrine uses environment modules to easily manage software environments. Environment modules facilitate the use of different versions of applications, libraries, and toolchains, which enables center staff to support current and legacy versions without disturbing your use of the system.

At login, the modules commands set up a basic environment for the default compilers, tools and libraries, such as the $PATH, $MANPATH and $LD_LIBRARY_PATH environment variables. Users that require other applications or non-default versions can tailor their environment using modules.

The modules in essence simply set environment variables that one might traditionally do via, for example, adding export or setenv commands to their login script. Modules add the ability to unset variables in an orderly manner as well, so that one can change their environment in a reversible way. As they are only scripts, they can be hosted in any location that the user makes known to the environment modules system via the MODULEPATH variable (which is modified via "module use" or "module unuse" commands, see below). By copying the reference modules we set up into custom locations, you can host personal, project, or multi-project collections that you have complete control over.

The module command accepts parameters that enable users to inquire about and change the module environment. Most of the basic functionality can be accessed through the following commands.

option description
avail Prints all available modules on the system. Paths searched for modules are set in $MODULEPATH
list Prints all currently loaded modules.
display 'name' Prints settings and paths specified for a particular module.
help 'name' Prints help message for a particular module.
load 'name' Loads particular module. For modules listed as the '(default)', the short package name is sufficient. To load another version of the package the long package name is required (eg. module load fftw2/2.1.5/gcc)
unload 'name' Unloads particular module
swap 'name1' 'name2' First unload modName1 and then load modName2
use {-a} A_PATH Prefix {suffix} the path $A_PATH to your $MODULEPATH variable, in order to find modules in that location
unuse A_PATH Remove the path $A_PATH from your $MODULEPATH variable

The modulefiles that we provide are only a starting point. For maximum control, users should copy these files from the locations in /nopt to their own locations for which they have write access.

  • Module files for baseline applications, libraries, frameworks, and toolchains are located in the /nopt/nrel/apps/modules/centos7/modulefiles directory
  • Users may and should freely copy these example modulefiles to preferred locations and customize them for their own use cases. This can be particularly desirable to preserve a critical workflow as the software environment changes on Peregrine, or to change the behavior, e.g., turn off automatic loading of prerequisites. In order to add a location to be searched regularly for available modules, the module use command may be added to a login script (e.g., .bash_profile) or issued in an interactive shell or job script:
module use -a /projects/{allocation}/modules/default/modulefiles
module use -a /home/{username}/modules/default/modulefiles

The -a flag appends the path that follows to environment variable MODULEPATH; leaving it out will prepend the path. The first module found in searching $MODULEPATH is used, so the search order is important.

Since new versions of software are periodically added to the system, check current availability with the module avail command. If a module is needed often, the module load <module_name> command can be put in .bash_profile or other shell startup files.

To load a module:

$ module load <module_name>/<version>

Here <module_name> is to be replaced by the name of the module to load. It is advised to ALWAYS include the full version number in your load statements. Without it, the actual environment loaded will depend on the order of your MODULEPATH variable first, “(default)” designations second, and finally lexical ordering. This can create unintended consequences!

To get a list of available modules, type

$ module avail

It's a good idea to look at two other commands to see what a module does, and what software dependencies there are, as illustrated below.

[user@login3 04:05:26 ~]$ module show R/3.4.2
-------------------------------------------------------------------
/nopt/nrel/apps/modules/centos7/modulefiles/R/3.4.2:

module-whatis R is an open source software environment for statistical computing and graphics.
Includes base installation packages plus NREL extras.
Built with gcc 7.2.0, OpenMPI 2.1.2-7.2.0 and MKL/2017.0.5
prepend-path PATH /nopt/nrel/apps/R/3.4.2-gcc720/bin
prepend-path LD_LIBRARY_PATH /nopt/nrel/apps/R/3.4.2-gcc720/lib64/R/lib
prepend-path LD_LIBRARY_PATH /nopt/nrel/apps/R/3.4.2-gcc720/lib
prepend-path LD_LIBRARY_PATH /nopt/nrel/apps/R/3.4.2-gcc720/lib64/R/lib
prepend-path MANPATH /nopt/nrel/apps/R/3.4.2-gcc720/share/man
prepend-path INCLUDE_PATH /nopt/nrel/apps/R/3.4.2-gcc720/include
setenv R_LIBS_USER /home/cchang
setenv R_HOME /nopt/nrel/apps/R/3.4.2-gcc720
-------------------------------------------------------------------

[user@login3 04:05:37 ~]$ module help R/3.4.2

----------- Module Specific Help for 'R/3.4.2' --------------------

Name: R
Version: 3.4.2, with base + common + MPI packages
Category: data, statistics, graphics
Help: help.search('function')' or ?function
URL: http://www.r-project.org
Version 3.4.2
Built against openmpi-gcc/2.1.2-7.2.0, gcc/7.2.0, and MKL/2017.0.5
Please make sure these module dependencies are loaded before attempting to use this R

The environment variables set by the module can then be used in build scripts. It is not necessary to load a module in order to use the module display command, this may be done at any time to see what a module does.

Module files for different versions can easily be swapped:

[user@login3 04:05:42 ~]$ module load openmpi-gcc/2.1.2-7.2.0
[user@login3 04:06:52 ~]$ module list
Currently Loaded Modulefiles:
1) gcc/7.2.0 2) openmpi-gcc/2.1.2-7.2.0
[user@login3 04:06:54 ~]$ module swap openmpi-gcc/2.1.2-7.2.0 openmpi-gcc/2.1.2-4.8.5
[user@login3 04:07:09 ~]$ module list
Currently Loaded Modulefiles:
1) openmpi-gcc/2.1.2-4.8.5
mkdir -p $HOME/modules/default/modulefiles
cd $HOME/modules/default/modulefiles
mkdir comp-intel impi-intel
export TMP_PREFIX=/nopt/nrel/apps/modules/centos7/modulefiles
cp $TMP_PREFIX/comp-intel/2017.0.5 comp-intel/.
cp $TMP_PREFIX/impi-intel/2017.0.5 impi-intel/.

cp $HOME/.bash_profile $HOME/.bash_profile.bak
echo >> $HOME/.bash_profile
echo "module use $HOME/modules/default/modulefiles" >> $HOME/.bash_profile

Assuming you're using the bash shell, once you logout and log back in, you should see your new modules via module avail. At that point, you are free to rename, edit, and configure as you see fit. For example, Intel compilers rely on a background GCC compiler in the environment. By default, the system version (4.8.5) is used, but you could add the gcc/6.2.0 module to your collection, and create a comp-intel dependency on it so your build environment automatically uses the more modern GCC version.

Of course, by changing $HOME in the instructions above to a project location (e.g., /projects/<your project name>/modules/default/modulefiles), you can create module collections that all users on a project can see and use.

Finally, the modules/default/modulefiles pattern is only convention—you can use any path that fits your needs.

Technical Assistance

Please contact us if you need assistance with your installation or have any questions.