Environment Modules
- What is Environment Modules?
- Requirements
- How To use EM?
- Advanced Usage
- Other Things to Note
- Summary of Demonstrated Commands
- Where to Get More Information
- Requesting Additional Software
What is Environment Modules?
Environment Modules(EM) is a tool that simplifies modifying your shell environment. Each module defines the changes to be made to the user's shell environment(PATHs, environment variables, aliases, etc). Modules can be loaded and unloaded to enable and disable software packages or specific versions of software packages.
Through EM you almost instantly switch between different versions of a software package or library. You can have multiple modules loaded at any one time. In fact, a module may depend on another module and load it automatically. You can also define "collections" of modules and save them so you can later load the entire collection.
Requirements
The only requirement is that you use the Bash shell which the default for most users. If you are using CSH, Tcsh, or another shell please contact the Math Department IT staff and we'll work with you to determine the best way forward.
How To use EM?
The commands: ml and module
The ml command acts as a shortcut to the module command thus it supports all the command line switches and module sub-commands that are supported by module. We'll use the ml command in all following examples as it is shorter and supports a few additional features.
Available Modules
To list all available modules, simply run ml:
youser@server:~$ ml avail
------------------- /usr/local/modules/spack/linux-ubuntu22.04-x86_64_v4 --------------------
cuda/11.8.0-gcc-11.4.0 cuda/12.2.1-gcc-11.4.0
------------------- /usr/local/modules/spack/linux-ubuntu22.04-x86_64_v3 --------------------
hdf5/1.12.2-gcc-11.4.0-intel-oneapi-mpi-2021.10.0 openmpi/4.1.6-gcc-11.4.0
hdf5/1.12.2-gcc-11.4.0-openmpi-4.1.6
intel-mpi-benchmarks/2021.3-gcc-11.4.0-intel-oneapi-mpi-2021.10.0
intel-oneapi-advisor/2023.2.0-gcc-11.4.0
intel-oneapi-ccl/2021.10.0-gcc-11.4.0
intel-oneapi-compilers/2023.2.1-gcc-11.4.0
intel-oneapi-dal/2023.2.0-gcc-11.4.0
intel-oneapi-dnn/2023.2.0-gcc-11.4.0
intel-oneapi-dpct/2023.2.0-gcc-11.4.0
intel-oneapi-dpl/2022.2.0-gcc-11.4.0
intel-oneapi-inspector/2023.2.0-gcc-11.4.0
intel-oneapi-ipp/2021.9.0-gcc-11.4.0
intel-oneapi-ippcp/2021.8.0-gcc-11.4.0
intel-oneapi-itac/2021.10.0-gcc-11.4.0
intel-oneapi-mpi/2021.10.0-gcc-11.4.0
intel-oneapi-tbb/2021.10.0-gcc-11.4.0
intel-oneapi-vtune/2023.2.0-gcc-11.4.0
llvm/18.1.3-gcc-11.4.0
netcdf-c/4.8.1-gcc-11.4.0-openmpi-4.1.6
netcdf-cxx4/4.3.1-gcc-11.4.0-openmpi-4.1.6
netcdf-fortran/4.5.4-gcc-11.4.0-openmpi-4.1.6
opencl-headers/3.0-gcc-11.4.0
---------------------------------- /usr/local/modules/main ----------------------------------
bertini/1.6 julia/1.9.4 magma/2.28-4-avx2 mathematica/13.3
bertini/2.0-alpha6 julia/1.10.1 magma/2.28-4-avx2-cuda11 matlab/2023b
bertini/2.0-HEAD julia/latest magma/2.28-6-auto phc/2.4.74
dedalus/3.0.2 julia/lts magma/2.28-6-avx phc/2.4.89
dedalus/latest magma/2.28-2-auto magma/2.28-6-avx2 sage/9.5
gap/4.12.0 magma/2.28-2-avx magma/2.28-6-avx2-cuda11 sage/10.0
gap/4.12.2 magma/2.28-2-avx2 maple/2021
intel-oneapi magma/2.28-4-auto maple/2023
julia/1.6.7 magma/2.28-4-avx mathematica/13.2
------------------------------ /usr/local/Modules/modulefiles -------------------------------
dot module-git module-info modules null use.own
Key:
modulepath
Tip: You can abbreviate 'avail' as 'av'
Loading a Module and Listing Loaded Modules
To load a module just list it's full name (name/version) on the ml command. Multiple modules can be loaded this way:
youser@server:~$ ml julia/1.10.1 boost/1.84.0
To list loaded modules just run ml with no arguments:
youser@server:~$ ml
Currently Loaded Modulefiles:
1) julia/1.10.1 2) boost/1.84.0
Module Information
To get some information on a module
youser@server:~$ ml whatis julia/latest
-------------------------------------- /usr/local/modules/main --------------------------------------
julia/latest: Julia v1.10.1
More information may be available using the help command:
youser@server:~$ ml help bertini/2.0-HEAD
-------------------------------------------------------------------
Module Specific Help for /usr/local/modules/main/bertini/2.0-HEAD:
Bertini 2.0-HEAD
Latest code from the B2 GIT repository
Not recommended for general use.
-------------------------------------------------------------------
Tip: You can abbreviate 'whatis' to 'wh'
Tip: At any time you can use the TAB key to auto-complete the ml command. Double-tapping TAB will show you the current possible completions. Try: ml magma<TAB><TAB><TABTAB>
Unloading a Module
youser@server:~$ ml -boost/1.84.0
youser@server:~$ ml
Currently Loaded Modulefiles:
1) julia/1.10.1
Swapping a module
To switch modules in one command:
youser@server:~$ ml swap julia/1.10.1 julia/lts
youser@server:~$ ml
Currently Loaded Modulefiles:
1) julia/lts
Or in this case we are swapping a module of the same name for a different version so just give it the new version:
youser@server:~$ ml swap julia/lts
Removing all modules
youser@server:~$ ml purge
Fixing Problems
There may be times when your shell environment gets changed in a way that causes problems. For example an environment variable set by loading a module is changed and this causes the module to not function properly. You can reload all modules using the reload command to fix this:
youser@server:~$ echo $GAP_DIR
youser@server:~$ ml gap/4.12.0
youser@server:~$ echo $GAP_DIR
/usr/local/Gap/gap-4.12.0
youser@server:~$ unset GAP_DIR # We just broke it!
youser@server:~$ echo $GAP_DIR # Oh noes!
youser@server:~$ ml reload # reload to the rescue
youser@server:~$ echo $GAP_DIR # Yay!
/usr/local/Gap/gap-4.12.0
Advanced Usage
Creating Collections for Later Use
Lets say you have some projects that each use a number of modules. In order to set up your shell environment to work in one project or to switch between them projects requires some EM commands that you need to remember. You can bundle the requirements into "collections" and recall those with a single command.
Create a Collection
youser@server:~$ ml boost/1.84.0 fftw/3.3.10-avx2 intel-oneapi julia/1.10.1
youser@server:~$ ml
Currently Loaded Modulefiles:
1) boost/1.84.0 2) fftw/3.3.10-avx2 3) intel-oneapi 4) julia/1.10.1
youser@server:~$ ml save project1
youser@server:~$ ml purge
youser@server:~$ ml
Load a Collection
youser@server:~$ ml restore project1
Loading boost/1.84.0
Loading fftw/3.3.10-avx2
Loading intel-oneapi
Loading julia/1.10.1
List Collections
youser@server:~$ ml savelist
Named collection list:
1) project1
Delete a Collection
youser@server:~$ ml saverm project1
youser@server:~$ ml savelist
No named collection.
Using Modules in Scripts
If you want to activate modules in Bash scripts you can just add the required commands to the script. Take note: If you load modules on the command line and then run a shell script those modules will continue to be loaded for that script as your shell environment at the command line is inherited by the shell script.
#!/bin/bash
ml maple/2023
# The rest of the script...
Other Things to Note
- You can only load one version of a module at a time
- Some modules cannot be loaded at the same time because their software conflicts. This is usually due to Conda-based installations.
- Some modules will work only on certain servers because there are CPU and/or GPU requirements. These requirements are usually made clear in the module name.
Summary of Demonstrated Commands
| Command | Use |
| ml | List loaded modules |
| ml avail | List available modules |
| ml <module name/version> ... | Load modules |
| ml -<module name/version> ... | Unload modules |
| ml whatis <module name/version> | Display brief module information |
| ml help <module name/version> | Display module help information |
| ml swap <module name> OR ml swap <module name/version> <module name/version> |
Swap a loaded module with another one |
| ml purge | Unload all loaded modules |
| ml reload | Reload all modules |
| ml save <collection name> | Create a collection of modules |
| ml savelist | Lists saved collections |
| ml restore <collection name> | Load a collection of modules |
| ml saverm <collection name> | Delete a collection |
Where to Get More Information
Command help: module help
Manual page(advanced): man module
Documentation:
Web pages:
- Github page: https://github.com/cea-hpc/modules
- Home page: https://modules.sourceforge.net/
- All documentation: https://modules.readthedocs.io/en/latest/
Requesting Additional Software
Contact the Math Department IT staff if there is any software(including libraries, build tools, languages, etc) you would like installed on the servers: mathitstaff@g-groups.wisc.edu