Platform R: Using the GPUs in Slurm
The Platform R cluster has a mix GPUs from Nvidia: L40Ss and H200s. You can see what's available with sinfo -o "%5D %35N %30G"
In Slurm terminology a GPU is a kind of generic resource (GRES). The output format describing a GPU resource is gpu:<GPU type>:<GPU count>. You can see, for example, that there are eight nodes which have four Nvidia L40S GPUs in them.
Updating Your Environment to Use GPUs
CUDA is an API that gives programs access to Nvidia GPUs. Typically your software will say in the documentation which versions of CUDA it will run with. Some tools can discover for themselves what version is installed, but it is not uncommon to find software where you may need to select a library's version to match the version of CUDA installed on the Platform R nodes.
To help your software find CUDA, you need to make a few changes to your shell environment. Platform R uses environment modules to manage this. See Platform R: Environment Modules for details of use. Run this command to add CUDA version 12.8 to your environment:
module load cuda/12.8.2
See GPU Activity
First, connect to a node with a GPU: srun --partition=gpu --gres=gpu:1 --pty bash -I -l
Next, load the CUDA environment: module load cuda/12.8.2
The command nvidia-smi shows what GPUs are available, if they are busy, and with what.
Slurm GPU Options
All Slurm GPU nodes exist in the gpu partition, which is specified with either -p gpu or --partition=gpu in the job options. The default partition, cpu, will not run GPU jobs.
GPU resources are requested with the --gres command line option to sbatch or srun.
| Command Line Option | Meaning |
--gres=gpu:1 |
one GPU of any kind |
--gres=gpu:2 |
two GPUs of any kind |
--gres=gpu:<named-gpu>:1 |
one GPU of the named kind |
GPUs currently available on Platform R are:
| GPU Name | Use Cases |
nvidia_l40s |
graphics, general compute, smaller machine learning workloads |
nvidia_h200 |
image processing, generative AI and LLM training and inference |
nvidia_h200_nvl |
multiple GPU-jobs, or if the memory of one GPU is insufficient |
Container Options
To enable a container to use the GPU you need to include the command line option --nv with sbatch or srun (in addition to the environment variables listed above).
See Platform R: Using Containers in Slurm for examples of interactive and batch workflows with containers using GPUs.
Applications
Some applications need specific instructions during installation to get the correct GPU information and support libraries.
Python Library and CUDA Versions
When using Conda to install a Python library which uses GPUs, you have to pay attention both to the version of the library as well as the CUDA version that library can use. In the example below Ollama is used to show the process of matching library and CUDA versions, but the process is the same for other libraries, such as PyTorch.
When you install a package using Conda the default is to grab the newest version along with support for the newest CUDA libraries. It may be necessary to adjust the CUDA library version to match the Platform R version of CUDA. Especially if it is built with a version of CUDA with a different major version than is installed on Platform R, the library will likely not discover the GPU.
Running conda search ollama will give a long list of versions and builds of Ollama. The builds encode the CUDA version the library has been built with. For example, cuda_129h63bc95c_0 is built against CUDA 12.9. There are also ollama builds using CUDA 13.0, which will not see the GPUs on Platform R due to the mismatch in the library major version with the Platform R installed version of CUDA, which is 12.8, as we saw above.
To select the right version and build run this:
conda install ollama=0.13.4=cuda_129h63bc95c_0
tensorflow
When installing Tensorflow from Conda, use the tensorflow-gpu package instead of just tensorflow. This ensures that the GPU library dependencies are installed.