Using Conda on Euler

Overview

Euler users sometimes want to use third-party environment and package managers to set up the computational environment for their projects. This article aims to demystify a few of the obstacles that users might encounter when using conda for that purpose.

Using Conda

Using conda on Euler is a bit different than on other systems. Batch jobs and environmentmodules increase the complexity of using an environment manager, and as such, there are some eccentricities that need to be taken into account. The following two commands are used to access and initialize conda in a shell or batch script.

module load conda/miniforge
bootstrap-conda

After running these commands, other commands such as conda activate should work as normal.

Recommended Practices

• Don't use conda init
• Don't mix conda and pip if you can help it.
  • If you must combine them, always install all conda packages first, and only add pip ones afterward.
• Be mindful of the order in which you load environment modules for a particular conda environment.
  • If you load an environment module before initializing conda, be sure to always load it beforehand. If you load it afterward, always load it afterward.
• Clean up old environments when you're done with them. Conda caches a lot of packages. If you have completed a project, export the environment and then remove it.

Why not use conda init?

If you've installed a distribution of conda before, you may have noticed that the installer recommends that you run conda init at the end of the installation to add conda to your shell startup files. However, we recommend that you DO NOT configure it that way onEuler. Conda can conflict with loadable environment modules, can override system packages, and often initializes incorrectly when used in a Slurm batch script. All of these are potential problems for reproducibility in your research and they complicate supporting you if you run into an environment-related problem.

Should you choose to use conda init, understand that you do so at your own risk and that CAE will only provide a minimum of assistance if you run into problems in your environment.

Supported Tools

Euler provides access to a few conda distributions for students and researchers to use.

• Miniforge (recommended)

  • A conda distribution configured for use with conda-forge.
  • Contains mamba¹ as well, for increased installation performance.
  • Preinstalled Module: conda/miniforge

• Miniconda (deprecated)

  • A smaller alternative to the commercial "Anaconda Distribution".
  • Preinstalled Module: anaconda/mini²

Footnotes