Python, Anaconda and how to install additional packages

A note on Anaconda/Miniconda

Dear users,

we know that many of you would like to install or are using Anaconda/Miniconda already to manage their Python packages. No one can prevent you from doing so, and in some cases this is justified. We know its ease of use is very appealing, however, we strongly recommend against using it. The reasons for this are the following:

• Conda provides not just Python packages but includes various other (system-)software dependencies which are installed alongside
• Conda packages are distributed as pre-compiled binaries which might perform poorly on the cluster
• Conda adds itself to your .bashrc, which can cause conflicts with the Python versions we provide when loading other modules (many users do not notice this!)
• Conda environments do not create reproducible software builds, i.e. versions of dependencies might change without you noticing

We do provide Python installations and various packages (i.e. scipy, numpy, pandas, matplotlib etc.) based on the intel or gcc compiler, which are optimized to perform very well on the cluster. If you need to install Python packages yourself, we will explain how to do so in this article. If you run into problems you can always contact support at hpc@uni-muenster.de. There are some occasions, where anything other than conda fails but we would like to try every other option before going this route.

Thank you.

Installing additional packages

You may want to install Python libraries that are not provided by the default  installation or any other module. These can be installed directly to your home directory without any admin privileges using the following procedure.  Note that some packages might need external libraries. Requirements can usually be found in the package documentation. These libraries have to be loaded before by using the module load command.

1. Load the base Python module from the intel or foss toolchain

module load foss/2019a
module load Python/3.7.2

2. Install a Python package using pip and add the --user flag to make sure pip will install the package to your home directory. The default location for where packages are installed will be ~/.local/lib/<python-version>/site-packages/ where <python-version> would be for example python37

pip install --user <PACKAGE-NAME>