Skip to main content

Building Conda environments in Apptainer

Apptainer is a containerization technology forked from Singularity. It's free, open source and compatible with Docker and Singularity images. This has the advantage of keeping all of your Conda environment programs in one convienent file and avoiding conflicts and problematic conflicts with other Conda environments on the same machine.

Apptainer containers are built from a definition file, let's start with this example below:

Bootstrap: docker
From: continuumio/miniconda3

%post -c /bin/bash
    # Update and install system wide software here
    apt-get update -y
    # if you need apt packages install them here
    # apt-get install <package>

    # install conda stuff here, numpy just for example
    conda install -y numpy

%environment
    # Ensure Conda environment is in the PATH
     PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$PATH"
     source /opt/conda/etc/profile.d/conda.sh
     conda activate base

Save this file to MyCondaEnv.def on a machine with Apptainer installed. HPC1 is a fine place to do this as we have Apptainer available there for building images. You can also do it on your local machine and upload the resulting SIF file to HPC1 at a later date.

Now is time to build the image:

apptainer build --fakeroot MyCondaEnv.sif MyCondaEnv.def

This may take a minute or two. 

After it has successfully finished you'll be left with the MyCondaEnv.sif file. This is where the actual Conda binaries and packages are stored. Once created it cannot be modified and has to be rebuilt if you want to add a new package. 

Now you can run the Apptainer image and interact with your data in one of two primary ways. Firstly through apptainer exec:

apptainer exec MyCondaEnv.sif conda list

Where everything after the SIF file will be run inside the container. This prints a list of installed conda packages. Alternatively you can use apptainer run and it will drop you into a shell on the container.

[lh59281@hpc2 ~]$ apptainer run MyCondaEnv.sif
(base) ls
AstroInteractives  Pictures           apptainer_test.sh                gauss_test.log       jupyter2                   jupyternoRoot.def  other_data       processing        test_mpi
Desktop            Public             conda.def                        gpu_test.bsh         jupyterhub.sqlite          lh59281_lambda     output-9604.log  shiny-server.def  test_mpi.c
(base)

Notice that here I just did a ls and it lists my home directory. You can any program installed in the Apptainer container on data stored in your home directory.

EnvironmentNotWritableError: The current user does not have write permissions to the target environment.
  environment location: /opt/conda
  uid: 1002
  gid: 1002

These containers are create once, read only and easily rebuilt so the solution is to exit your container shell (Crtl+D) and make a new container. First let's update the DEF file:

Bootstrap: docker
From: continuumio/miniconda3

%post -c /bin/bash
    # Update and install system wide software here
    apt-get update -y
    # if you need apt packages install them here
    # apt-get install <package>

    # install conda stuff here, numpy just for example
    # adding astropy here for a rebuild example
    conda install -y numpy astropy

%environment
    # Ensure Conda environment is in the PATH
     PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$PATH"
     source /opt/conda/etc/profile.d/conda.sh
     conda activate base

Now build a new container:

apptainer build --fakeroot MyCondaEnv2.sif MyCondaEnv.def

After that has completed, run it as before:

apptainer run MyCondaEnv2.sif

Verify the new packages are loaded:

[lh59281@hpc2 ~]$ apptainer run MyCondaEnv2.sif
(base) conda list | grep astropy
astropy                   7.0.0           py313h5eee18b_0
astropy-iers-data         0.2025.6.23.0.39.50 py313h06a4308_0
(base)
Back to top