Creating and Using Bundle Overlays

📦 CondaTainer allows you to create read-only, highly-compressed bundle overlays for your project.

They are ideal for sharing pre-configured environments with collaborators.

Table of Contents

Create a Bundle Overlay
Launch a Shell within the Bundle Overlay
Use Bundle Overlays in a Script

Create a Bundle Overlay

You can create a read-only SquashFS bundle overlay using the following command:

condatainer create -p prefix_name -f environment.yml

Make sure the conda env YAML file is properly defined.

Warning

CondaTainer uses the name to locate resources inside the overlay. The bundle overlay should not be renamed after creation. See Naming Convention for more details.

Note

If your environment uses packages not available in conda, like using pip or remotes::install_github(), check Custom OS Overlays for more details.

But you can also share the writable workspace overlay with others.

Note

If you only need a slight modification of a conda env, like editing Python package code, see Custom Bundle Overlays using Build Scripts.

Launch a Shell within the Bundle Overlay

To activate the bundle overlay, run the following command:

condatainer exec -o prefix_name.sqf

Then you can use the applications installed in the overlay.

Bundle overlays are read-only and stackable. You can mount multiple overlays together:

condatainer exec \
  -o grch38/salmon/1.10.2/gencode47 \
  -o prefix_name.sqf

Note

Overlays mounted later will appear earlier in the $PATH (i.e., be prepended).

Use Bundle Overlays in a Script

You can use the bundle overlay in your job scripts as follows:

For example, you have the following project structure:

project/
├── custom.sqf
└── src/
    ├── run_job.sh
    └── test.py

run_job.sh:

#!/bin/bash
#SBATCH --job-name=test_env
## Other SBATCH directives

condatainer exec \
    -o custom.sqf \
    python src/test.py

You can also write the run_job.sh in this way:

#!/bin/bash
#SBATCH --job-name=test_env
## Other SBATCH directives
#DEP:custom.sqf

python src/test.py

You should run this from the project directory:

condatainer run src/run_job.sh