Docs
Jaspy Software Environments (Python 3, R and other tools)
Jaspy Software Environments (Python 3, R and other tools)
Important changes took place in September 2024 affecting what software can be used on JASMIN. Please read this announcement carefully.
The information below has been updated in line with this announcement.
This page provides details of the “Jaspy” software environments that provide access to Python 3, R and a range of other tools on JASMIN.
Overview
Jaspy is a toolkit for managing and deploying Conda environments that include Python, R and other packages. Jaspy is used to provide software environments of common packages on the scientific analysis servers and LOTUS cluster on JASMIN.
One advantage of Jaspy is that multiple environments can co-exist on the same platform. This allows us to retain previous environments and provide new ones simultaneously. This may be particularly useful for scientists undertaking long-running studies that require a consistent software environment to ensure reproducibility and continuity.
Working with Jaspy environments
Quickstart for Python 3 environment
If you want to get on, you can select a Jaspy environment to “activate”. This means that once you have run these commands then the various tools and libraries will be available in your current session.
module load jaspyActivating the environment in scripts
If you want a particular script to activate a Jaspy environment then add the “module” command to it, e.g.:
#!/bin/bash
module load jaspy
python do-something.pySetting your profile to always use a Jaspy environment
If you want all your JASMIN sessions to use a particular Jaspy environment
then you can add the module load jaspy command to your $HOME/.bashrc file.
In order to avoid issues with using “module load” on
unsupported servers, please wrap the call in an “if” clause, such as:
if [[ $(hostname) =~ (sci[0-9]|host[0-9]|cylc) ]] ; then
module load jaspy
fiDiscover which environments are available
You can list the currently available Jaspy environments using:
module avail jaspy
-------------------------------- /apps/jasmin/modulefiles ----------------------------------
jaspy/3.10/v20230718 jaspy/3.11/v20240508 snappy/8.0/jaspy-3.7-r20210320
jaspy/3.11/v20240302 jaspy/3.11/v20240815 (D)This lists all jaspy modules (i.e. environments) that can be loaded.
Jaspy Python (plus other tools) environments
The packages available in the Jaspy environments can be found by searching the GitHub repository where the Conda environment files are defined. This table lists all the Jaspy Python 3.7+ environments provided on JASMIN and specifies the current (default) version.
| Jaspy Python Environment | Versioned list of software packages | Default? | Comments / Issues |
|---|---|---|---|
| jaspy/3.10/v20230718 | List of packages including versions | No | |
| jaspy/3.11/v20240302 | List of packages including versions | No | |
| jaspy/3.11/v20240508 | List of packages including versions | No | |
| jaspy/3.11/v20240815 | List of packages including versions | Yes | Release notes |
Jaspy Python 2.7 (plus other tools) environments
Python 2.7 environments are no longer supported.
Jasr R environments
Environments for the “R” programming language are packaged into separate software environments, known as “Jasr”. This table lists all the Jaspy R environments provided on JASMIN and specifies the current (default) version.
We are aware of a newly discovered vulnerability in the R Language (CVE-2024-27322) which allows arbitrary code execution from maliciously built RDS (R Data Serialisation) files.
We will be updating to the latest version of R as soon as possible to remove this vulnerability, but we do not plan to remove access to R beforehand. Our advice, as always, is to not open data from untrusted sources and not to install untrusted packages from CRAN.
Please note that this position may change at short notice as more information becomes available- this notice was last updated on Friday 10th May 2024.
| Jaspy R Environment (“Jasr”) | Versioned list of software packages | Default? |
|---|---|---|
| jasr/4.2/v20230718 | List of packages including versions | No |
| jasr/4.3/v20240320 | List of packages including versions | No |
| jasr/4.3/v20240815 | List of packages including versions | Yes |
The available R environments can be listed with:
module avail jasrUnderstanding versioning with Jaspy/Jasr
Jaspy environments are labelled as “jaspy/<python_version>/
module load jaspy/3.10/v20230718However, if you wish to get the latest environment for a given Python version
you can omit the “
module load jaspy/3.10And if you just want the most up-to-date Python you can even omit the
<python_version>, as follows:
module load jaspy<release> and <python_version>
components then it is important to be aware that the resulting environment may
differ over time. For continuity, you ay wish to use the full
environment specification.
How Jaspy works: managing Python and non-Python packages using conda
Jaspy is a framework for managing multiple Python (and other) environments simultaneously on a single platform. It was created in order to meet the requirements tabulated below.
| Requirement | Details | Jaspy solution | Further info |
|---|---|---|---|
| Reproducibility | 1. Generate a specific set of packages and versions from a generic set of requirements. | 1. Conda has a powerful package-management workflow: a. Begin with a minimal set of package/version requirements. b. Generate a consistent environment. c. Provide a detailed description of all exact packages/versions in the environment. |
Conda: https://docs.conda.io jaspy-manager: https://github.com/cedadev/jaspy-manager/blob/master/README.md CEDA jaspy environments: https://github.com/cedadev/ceda-jaspy-envs |
| Documentation | Provide an appropriate level of documentation detailing which software packages exist in each release. | We use Conda “environment files” to build the environments. These list the packages and versions and are stored in public GitHub repositories, so each environment is documented as a collection of packages/versions. | See: https://github.com/cedadev/jaspy-manager/blob/master/README.md Example package list: https://github.com/cedadev/ceda-jaspy-envs/blob/master/environments/py3.7/m3-4.5.11/jaspy3.7-m3-4.5.11-r20181219/packages.txt |
| Multiple simultaneous environments | Allow multiple, but separate, software environments to co-exist on a single operating system. | Conda is designed to allow multiple environments to co-exist. Within jaspy it is possible to document each environment. Therefore, multiple environments can be deployed on one system. Key advantages are: - Supporting multiple versions of Python and side-by-side. - Releasing an update to an environment as a “pre-release” so that users can adapt their code and test it whilst still having access to the “current” (production) environment. |
|
| Manageability | Provide tools to easily construct, test, deploy, document and reproduce software environments. | Jaspy builds upon a set of excellent Conda command-line tools that simplify the package management process. Jaspy wraps the Conda functionality so that command-line tools can be used to build, test, deploy and distribute Conda environments for use by our community. |
Updates and tracking of Jaspy/Jasr environments
History of environments on JASMIN
Please see the Jaspy Python (and other tools) environments section above for information about releases on JASMIN.
Which environment is “current”?
Please refer to the Jaspy Python (and other tools) environments section above for information about the current release on JASMIN.
Citing Jaspy environments
Can I cite a jaspy (conda) environment?
We do not yet have an agreed approach for citing a Jaspy environment. However, you can refer to the environment description URLs given in the table above. These provide a definitive list of the software packages, their versions and other information.
Requesting updates to a Jaspy environment
If you would like us to add a new package, or an updated version, to the Jaspy environments on JASMIN then please use one of the following approaches:
- Contact the JASMIN helpdesk with the subject: “Request for Jaspy update:
” - Get a GitHub account and add an issue to the
ceda-jaspy-envsrepository at: 1. https://github.com/cedadev/ceda-jaspy-envs/issues/new
Conda method of “activating” Jaspy environments
Jaspy environments can also be activated in a more traditional way using standard the standard conda approach, for example:
List the available environments:
conda info --envs
# conda environments:
#
base * /apps/jasmin/jaspy/miniforge_envs/jaspy3.11/mf3-23.11.0-0
jaspy3.11-mf3-23.11.0-0-v20240302 /apps/jasmin/jaspy/miniforge_envs/jaspy3.11/mf3-23.11.0-0/envs/jaspy3.11-mf3-23.11.0-0-v20240302
jaspy3.11-mf3-23.11.0-0-v20240508 /apps/jasmin/jaspy/miniforge_envs/jaspy3.11/mf3-23.11.0-0/envs/jaspy3.11-mf3-23.11.0-0-v20240508
jaspy3.11-mf3-23.11.0-0-v20240815 /apps/jasmin/jaspy/miniforge_envs/jaspy3.11/mf3-23.11.0-0/envs/jaspy3.11-mf3-23.11.0-0-v20240815Select one of them, e.g. jaspy3.11-mf3-23.11.0-0-v20240302 and set up to activate it:
export PATH=/apps/jasmin/jaspy/miniforge_envs/jaspy3.11/mf3-23.11.0-0/envs/jaspy3.11-mf3-23.11.0-0-v20240302:$PATH
source activate
conda activate jaspy3.11-mf3-23.11.0-0-v20240302Prompt changes to:
This has the same result as the module load approach. The naming of the
environment identifiers includes the “miniforge” version used to generate the
environment. The module load approach is recommended as the standard method
for activating Jaspy environments.
Using Jaspy beyond JASMIN
Jaspy is a versatile and generic tool for managing multiple conda environments. The code is open source, and more information is available at:
https://github.com/cedadev/jaspy-manager