WORK IN PROGRESS!

 Table of Contents

Introduction

EasyBuild is a framework for building and managing software on HPC systems. It automates compiling, installing, and configuring scientific applications while handling dependencies, toolchains, and environment modules.

A key feature concept is easyconfig files — text-based receipts that define build parameters, dependencies, versions, and toolchains. By using these receipts, EasyBuild installs software along with all its dependencies, ensuring a consistent, reproducible, and optimized HPC software environment.

...

Before you start, make sure that:

• Easybuild is installed and publicly available via module system

• Easybuild configuration file exists and is properly configured

• IMAS related receipts are

  downloaded from all their sources being used

  downloaded 

• All modules are purged and EasyBuild module is the only module loaded:

  Code Block
  module purge module load EasyBuild

• $EASYCONFIG_PATHS system variable is set to point to all directories with easyconfig files 

• Your TMP  directory is set up to use $ITMWORK

...

Downloading easyconfigs

To allow the users to seamlessly switch between EUROfusion and ITER clusters, the environment that is built on the Gateway mimics (to some extent) the one of the ITER SDCC machines. 

...

This simple mechanism will be replaced with a more complex one, where receipts would be maintained in GIT repositories.

Downloading configuration (EB) files

Source SDCC: /work/imas/opt/EasyBuild/ebfiles_repo/ 

rsync -av --delete -e "ssh -J <user>@sdcc-login.iter.org"  <user>@sdcc-login02.iter.org:/work/imas/opt/EasyBuild/ebfiles_repo/ ./iter-receipts/

System variable EASYCONFIGS_PATH

Set system variable EASYCONFIGS_PATH  that points to the directory with receipts

...

eb --search <software_name> --robot-paths=$EASYCONFIG_PATHS

 --robot-paths=<path1>:<path2>  configuration option supports prepending or appending to the default robot search path:

See EasyBuild documentation for details and examples

A <software_name>  could be only part of the name being searched. Additionally EasyBuild search is case insensitive. E.g. "dox"  string can be provided while looking for "Doxygen" 

shell> eb --search dox --robot=$EASYCONFIG_PATHS

== found valid index for /gw/swimas/easybuild/opt/software/EasyBuild/5.0.0/easybuild/easyconfigs, so using it...
 * /afs/eufus.eu/user/g/g2bpalak/work/easybuild/all_iter/Doxygen/Doxygen-1.9.8-GCCcore-13.2.0.eb
 * /afs/eufus.eu/user/g/g2bpalak/work/easybuild/all_iter/Doxygen/Doxygen-1.13.2-GCCcore-13.2.0.eb
...
 * /gw/swimas/easybuild/opt/software/EasyBuild/5.0.0/easybuild/easyconfigs/d/Doxygen/Doxygen-1.9.8-GCCcore-13.2.0.eb
 * /gw/swimas/easybuild/opt/software/EasyBuild/5.0.0/easybuild/easyconfigs/d/Doxygen/Doxygen-1.11.0-GCCcore-13.3.0.eb  

Usually receipt name uses following syntax:

<software name>-<software version>-<toolchain>-<toolchain version>

e.g.:
Doxygen-1.9.8-GCCcore-13.2.0.eb

Installation

...

Code Block
language bash title IMAS
eb <receipt name>.eb --allow-modules-tool-mismatch --robot=$EASYCONFIG_PATHS e.g.: eb Doxygen-1.9.8-GCCcore-13.2.0.eb --robot --robot-paths=$EASYCONFIGS_PATHS

Info
--robot/-r command line option enables dependency resolution See EasyBuild documentation for details https://docs.easybuild.io/version-specific/eb-help

Info
title Hints
--robot/-r command line option enables dependency resolution See EasyBuild documentation for details It is recommended to run It is recommended to run eb  with option {{--dry-run/-D.     with option --extended-dry-run or -x See https://docs.easybuild.io/version-specific/eb-help for the other useful eb switches

Installation of IMAS

IMAS bundle

...

• IMAS High Level Interfaces
  
  • IMAS-AL-Cpp
  • IMAS-AL-Fortran
  • IMAS-AL-Java
  • IMAS-AL-Matlab
  • IMAS-AL-Python
• Related modules:
  • IDStools
    
  Clone repo with easyconfigs: TBD
  • 
    

• Run installation: 

  Code Block
  eb ./ebfiles_repo/IMAS/IMAS-3.42.0-2024.08.1-foss-2023b.eb --allow-modules-tool-mismatch --robot=$EASYCONFIG_PATHS  eb ./ebfiles_repo/IMAS/IMAS-3.42.0-2024.08.1-intel-2023b.eb --allow-modules-tool-mismatch --robot=$EASYCONFIG_PATHS

...

module use {prefix}/modules/all
module load IMAS-AL-Cpp/5.2.0dev-foss-2020b-DD-3.40.1
module load IMAS-AL-Fortran/5.2.0dev-foss-2020b-DD-3.40.1
module load IMAS-AL-Java/5.2.0dev-foss-2020b-Java-11-DD-3.40.1
module load IMAS-AL-Python/5.2.0dev-foss-2020b-DD-3.40.1

--installpath-modules=/gw/swimas/easybuild/etc --use-existing-modules --ignore-checksums

Installation of the 'production' software

...

--prefix='/gw/swimas/easybuild/opt' 
--installpath-modules='/gw/swimas/easybuild/etc'
--use-existing-modules 
--ignore-checksums (optional)
--allow-modules-tool-mismatch

# TO BE DISCUSSED: Accessing sources while debugging. Should 'source' directory be public, as well? 

# following modules must be installed 
# pkgconfig
# Cmake

...

The key feature of the EasyBuild is the ability to install a software component with all its dependencies. But Unfortunatlely EasyBuild installs module not only for the requested software but for every dependency as well.

The It results with the number of available modules is growing with every installation, becoming less and less intuitive / useful hard to navigate for the users. 

There are few options to hide modules, unfortunately every of them has its disadvantages, 

• eb --hide-deps=<moduleName1>,<moduleName2>,..., <moduleNameN>   option of eb requires for all modules to be explicitly listed

• eb --hide-all-dependencies -  implemented in working branch but PR#3217 has been not merged since 2020
• module-hide <module-name> - Lmod module system command (see documentation). . Modules have to be hide one by one (see documentation).

Change the default tmp directory

Anchor
tmp_dir_change tmp_dir_change

...