WORK IN PROGRESS!

Introduction

EasyBuild  is a framework for building and managing software on HPC systems. It automates compiling, installing, and configuring scientific applications while handling dependencies, toolchains, and environment modules.

A key concept is easyconfig files — text-based receipts that define build parameters, dependencies, versions, and toolchains. By using these receipts, EasyBuild installs software along with all its dependencies, ensuring a consistent, reproducible, and optimized HPC software environment.

How it works? The dependency resolution mechanism constructs a full dependency graph for the software package(s) being installed, after which a list of dependencies is composed for which no module is available yet. Each of the retained dependencies will then be built and installed, in the required order as indicated by the dependency graph.

This page shows how to install IMAS ecosystem using EasyBuild. The installation will be performed on the Gateway cluster but it can be done anywhere.

Prerequisites

Before you start, make sure that:

Downloading easyconfigs

To allow the users to seamlessly switch between EUROfusion and ITER clusters, the environment that is built on the Gateway mimics (to some extent) the one of the ITER SDCC machines. 

To achieve this a receipts from the ITER cluster are utilised to install software on the GW.

Downloading configuration (EB) files
Source SDCC: /work/imas/opt/EasyBuild/ebfiles_repo/ 

rsync -av --delete -e "ssh -J <user>@sdcc-login.iter.org"  <user>@sdcc-login02.iter.org:/work/imas/opt/EasyBuild/ebfiles_repo/ ./iter-receipts/
System variable EASYCONFIGS_PATH

Set system variable EASYCONFIGS_PATH  that points to the directory with receipts

export EASYCONFIGS_PATH=`pwd`/iter-receipts


This simple mechanism will be replaced with a more complex one, where receipts would be maintained in GIT repositories.


Installation of software components

Searching for receipts

EasyBuild needs a receipt to install any software component. The following command can be used to look for receipts:

eb --search <software_name> --robot-paths=$EASYCONFIG_PATHS

 --robot-paths=<path1>:<path2>  configuration option supports prepending or appending to the default robot search path:

  • To prepend one or more paths, a list of paths followed by a ‘:’ should be specified.

  • To append one or more paths, a list of paths preceded by a ‘:’ should be specified.

See EasyBuild documentation for details and examples

A <software_name>  could be only part of the name being searched. Additionally EasyBuild search is case insensitive. E.g. "dox"  string can be provided while looking for "Doxygen" 

shell> eb --search dox --robot=$EASYCONFIG_PATHS

== found valid index for /gw/swimas/easybuild/opt/software/EasyBuild/5.0.0/easybuild/easyconfigs, so using it...
 * /afs/eufus.eu/user/g/g2bpalak/work/easybuild/all_iter/Doxygen/Doxygen-1.9.8-GCCcore-13.2.0.eb
 * /afs/eufus.eu/user/g/g2bpalak/work/easybuild/all_iter/Doxygen/Doxygen-1.13.2-GCCcore-13.2.0.eb
...
 * /gw/swimas/easybuild/opt/software/EasyBuild/5.0.0/easybuild/easyconfigs/d/Doxygen/Doxygen-1.9.8-GCCcore-13.2.0.eb
 * /gw/swimas/easybuild/opt/software/EasyBuild/5.0.0/easybuild/easyconfigs/d/Doxygen/Doxygen-1.11.0-GCCcore-13.3.0.eb

Usually receipt name uses following syntax:

<software name>-<software version>-<toolchain>-<toolchain version>

e.g.:
Doxygen-1.9.8-GCCcore-13.2.0.eb

Installation


eb <receipt name>.eb --robot --robot-paths=$EASYCONFIG_PATHS

e.g.:
eb Doxygen-1.9.8-GCCcore-13.2.0.eb --robot --robot-paths=$EASYCONFIGS_PATHS 

  •  --robot/-r command line option enables dependency resolution See EasyBuild documentation for details

  • It is strongly advised to change temporary directories to use $ITMWORK

  • It is recommended to run before actual installation:

    • eb with option -missing-modules/-M  to check what dependencies will be installed

    • eb  with option --dry-run/-D  or --extended-dry-run/-x,  to check what and how will be installed

  • All 'massive' builds should be performed on the worker nodes 
  • See EB help for the other useful eb switches

Installation of the 'production' software

To install the 'production' version of the software belonging to IMAS ecosystem, a few extra options have to be added to point out EB to the dedicated public directory /gw/swimas/easybuild  


--prefix='/gw/swimas/easybuild/opt' 
--installpath-modules='/gw/swimas/easybuild/etc'
--ignore-checksums (optional)


# TO BE DISCUSSED: Accessing sources while debugging. Should 'source' directory be public, as well?

Installation of the IMAS bundle

The IMAS bundle is (usually) provided by ITER and loads all IMAS High Level Interfaces and related modules:

Content of IMAS bundle

IMAS bundle consist of:

IMAS bundle receipts

The IMAS bundle receipts name usually contains the 'IMAS' string, Data Dictionary version, date of release and toolchain version, e.g.: IMAS-3.42.0-2024.09-foss-2023b.eb  or IMAS-4.0.0-2024.12-foss-2023b.eb  

Before installation search for a proper IMAS bundle version

shell> eb --search IMAS-3 --robot-paths=$EASYCONFIG_PATHS

== found valid index for /gw/swimas/easybuild/opt/software/EasyBuild/5.0.0/easybuild/easyconfigs, so using it...
...
 * /gss_efgw_work/work/g2bpalak/easybuild/iter-receipts/IMAS/IMAS-3.42.0-2024.08-foss-2023b.eb
 * /gss_efgw_work/work/g2bpalak/easybuild/iter-receipts/IMAS/IMAS-3.42.0-2024.08-intel-2023b.eb
...
 * /gss_efgw_work/work/g2bpalak/easybuild/iter-receipts/IMAS/IMAS-3.42.0-2024.09-foss-2023b.eb
 * /gss_efgw_work/work/g2bpalak/easybuild/iter-receipts/IMAS/IMAS-3.42.0-2024.09-intel-2023b.eb
...


Every IMAS High Level Interface depends on the IMAS Access Layer Core ( IMAS-AL-Core ) 

Make sure that all IMAS HLIs use exactly the same version of  IMAS-AL-Core and other common dependencies !!!

Installation

Once the receipt with a proper version of Data Dictionary and toolchain is found, the IMAS bundle can be installed as any other software:

eb <IMAS bundle receipt>.eb --robot --robot-paths=$EASYCONFIG_PATHS 

e.g.:
eb IMAS-3.42.0-2024.08.1-intel-2023b.eb --robot --robot-paths=$EASYCONFIG_PATHS

How to use IMAS

After installation you can load IMAS  modules i.e:

module use {prefix}/modules/all
module load IMAS-AL-Cpp/5.2.0dev-foss-2020b-DD-3.40.1
module load IMAS-AL-Fortran/5.2.0dev-foss-2020b-DD-3.40.1
module load IMAS-AL-Java/5.2.0dev-foss-2020b-Java-11-DD-3.40.1
module load IMAS-AL-Python/5.2.0dev-foss-2020b-DD-3.40.1



imasenv module

Adding software components to existing installation


Useful tips

Installing dependencies as hidden modules

The key feature of the EasyBuild is the ability to install a software component with all its dependencies. Unfortunatlely EasyBuild installs module not only for the requested software but for every dependency as well.

It results with the number of available modules growing with every installation, becoming less and less hard to navigate for the users. 

There are few options to hide modules, unfortunately every of them has its disadvantages, 

Change the default tmp directory

Easybuild produces a lot of temporary content (logs, intermediate build files, etc)

The default /tmp  dir on the GW offers quite small capacity. Once tmp is full, system cannot work properly.

It is strongly recommended to change the default /tmp  to a directory in $ITMWORK 


tmpdir=$ITMWORK/tmp

export TMP=${tmpdir}
export TEMP=${tmpdir}
export TMPDIR=${tmpdir}

Run a massive builds on the worker nodes


# Run 'interactive' batch job using SLURM 'srun' command
# -p gw                  : 'gw' queue offering the longer walltime than the default 'gwdebug'
# --ntasks-per-node=48   : number of cores requested (1 node provides up to 48 cores)
# --time=48:0:0          : walltime (documentation says that the default walltime in 'gw' queue is 48h but not explicitly specified is set to 1h - To Be Checked)
# --x11                  : X11 forwarding (optional)
# --pty /bin/bash        : pseudoterminal

bash> srun -p gw --ntasks-per-node=48 --time=48:0:0 --pty /bin/bash


Once resources are allocated, a new shell opens on the working node usually reporting file system error:

bash: /afs/eufus.eu/user/g/g2<user>/.bashrc: Permission denied


It means that the Kerberos token should be set/renewed, to allow access to the filesystem:

bash> kinit; aklog


Now .bash_profile  could be 'sourced' to enable user's startup settings:

bash> source .bash_profile


Now EasyBuild can be used to build a software.


  • Add --parallel=48  to EB commands (on the working nodes EB doesn't recognise properly  a number of available cores and runs sequentially - To Be Checked)
  • EB has built-in feature that allows to submit build to the batch system (see this link) - To Be Checked


Increasing the length of error/warning message

Easybuild shows only the first 300 characters of error/warning message and cuts (usually the most interesting) remaining part of the message.

The message length value is hardcoded and can be change only by 'hacking' the EB codes:

  1. Open <easybuild virtual env>/lib/python3.10/site-packages/easybuild/framework/easyblock.py 
  2. Look for value "300" (line 4295 in EB version 4.9.4) 
    except EasyBuildError as err:
        first_n = 300   # <=== HERE is the limit hardcoded 
        errormsg = "build failed (first %d chars): %s" % (first_n, err.msg[:first_n])  
        _log.warning(errormsg)
        result = False
  3. Increase the limit to bigger value (e.g. 3000) or remove the message cutting err.msg[:first_n])  ==> err.msg 
  4. Save the file    


Other