WORK IN PROGRESS!

Introduction

EasyBuild  is a framework for building and managing software on HPC systems. It automates compiling, installing, and configuring scientific applications while handling dependencies, toolchains, and environment modules.

A key feature is easyconfig files — text-based receipts that define build parameters, dependencies, versions, and toolchains. By using these receipts, EasyBuild installs software along with all its dependencies, ensuring a consistent, reproducible, and optimized HPC software environment.

How it works? The dependency resolution mechanism constructs a full dependency graph for the software package(s) being installed, after which a list of dependencies is composed for which no module is available yet. Each of the retained dependencies will then be built and installed, in the required order as indicated by the dependency graph.

This page shows how to install IMAS ecosystem using EasyBuild. The installation will be performed on the Gateway cluster but it can be done anywhere.

Prerequisites

Before you start, make sure that:

• Easybuild is installed and publicly available via module system
• Your Easybuild configuration file exists and is properly configured
• IMAS related receipts are downloaded from all repositories being used
• $ROBOT_PATHS system variable is set to point to all directories with easyconfig files 
• Your TMP  directory is set up to use $ITMWORK

• All modules are purged and EasyBuild module is the only module loaded:

  module purge
  module load EasyBuild

•  

• sss

Software installation receipts

Sources of the receipts (aka 'easyconfig' files):

• EasyBuild built-in receipts
• EasyBuild receipts repository
• Gateway specific receipts
• 'Limbo' of the IMAS receipts
• ITER repository of the IMAS receipts 

Downloading configuration (EB) files

SDCC: /work/imas/opt/EasyBuild/ebfiles_repo/ 

rsync -av --delete -e "ssh -J <user>@sdcc-login.iter.org"  <user>@sdcc-login02.iter.org:/work/imas/opt/EasyBuild/ebfiles_repo/ ./all_iter/

Clone these repositories and  specify the paths to them in the --robot  in the next step.

export ROBOT_PATHS=`pwd`/all_iter

To enable dependency resolution, use the --robot command line option (or -r for short):

https://mdc-easybuild.readthedocs.io/en/latest/Using_the_EasyBuild_command_line.html#enabling-dependency-resolution-robot-r-and-robot-paths

Searching for receipts

EasyBuild needs a receipt to install any software component. The following command can be used to look for receipts:

eb --search <software_name> --robot-paths=$ROBOT_PATHS

A <software_name>  could be only part of the name being searched. Additionally EasyBuild search is case insensitive. E.g. "dox"  string can be provided while looking for "Doxygen" 

shell> eb --search dox --robot=$ROBOT_PATHS

== found valid index for /gw/swimas/easybuild/opt/software/EasyBuild/5.0.0/easybuild/easyconfigs, so using it...
 * /afs/eufus.eu/user/g/g2bpalak/work/easybuild/all_iter/Doxygen/Doxygen-1.9.8-GCCcore-13.2.0.eb
 * /afs/eufus.eu/user/g/g2bpalak/work/easybuild/all_iter/Doxygen/Doxygen-1.13.2-GCCcore-13.2.0.eb
...
 * /gw/swimas/easybuild/opt/software/EasyBuild/5.0.0/easybuild/easyconfigs/d/Doxygen/Doxygen-1.9.8-GCCcore-13.2.0.eb
 * /gw/swimas/easybuild/opt/software/EasyBuild/5.0.0/easybuild/easyconfigs/d/Doxygen/Doxygen-1.11.0-GCCcore-13.3.0.eb  

Usually receipt name uses following syntax:

<software name>-<software version>-<toolchain>-<toolchain version>

e.g.:
Doxygen-1.9.8-GCCcore-13.2.0.eb

Installation of software components

eb <receipt name>.eb --allow-modules-tool-mismatch --robot=$ROBOT_PATHS

e.g.:
eb IMAS-AL-Cpp-5.2.0dev-foss-2020b-DD-3.40.1.eb --allow-modules-tool-mismatch --robot=$ROBOT_PATHS 

Remember to change the --robot and pass paths to your receipt  directories. 

Installation of IMAS

IMAS bundle

The IMAS bundle is (usually) provided by ITER and loads all IMAS High Level Interfaces and related modules:

• IMAS High Level Interfaces
  
  • IMAS-AL-Cpp
  • IMAS-AL-Fortran
  • IMAS-AL-Java
  • IMAS-AL-Matlab
  • IMAS-AL-Python
• Related modules:
  • IDStools
    

• Clone repo with easyconfigs: TBD
• Run installation: 

  eb ./ebfiles_repo/IMAS/IMAS-3.42.0-2024.08.1-foss-2023b.eb --allow-modules-tool-mismatch --robot=<path_to_ebfiles_repo>  
  eb ./ebfiles_repo/IMAS/IMAS-3.42.0-2024.08.1-intel-2023b.eb --allow-modules-tool-mismatch --robot=<path_to_ebfiles_repo> 

How to use IMAS

After installation you can load IMAS  modules i.e:

module use {prefix}/modules/all
module load IMAS-AL-Cpp/5.2.0dev-foss-2020b-DD-3.40.1
module load IMAS-AL-Fortran/5.2.0dev-foss-2020b-DD-3.40.1
module load IMAS-AL-Java/5.2.0dev-foss-2020b-Java-11-DD-3.40.1
module load IMAS-AL-Python/5.2.0dev-foss-2020b-DD-3.40.1

How it works

• eb - EasyBuild

• IMAS-AL-Cpp/IMAS-AL-Cpp-5.2.0dev-foss-2020b-DD-3.40.1.eb - path to EasyBuild configuration file

• --allow-modules-tool-mismatch - flag   BP: Probably unnecessary - to be checked
• --robot - paths to configuration files

--installpath-modules=/gw/swimas/easybuild/etc --use-existing-modules --ignore-checksums

Installation of the 'production' software

To install the 'production' version of IMAS ecosystem, a few extra options have to be added to point out EB to the dedicated public directory /gw/swimas/easybuild  

--prefix='/gw/swimas/easybuild/opt' 
--installpath-modules='/gw/swimas/easybuild/etc'

# TO BE DISCUSSED: Accessing sources while debugging. Should 'source' directory be public, as well? 

# following modules must be installed 
# pkgconfig
# Cmake

imasenv module

Adding software components to existing installation

• Set  MODULEPATH system variable 

   export MODULEPATH=/gw/swimas/easybuild/etc/all

• Add --use-existing-modules  switch to eb  command

Useful tips

Installing dependencies as hidden modules

--hide-deps

Change the default tmp directory

Easybuild produces a lot of temporary content (logs, intermediate build files, etc)

The default /tmp  dir on the GW offers quite small capacity. Once tmp is full, system cannot work properly.

It is strongly recommended to change the default /tmp  to a directory in $ITMWORK 

tmpdir=$ITMWORK/tmp

export TMP=${tmpdir}
export TEMP=${tmpdir}
export TMPDIR=${tmpdir}

Run a massive builds on the worker nodes

# Run 'interactive' batch job using SLURM 'srun' command
# -p gw                  : 'gw' queue offering the longer walltime than the default 'gwdebug'
# --ntasks-per-node=48   : number of cores requested (1 node provides up to 48 cores)
# --time=48:0:0          : walltime (documentation says that the default walltime in 'gw' queue is 48h but not explicitly specified is set to 1h - To Be Checked)
# --x11                  : X11 forwarding (optional)
# --pty /bin/bash        : pseudoterminal

bash> srun -p gw --ntasks-per-node=48 --time=48:0:0 --pty /bin/bash

Once resources are allocated, a new shell opens on the working node usually reporting file system error:

bash: /afs/eufus.eu/user/g/g2<user>/.bashrc: Permission denied

It means that the Kerberos token should be set/renewed, to allow access to the filesystem:

bash> kinit; aklog

Now .bash_profile  could be 'sourced' to enable user's startup settings:

bash> source .bash_profile

Now EasyBuild can be used to build a software.

Increasing the length of error/warning message

Easybuild shows only the first 300 characters of error/warning message and cuts (usually the most interesting) remaining part of the message.

The message length value is hardcoded and can be change only by 'hacking' the EB codes:

1. Open <easybuild virtual env>/lib/python3.10/site-packages/easybuild/framework/easyblock.py 
2. Look for value "300" (line 4295 in EB version 4.9.4) 
   easyblock.py

   except EasyBuildError as err:
           first_n = 300   # <=== HERE is the limit hardcoded 
           errormsg = "build failed (first %d chars): %s" % (first_n, err.msg[:first_n])  
           _log.warning(errormsg)
           result = False

3. Increase the limit to bigger value (e.g. 3000) or remove the message cutting err.msg[:first_n])  ==> err.msg 
4. Save the file    

Other

• If you have compilation problems try adding the --parallel=1  option
• If you have problem with Perl : 

  make[1]: Entering directory `/afs/gw/swimas/easybuild/opt/build/Perl/5.38.0/GCCcore-13.2.0/perl-5.38.0/utils'
  make[1]: Nothing to be done for `all'.
  make[1]: Leaving directory `/afs/gw/swimas/easybuild/opt/build/Perl/5.38.0/GCCcore-13.2.0/perl-5.38.0/utils'
  
          Everything is up to date. Type 'make test' to run test suite.
  ./perl -Ilib -I. installperl --destdir=
  mkdir /afs/.gw: Read-only file system at ./install_lib.pl line 146.
  make: *** [install-all] Error 30
   (at easybuild/tools/run.py:682 in parse_cmd_output)

  The reason may be the installation path starting with /afs . Try changing it.
• If you have problem with cairo  and old ttx you can link old ttx  to /usr/bin/true .
• Problem with IMAS modules → change easyblock to Bundle