How to install split IMAS  using EasyBuild

This tutorial shows how to install IMAS (after repository split) using EasyBuild. The installation in this tutorial will be performed on the Gateway cluster but it can be done anywhere.

Install EasyBuild

You should install EasyBuild using Python and venv (virtual environment).

module load itm-python/3.10.13
python3 -m venv venv
module purge
source venv/bin/activate
pip3 install easybuild

You can use a different version of Python. Remember to purge all unnecessary modules that can cause conflicts with new modules created by EasyBuild.

Configuring EasyBuild

The configuration is located in the file $HOME/.config/easybuild/config.cfg. Create this file if it doesn't exist.

[config]
prefix=/pfs/work/g2pbloch/imasenv_eb
modules-tool=EnvironmentModulesC
module-syntax=Tcl
[override]
insecure-download=True

Change the prefix to the path where do you want your installation.

Create a secret.txt  file in $HOME/.config/easybuild/ and paste the HTTPS ACCESS TOKEN  from ITER  repo (https://git.iter.org).

Authorization: Bearer <token>

Downloading configuration (EB) files

Clone these repositories and  specify the paths to them in the --robot  in the next step.

https://gitlab.eufus.psnc.pl/g2pbloch/easybuild-files 
https://gitlab.eufus.psnc.pl/g2pbloch/easybuild-imas-files 

# [Bartek Palak] Probably only the ITER repository of EB receipts is required (ssh://git@git.iter.org/imex/easybuild-easyconfigs.git) To be checked.

Installation of Data Dictionary (DD)

Run the DD installation (in the easybuild-imas-files directory):

eb Data-Dictionary/Data-Dictionary-3.40.1-GCCcore-10.2.0.eb --allow-modules-tool-mismatch --robot=/pfs/work/g2pbloch/test_instruction/easybuild-imas-files:/pfs/work/g2pbloch/test_instruction/eb_files_repo  --http-header-fields-urlpat="^https://git.iter.org::$HOME/.config/easybuild/secret.txt"

Remember to change the --robot and pass paths to your easybuild-files  and easybuild-imas-files  directories. 

Configuring paths in IMAS configuration (EB) files

After DD installation change path in this line in the HLI easyconfigs to your  {prefix}/software/Data-Dictionary/3.40.1-GCCcore-10.2.0/dd_3.40.1/

 preconfigopts = 'export IMAS_VERSION="0.0.0" &&  export IMAS_PREFIX="/pfs/work/g2pbloch/test_instruction/software/Data-Dictionary/3.40.1-GCCcore-10.2.0/dd_3.40.1/" &&' 

In Python configuration files IMAS-AL-Python, IMAS-AL-Python-Bindings  you have to change path  to python to $prefix/software/Python/3.8.6-GCCcore-10.2.0/ 

Installation of IMAS HLIs - 5.2.0dev

eb IMAS-AL-Cpp/IMAS-AL-Cpp-5.2.0dev-foss-2020b-DD-3.40.1.eb --allow-modules-tool-mismatch --robot=/pfs/work/g2pbloch/test_instruction/easybuild-imas-files:/pfs/work/g2pbloch/test_instruction/eb_files_repo --http-header-fields-urlpat="^https://git.iter.org::$HOME/.config/easybuild/secret.txt"
eb IMAS-AL-Fortran/IMAS-AL-Fortran-5.2.0dev-foss-2020b-DD-3.40.1.eb  --allow-modules-tool-mismatch --robot=/pfs/work/g2pbloch/test_instruction/easybuild-imas-files:/pfs/work/g2pbloch/test_instruction/eb_files_repo --http-header-fields-urlpat="^https://git.iter.org::$HOME/.config/easybuild/secret.txt" 
eb IMAS-AL-Java/IMAS-AL-Java-5.2.0dev-foss-2020b-Java-11-DD-3.40.1.eb  --allow-modules-tool-mismatch --robot=/pfs/work/g2pbloch/test_instruction/easybuild-imas-files:/pfs/work/g2pbloch/test_instruction/eb_files_repo --http-header-fields-urlpat="^https://git.iter.org::$HOME/.config/easybuild/secret.txt"   
eb IMAS-AL-Python/IMAS-AL-Python-5.2.0dev-foss-2020b-DD-3.40.1.eb  --allow-modules-tool-mismatch --robot=/pfs/work/g2pbloch/test_instruction/easybuild-imas-files:/pfs/work/g2pbloch/test_instruction/eb_files_repo --http-header-fields-urlpat="^https://git.iter.org::$HOME/.config/easybuild/secret.txt"

Remember to change the --robot and pass paths to your easybuild-files  and easybuild-imas-files  directories. 


Installation of IMAS/3.42.0-2024.08.1

How to use IMAS

After installation you can load IMAS  modules i.e:

module use {prefix}/modules/all
module load IMAS-AL-Cpp/5.2.0dev-foss-2020b-DD-3.40.1
module load IMAS-AL-Fortran/5.2.0dev-foss-2020b-DD-3.40.1
module load IMAS-AL-Java/5.2.0dev-foss-2020b-Java-11-DD-3.40.1
module load IMAS-AL-Python/5.2.0dev-foss-2020b-DD-3.40.1

How it works


Installation of the 'production' version

To install the 'production' version of IMAS ecosystem, a few extra options have to be added to point out EB to the dedicated public directory /gw/swimas/easybuild  


--prefix='/gw/swimas/easybuild/opt' 
--installpath-modules='/gw/swimas/easybuild/etc'

# TO BE DISCUSSED: Accessing sources while debugging. Should 'source' directory be public, as well?


Adding software components to existing installation



Useful tips

Purge loaded modules before using Easybuild

EasyBuild requires no modules to be loaded ! 

Please execute:

module purge

 

Change the default tmp directory

Easybuild produces a lot of temporary content (logs, intermediate build files, etc)

The default /tmp  dir on the GW offers quite small capacity. Once tmp is full, system cannot work properly.

It is strongly recommended to change the default /tmp  to a directory in $ITMWORK 


tmpdir=$ITMWORK/tmp

export TMP=${tmpdir}
export TEMP=${tmpdir}
export TMPDIR=${tmpdir}

Increasing the length of error/warning message

Easybuild shows only the first 300 characters of error/warning message and cuts (usually the most interesting) remaining part of the message.

The message length value is hardcoded and can be change only by 'hacking' the EB codes:

  1. Open <easybuild virtual env>/lib/python3.10/site-packages/easybuild/framework/easyblock.py 
  2. Look for value "300" (line 4295 in EB version 
    except EasyBuildError as err:
        first_n = 300   # <=== HERE is the limit hardcoded 
        errormsg = "build failed (first %d chars): %s" % (first_n, err.msg[:first_n])  
        _log.warning(errormsg)
        result = False
  3. Increase the limit to bigger value (e.g. 3000) or remove the message cutting err.msg[:first_n])  ==> err.msg 
  4. Save the file    


Other