WORK IN PROGRESS!


Introduction

This tutorial shows how to install IMAS ecosystem using EasyBuild. The installation will be performed on the Gateway cluster but it can be done anywhere.

Prerequisites

Before you start, make sure that:


Software installation receipts

Sources of the receipts (aka 'easyconfig' files):

Downloading configuration (EB) files

SDCC: /work/imas/opt/EasyBuild/ebfiles_repo/ 

rsync -av --delete -e "ssh -J <user>@sdcc-login.iter.org"  <user>@sdcc-login02.iter.org:/work/imas/opt/EasyBuild/ebfiles_repo/ ./all_iter/


Clone these repositories and  specify the paths to them in the --robot  in the next step.

TBD

Remember to change the --robot and pass paths to your  and TBD  directories. 


Installation of IMAS HLIs - 5.2.0dev

eb IMAS-AL-Cpp/IMAS-AL-Cpp-5.2.0dev-foss-2020b-DD-3.40.1.eb --allow-modules-tool-mismatch --robot=/pfs/work/g2pbloch/test_instruction/easybuild-imas-files:/pfs/work/g2pbloch/test_instruction/eb_files_repo 
eb IMAS-AL-Fortran/IMAS-AL-Fortran-5.2.0dev-foss-2020b-DD-3.40.1.eb  --allow-modules-tool-mismatch --robot=/pfs/work/g2pbloch/test_instruction/easybuild-imas-files:/pfs/work/g2pbloch/test_instruction/eb_files_repo 
eb IMAS-AL-Java/IMAS-AL-Java-5.2.0dev-foss-2020b-Java-11-DD-3.40.1.eb  --allow-modules-tool-mismatch --robot=/pfs/work/g2pbloch/test_instruction/easybuild-imas-files:/pfs/work/g2pbloch/test_instruction/eb_files_repo 
eb IMAS-AL-Python/IMAS-AL-Python-5.2.0dev-foss-2020b-DD-3.40.1.eb  --allow-modules-tool-mismatch --robot=/pfs/work/g2pbloch/test_instruction/easybuild-imas-files:/pfs/work/g2pbloch/test_instruction/eb_files_repo

Remember to change the --robot and pass paths to your easybuild-files  and easybuild-imas-files  directories. 


Installation of IMAS

IMAS bundle

imasenv module

How to use IMAS

After installation you can load IMAS  modules i.e:

module use {prefix}/modules/all
module load IMAS-AL-Cpp/5.2.0dev-foss-2020b-DD-3.40.1
module load IMAS-AL-Fortran/5.2.0dev-foss-2020b-DD-3.40.1
module load IMAS-AL-Java/5.2.0dev-foss-2020b-Java-11-DD-3.40.1
module load IMAS-AL-Python/5.2.0dev-foss-2020b-DD-3.40.1

How it works


Installation of the 'production' version

To install the 'production' version of IMAS ecosystem, a few extra options have to be added to point out EB to the dedicated public directory /gw/swimas/easybuild  


--prefix='/gw/swimas/easybuild/opt' 
--installpath-modules='/gw/swimas/easybuild/etc'

# TO BE DISCUSSED: Accessing sources while debugging. Should 'source' directory be public, as well? 

# following modules must be installed 
# pkgconfig
# Cmake


Adding software components to existing installation


Useful tips

Installing dependencies as hidden modules

TO BE CHECKED !!!!!

{{--hide-deps}}


Purge loaded modules before using Easybuild

EasyBuild requires no modules to be loaded ! 

Please execute:

module purge

 

Change the default tmp directory 

Easybuild produces a lot of temporary content (logs, intermediate build files, etc)

The default /tmp  dir on the GW offers quite small capacity. Once tmp is full, system cannot work properly.

It is strongly recommended to change the default /tmp  to a directory in $ITMWORK 


tmpdir=$ITMWORK/tmp

export TMP=${tmpdir}
export TEMP=${tmpdir}
export TMPDIR=${tmpdir}

Run a massive builds on the worker nodes


# Run 'interactive' batch job using SLURM 'srun' command
# -p gw                  : 'gw' queue offering the longer walltime than the default 'gwdebug'
# --ntasks-per-node=48   : number of cores requested (1 node provides up to 48 cores)
# --time=48:0:0          : walltime (documentation says that the default walltime in 'gw' queue is 48h but not explicitly specified is set to 1h - To Be Checked)
# --x11                  : X11 forwarding (optional)
# --pty /bin/bash        : pseudoterminal

bash> srun -p gw --ntasks-per-node=48 --time=48:0:0 --pty /bin/bash


Once resources are allocated, a new shell opens on the working node usually reporting file system error:

bash: /afs/eufus.eu/user/g/g2<user>/.bashrc: Permission denied


It means that the Kerberos token should be set/renewed, to allow access to the filesystem:

bash> kinit; aklog


Now .bash_profile  could be 'sourced' to enable user's startup settings:

bash> source .bash_profile


Now EasyBuild can be used to build a software.


  • Add --parallel=48  to EB commands (on the working nodes EB doesn't recognise properly  a number of available cores and runs sequentially - To Be Checked)
  • EB has built-in feature that allows to submit build to the batch system (see this link) - To Be Checked


Increasing the length of error/warning message

Easybuild shows only the first 300 characters of error/warning message and cuts (usually the most interesting) remaining part of the message.

The message length value is hardcoded and can be change only by 'hacking' the EB codes:

  1. Open <easybuild virtual env>/lib/python3.10/site-packages/easybuild/framework/easyblock.py 
  2. Look for value "300" (line 4295 in EB version 4.9.4) 
    except EasyBuildError as err:
        first_n = 300   # <=== HERE is the limit hardcoded 
        errormsg = "build failed (first %d chars): %s" % (first_n, err.msg[:first_n])  
        _log.warning(errormsg)
        result = False
  3. Increase the limit to bigger value (e.g. 3000) or remove the message cutting err.msg[:first_n])  ==> err.msg 
  4. Save the file    


Other