Table of contents

Why should I use Docker containers with IMAS in a first place

If you find it hard to have your local IMAS installation, it might be you find yourself in one of these situations:

  • you have a computing cluster but you don't have root permissions to install new software,
  • you need just bare bones IMAS installation, with basic components, but you don't have a computing machine and manpower to install it,
  • you don't have time to perform regular upgrades of your IMAS installation,
  • you want to utilise Cloud resources.

Indeed, having a local IMAS installation may be a challenge. However, there is a solution. In case you need your private IMAS release, with slightly limited functionality, a Docker based IMAS release if the way for you to go.

We have successfully used this approach across various projects and were able to deliver solutions from embedding single physical code to complex, micro-services based, server solutions. We were able to provide UDA server and UDA client based on IMAS inside Docker, we have released SimDB installations based on IMAS inside Docker, finally, we provide CI/CD based tests at https://gitlab.eufus.psnc.pl that are based on IMAS inside Docker. And all of that thanks to Docker images with IMAS inside.

You can benefit from this solution, and it is extremely simple. All you have to do is to create your Dockerfile such way it is based on ours

Create your own Docker image based on IMAS

> cat ~/mycontainer/Dockerfile

FROM gitlab.eufus.psnc.pl:5050/containerization/imas/imas-installer/al-iwrap:DD-4.0.0_AL-5.4.1_IWRAP-0.10.0-UDA-2.8.1-UDA-PLUGIN-1.5.0
...
...


That's it. You have a Docker image with IMAS inside. Feel free to add anything you like on top of it.

Login required

Please note that accessing gitlab.eufus.psnc.pl registry requires login:

docker login gitlab.eufus.psnc.pl:5050

You will need EUROfusion Software Stack login (Getting access to JIRA/Confluence/Gitlab/Mattermost). You will also need access token (password replacement). You can learn how to create and use access tokens here: Generating access tokens.

What options are available when it comes to Docker images with IMAS

You have a number of choices when it comes to deciding what you want to use. Everything depends on your particular use case.

# Just base components - foundation for other images
base image               - gitlab.eufus.psnc.pl:5050/containerization/imas/imas-installer/base


# Everything in base + al core components
AL core image            - gitlab.eufus.psnc.pl:5050/containerization/imas/imas-installer/al-core


# Everything in AL core + components required for a specific language developments
AL Java image            - gitlab.eufus.psnc.pl:5050/containerization/imas/imas-installer/al-java
AL Python image          - gitlab.eufus.psnc.pl:5050/containerization/imas/imas-installer/al-python
AL CPP image             - gitlab.eufus.psnc.pl:5050/containerization/imas/imas-installer/al-cpp
AL Fortran image         - gitlab.eufus.psnc.pl:5050/containerization/imas/imas-installer/al-fortran


# Everything in AL core image + all AL interfaces
AL image                 - gitlab.eufus.psnc.pl:5050/containerization/imas/imas-installer/al


# Everything in AL image + FC2K application
FC2K image               - gitlab.eufus.psnc.pl:5050/containerization/imas/imas-installer/al-fc2k


# Everything in AL image + iWrap application
iWrap image              - gitlab.eufus.psnc.pl:5050/containerization/imas/imas-installer/al-iwrap


# Everything in AL image + SimDB server
SimDB image              - gitlab.eufus.psnc.pl:5050/containerization/imas/imas-installer/al-simdb

List of features inside various images

Successful projects based on Docker images with IMAS

Running IMAS based codes on your laptop

You can run IMAS based codes using Docker images with IMAS. All you have to do is to get the image and run the container (some changes inside container might be needed, it contains only bare IMAS installation).

Source code: https://gitlab.eufus.psnc.pl/containerization/imas/imas-installer

HOWTO: https://gitlab.eufus.psnc.pl/containerization/imas/uda-demo/-/blob/master/README.md

Simulation Catalogue - Data Management Plan project

Simulation Catalogue components responsible for data ingestion are based on Docker with IMAS. This way it is possible to easily deploy micro-services based solution within cloud based environment.

Source code: https://gitlab.eufus.psnc.pl/dmp

HOWTO: https://gitlab.eufus.psnc.pl/dmp/installer/-/blob/develop/README.md 

UDA (Universal Data Access) server and client

UDA (Universal Data Access) is a client-server infrastructure designed to enable the reading and passing of heterogeneous data format back to a client in a universal data format (source: https://ukaea.github.io/UDA/). With Docker based approach, installation procedure was heavily simplified and allows to install UDA wherever you want it installed. We have prepared a simple Hello world! demo with both UDA Server and UDA Client - you can run it on your laptop.

git clone git@gitlab.eufus.psnc.pl:containerization/imas/uda-demo.git
cd uda-demo
docker logout https://gitlab.eufus.psnc.pl:5050
docker login https://gitlab.eufus.psnc.pl:5050

With UDA Server and UDA Client images you can establish connection between virtually any machines and transfer data directly from UDA Server inside your IMAS compatible code

Source code: https://gitlab.eufus.psnc.pl/containerization/imas/uda-demo

HOWTO: https://gitlab.eufus.psnc.pl/containerization/imas/uda-demo/-/blob/master/README.md

JINTRAC codes inside IMAS based Docker images

We were able to adapt Docker based environment for JINTRAC project into IMAS AL4, and we have demonstrated that the very same approach can be used to run codes inside IMAS AL5 environment.

Source code: https://gitlab.eufus.psnc.pl/jintrac-mirror/jintrac/container_registry/39

Hello world with IMAS based Docker

For one of our tutorials (live-demo) we have prepared extremely simple use case: "Hello world" with IMAS based Docker.

This example illustrates how one can:

  • create data using IMAS based Docker container
  • modify data using IMAS based Docker container
  • read data using IMAS based Docker container

Examples are provided using Python and are really simple. Just a basics.

Source code: https://gitlab.eufus.psnc.pl/g2awisz/testing_repository/-/tree/Docker/multi-container-usage

HOWTO: https://gitlab.eufus.psnc.pl/g2awisz/testing_repository/-/blob/Docker/multi-container-usage/README.md

SimDB inside Docker

We have successfully embedded and deployed SimDB server based completely on Docker. By embedding SimDB inside Docker container we are free to use virtually any hardware environment for deployment. All we need is a Docker compatible setup on target machine.

Source code: https://gitlab.eufus.psnc.pl/g2dfigat/simdb-docker/-/tree/develop

HOWTO: https://gitlab.eufus.psnc.pl/g2dfigat/simdb-docker/-/blob/develop/README.md

Example of running IMAS based Docker using Podman

It is perfectly possible to run IMAS based Docker containers using Podman. This way, you can run containers inside root-less environment. You still need Podman, though.

Source code: https://gitlab.eufus.psnc.pl/-/snippets/38

Example of running IMAS based Docker using Apptainer

It is perfectly possible to run IMAS based Docker containers using Apptainer. This way, you can run containers inside root-less environment. You still need Apptainer, though.

Source code: https://gitlab.eufus.psnc.pl/-/snippets/39

Example of running MUSLE3 based workflow using IMAS based Docker containers inside Apptainer

We have demonstrated how MUSLE3 workflow can be run from Docker images and spawn another containers with actors that communicate via MUSCLE3 manager. These codes can be run completely root-less thanks to Apptainer.

Source code: https://gitlab.eufus.psnc.pl/g2jwasik/muscle3-contenerized-test.git

Running CI/CD pipelines using Docker containers with IMAS

If you have wondered whether it is possible to use Docker based IMAS for running regular tests, the answer is: YES. This is perfectly possible. In fact ACH-04 team benefits from this approach every day, whenever we push changes to our code base. We test various projects using pipelines composed of layers of Docker containers.

There is a dedicated tutorial on how to use Docker inside CI/CD plans. You can find it here: 05. ACH GitLab CI/CD - Docker runner (running inside Docker container) There is even a step by step video tutorial you can find on Youtube.

 

How can I get some help with making my code containers ready

The best way to get support is to reach out to ACH-04 through E-TASC activities. If you have reserved time in your project for adapting code to IMAS, feel free to reach out to us and we can discuss your case and suggest the best approach for your code. We occasionally provide tutorial sessions related to running code inside the Docker - make sure to attend these whenever they are available.

  • No labels

The scientific work is published for the realization of the international project co-financed by Polish Ministry of Science and Higher Education in 2019 from financial resources of the program entitled "PMW"; Agreement No. 5040/H2020/Euratom/2019/2.

This work has been carried out within the framework of the EUROfusion Consortium and has received funding from the Euratom research and training programme 2014–2020 under grant agreement No 633053.

This work has been carried out within the framework of the EUROfusion Consortium, funded by the European Union via the Euratom Research and Training Programme (Grant Agreement No. 101052200-EUROfusion). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Commission. Neither the European Union nor the European Commission can be held responsible for them.

The project activities are co-financed by Polish Ministry of Science and Higher Education from financial resources of the programme entitled "PMW" in 2023, 2024.

The views and opinions expressed herein do not necessarily reflect those of the European Commission or ITER