Provide option to use EESSI modules instead of init script

[casparvl]

Make run_reframe.sh use EESSI modules. Also: change how defaults are assigned in run_reframe.sh by using the ${X:-default_X} construct, which is more compact than what we currently have.

[casparvl]

We will probably replace this by an alternative approach using ReFrame environments. Specifically, if you set:

'environs': ['default', 'EESSI-2023.06', 'EESSI-2025.06'],
...
{
            'name': 'EESSI-2023.06',
            'modules': ['EESSI/2023.06']
        },
        {
            'name': 'EESSI-2025.06',
            'modules': ['EESSI/2025.06']
        },
...
# Add each partition name as feature programmatically
# Note that we don't retain the colon to separate - only normal characters are allowed in a feature
for system in site_configuration.get('systems', []):
    for partition in system.get('partitions', []):
        features = partition.get('features', [])
        if 'name' in partition and 'name' in system:
            new_feature = f"{system['name']}{partition['name']}"
            partition['features'] = partition['features'] + [new_feature]

in the ReFrame config, and then use the upstream find_modules function to do:

from reframe.utility import find_modules
...

class EESSI_XYZ(rfm.RunOnlyRegressionTest):
...
    module_info = parameter(find_modules('SciPy-bundle'))
...
    @run_after('init', always_last=True)
    def apply_module_info(self):
        s, e, m = self.module_info
        # Get the system:partition from s, and strip the colon character as those are not allowed in features
        self.valid_systems = [f"+{s.replace(':','')}"]
        self.valid_prog_environs = [e]
        self.modules = [m]
        self.tags = {self.scale}

The regular hooks can then add additional features, and those will be repsected by reframe. E.g. if I hardcode:

self.valid_systems = [f"+{s.replace(':','')} +gpu"]

the above only runs on the subset of partitions that are marked with the "gpu" feature.

[casparvl]

Merged #326 into this, and succesfully ran with

EESSI_TESTSUITE_URL=https://github.com/casparvl/test-suite.git EESSI_TESTSUITE_BRANCH=support_multiple_eessi_environments EESSI_CI_SYSTEM_NAME=surf_snellius ./run_reframe.sh

[casparvl]

Ok, I need to dive back into this. On our AWS cluster (with no native module command):

[  FAILED  ] Ran 168/168 test case(s) from 168 check(s) (168 failure(s), 0 skipped, 0 aborted)
[==========] Finished on Wed Jun 17 15:27:16 2026+0000
============================= SUMMARY OF FAILURES ==============================
--------------------------------------------------------------------------------
FAILURE INFO for EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('Magic_Castle:x86_64-generic-16c-30gb', 'EESSI-2023.06', 'GROMACS/2024.1-foss-2023b') (run: 1/1)
  * Description: GROMACS HECBioSim/Crambin benchmark (NB: cpu)
  * System partition: Magic_Castle
  * Environment: EESSI-2023.06
  * Test file: /home/casparvl/rfm.SLkKKtHSA0/test-suite/eessi/testsuite/tests/apps/gromacs.py
  * Stage directory: /home/casparvl/reframe_CI_runs/stage/Magic_Castle/x86_64-generic-16c-30gb/EESSI-2023.06/EESSI_GROMACS_0f2f0be3
  * Node list: x86-64-generic-node1
  * Job type: batch job (id=167560)
  * Dependencies (conceptual): []
  * Dependencies (actual): []
  * Maintainers: []
  * Failing phase: sanity
  * Rerun with '-n /0f2f0be3 -p EESSI-2023.06 --system Magic_Castle -r'
  * Reason: sanity error: md.log: No such file or directory
--- rfm_job.out (last 10 lines) ---
EESSI_CVMFS_REPO:
EESSI_SOFTWARE_SUBDIR:
FULL_MODULEPATH: /var/spool/slurmd/job167560/slurm_script: line 31: module: command not found
--- rfm_job.out ---
--- rfm_job.err (last 10 lines) ---
PROCESS BINDING WARNING: hwloc not available, skipping process binding check
/var/spool/slurmd/job167560/slurm_script: line 28: mpirun: command not found
--- rfm_job.err ---

The job in the staging dir is:

#!/bin/bash
#SBATCH --job-name="rfm_EESSI_GROMACS_0f2f0be3"
#SBATCH --ntasks=8
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=2
#SBATCH --output=rfm_job.out
#SBATCH --error=rfm_job.err
#SBATCH --time=0:30:0
#SBATCH --nodes=1
#SBATCH --partition=x86-64-generic-node
#SBATCH --export=NONE
#SBATCH --mem=28610M
source /cvmfs/software.eessi.io/2025.06/init/lmod/bash
module unload EESSI
export SLURM_EXPORT_ENV=ALL
module load EESSI/2023.06
module load GROMACS/2024.1-foss-2023b
export OMP_NUM_THREADS=2
export I_MPI_PIN_CELL=core
export I_MPI_PIN_DOMAIN=1:compact
export I_MPI_DEBUG=4
curl -LJO https://github.com/victorusu/GROMACS_Benchmark_Suite/raw/1.0.0/HECBioSim/Crambin/benchmark.tpr
if command -v hwloc-calc >/dev/null; then
mpirun -np 8 --map-by slot:PE=1 --report-bindings /home/casparvl/rfm.SLkKKtHSA0/test-suite/eessi/testsuite/get_process_binding.sh | tee /dev/stderr | /home/casparvl/rfm.SLkKKtHSA0/test-suite/eessi/testsuite/check_process_binding.py --cpus-per-proc 2 --procs 8 --nodes 1
else
echo 'PROCESS BINDING WARNING: hwloc not available, skipping process binding check' >/dev/stderr
fi
mpirun -np 8 --map-by slot:PE=1 --report-bindings gmx_mpi mdrun -dlb yes -npme -1 -nb cpu -s benchmark.tpr -ntomp 2
echo "EESSI_CVMFS_REPO: $EESSI_CVMFS_REPO"
echo "EESSI_SOFTWARE_SUBDIR: $EESSI_SOFTWARE_SUBDIR"
echo "FULL_MODULEPATH: $(module --location show GROMACS/2024.1-foss-2023b 2>&1)"

It seems that this doens't have the desired effect. Will have to dive into it, but no time now...

[casparvl]

Ok, the issue was a missing versions in the path to the init script in the aws_mc.py config file. Fixed that now. Running with:

[casparvl@login1 CI]$ REFRAME_ARGS='-t CI -t 1_node --system=Magic_Castle:x86_64-generic-16c-30gb -n /f7cb5d03' EESSI_TESTSUITE_URL=https://github.com/casparvl/test-suite.git EESSI_TESTSUITE_BRANCH=make_run_reframe_loop_eessi_versions EESSI_CI_SYSTEM_NAME=aws_mc ./run_reframe.sh

Succeeded:

[ReFrame Setup]
  version:           4.8.1
  command:           '/home/casparvl/rfm.AdVPC1AflS/reframe_venv/bin/reframe -t CI -t 1_node --system=Magic_Castle:x86_64-generic-16c-30gb -n /f7cb5d03 --run --setvar EESSI_CONFIGS_URL=https://github.com/casparvl/test-suite.git --setvar EESSI_CONFIGS_BRANCH=make_run_reframe_loop_eessi_versions'
  launched by:       casparvl@login1.int.aws-rocky88-202310.eessi.io
  working directory: '/home/casparvl/rfm.AdVPC1AflS'
  settings files:    '<builtin>', '/home/casparvl/rfm.AdVPC1AflS/configs/config/aws_mc.py'
  selected system:   'Magic_Castle'
  check search path: (R) '/home/casparvl/rfm.AdVPC1AflS/test-suite/eessi/testsuite/tests/apps/gromacs.py'
  stage directory:   '/home/casparvl/reframe_CI_runs/stage'
  output directory:  '/home/casparvl/reframe_CI_runs/output'
  log files:         '/home/casparvl/reframe_CI_runs/logs/reframe_20260701_141133.log'
  results database:  [off] '/home/casparvl/.reframe/reports/results.db'

[==========] Running 1 check(s)
[==========] Started on Wed Jul  1 14:12:00 2026+0000

[----------] start processing checks
[ RUN      ] EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('Magic_Castle:x86_64-generic-16c-30gb', 'EESSI-2025.06', 'GROMACS/2026.2-foss-2025b') /f7cb5d03 @Magic_Castle:x86_64-generic-16c-30gb+EESSI-2025.06
WARNING: PROCESS BINDING WARNING: processes with cores shared by processing units, indicating hyperthreading: Counter({2: 8}),
[       OK ] (1/1) EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('Magic_Castle:x86_64-generic-16c-30gb', 'EESSI-2025.06', 'GROMACS/2026.2-foss-2025b') /f7cb5d03 @Magic_Castle:x86_64-generic-16c-30gb+EESSI-2025.06
P: perf: 52.043 ns/day (r:0, l:None, u:None)
[----------] all spawned checks have finished

[  PASSED  ] Ran 1/1 test case(s) from 1 check(s) (0 failure(s), 0 skipped, 0 aborted)
[==========] Finished on Wed Jul  1 14:15:09 2026+0000
Log file(s) saved in '/home/casparvl/reframe_CI_runs/logs/reframe_20260701_141133.log'