Skip to content

bug: SlurmJob fluent setters write wrong kwargs keys — cpus/memory/time silently dropped #10

Description

@prabindersinghh

Bug Report

File: metaflow_extensions/slurm_ext/plugins/slurm/slurm_job.py, lines 105, 109, 113

Summary

The cpus_per_task(), memory(), and run_time_limit() fluent setter
methods write incorrect keys into self.kwargs. However,
create_slurm_script() reads different keys. The mismatch means all
three values are silently dropped — every job runs with default CPUs,
memory, and time limit regardless of what the user specified.

Setter method Key written Key read by create_slurm_script()
cpus_per_task() "cpus-per-task" (hyphen) "cpus_per_task" (underscore)
memory() "mem" "memory"
run_time_limit() "time" "run_time_limit"

Reproducer

job = SlurmJob({"job_name": "test"}, None)
job.cpus_per_task(4).memory("8G").run_time_limit(60)
script = job.create_slurm_script("python test.py")

# All three directives are missing from the generated script
assert "#SBATCH --cpus-per-task=4" in script  # FAILS
assert "#SBATCH --mem=8G" in script           # FAILS
assert "#SBATCH --time=60" in script          # FAILS

Expected behavior

The generated sbatch script contains the correct #SBATCH directives
for CPUs, memory, and time limit when set via the fluent API.

Actual behavior

All three directives are silently absent. Jobs run with cluster
defaults regardless of user-specified values.

Root cause

The setter methods and create_slurm_script() use different key names.
The fix is a 3-line rename to align the setter keys with what
create_slurm_script() reads.

Environment

  • metaflow-slurm: latest main
  • Python: 3.11+

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions