Make every datagen MLflow run self-describing - #319
Conversation
The model identity of a data-generation run was previously recoverable only
from experiment-name conventions ({model}-data-generation), which hold only
for orchestrated submissions. Downstream consumers (LANfactory training,
the network registry) need it on the run itself.
- Log identity params on every run: model, generator_approach, n_files,
config_sha256 (of the YAML bytes), ssm_simulators_version
- Tags: schema_version=1, phase=datagen, plus slurm_job_id /
slurm_array_job_id / slurm_array_task_id picked up from the environment
- New repeatable `--mlflow-tag KEY=VALUE` so an orchestrator can stamp e.g.
generation_batch_id without this CLI knowing about it; caller tags win on
collision; malformed tags fail fast even when MLflow is inactive
- Persist generator_approach into data_config: previously it only *selected*
the config template and was dropped, so it never reached the training-data
pickles (which embed data_config as generator_config) or any MLflow record
Schema documented in HSSMSpine _docs/mlflow-schema.md (forthcoming).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four issues from adversarial review, each reproduced against a live store: - Malformed --mlflow-tag left a junk empty run: parsing now happens before setup_mlflow starts the run, matching the fail-fast intent - --mlflow-tag could silently overwrite schema_version/phase, making runs invisible to schema-filtered consumers: reserved keys now rejected at parse - Dry runs were stamped phase=datagen with n_files=<requested> despite zero files: they now get phase=datagen_dry_run - config_sha256 was omitted whenever the packaged default config drove the run: load_config now computes the sha itself (the default file only exists inside its as_file context) and returns it alongside the config Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Warning Review limit reached
Next review available in: 34 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests.
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 1 file with indirect coverage changes 🚀 New features to boost your workflow:
|
1 participant
What
First PR of the HSSM ecosystem's "MLflow as database of record" workstream (PR-0.1 of the plan coordinated in HSSMSpine). Every
generateinvocation now logs enough identity onto its MLflow run that a catalog can answer "what data is this?" without unpickling artifacts or decoding experiment-name conventions:model,generator_approach,n_files,config_sha256,ssm_simulators_version,data_output_folderschema_version=1,phase=datagen(ordatagen_dry_run), SLURM ids from env when present--mlflow-tag KEY=VALUEoption so an orchestrator can stamp batch-level tags (e.g. a generation-batch id) without this repo knowing about them; reserved keys (schema_version,phase) are rejected at parse time, and parsing happens before MLflow setup so a malformed tag can't leave a junk run behindgenerator_approachis now persisted intodata_configinside the training-data pickles, so downstream consumers (LANfactory) see provenance tooconfig_sha256is computed insideload_config(the packaged default config only exists inside theas_filecontext, so hashing outside it was impossible)Ecosystem impact
Downstream: LANfactory's training runs (companion PR lnccbrown/LANfactory) log a matching schema so datagen ↔ training runs can be joined; LAN_pipeline_minimal will consume
--mlflow-tagfor batch stamping. No API breaks — all changes are additive.Review provenance
This diff went through an adversarial multi-agent review before opening (12 findings across the three Phase-0 branches, all addressed or empirically refuted). Findings fixed here: junk-run-on-malformed-tag, reserved-tag overwrite, dry-run phase pollution, missing sha for the packaged default config.
Commands run
🤖 Generated with Claude Code