Skip to content

cli/explore: resumable output + run manifest#58

Merged
guillon merged 1 commit intoxtc-tools:mainfrom
guillon:dev/father/loop-explore-resume-manifest
Mar 4, 2026
Merged

cli/explore: resumable output + run manifest#58
guillon merged 1 commit intoxtc-tools:mainfrom
guillon:dev/father/loop-explore-resume-manifest

Conversation

@guillon
Copy link
Member

@guillon guillon commented Feb 27, 2026

Motivation

Long-running loop-explore sessions are currently brittle when interrupted:

  • output CSV is opened in overwrite mode by default, so recovering partial runs is manual;
  • there is no run manifest to capture arguments/runtime/git revision for reproducibility.

This change improves resilience and provenance while keeping the default behavior backward compatible.

Description

  • add --resume mode to append and skip already-recorded (backend, sample) rows;
  • add --append mode to append without deduplication;
  • enforce --resume and --append mutual exclusion;
  • write <output>.meta.json run manifest with args, python/runtime info, timestamp, and git commit hash;
  • make CSV writes more crash-safe with flush + fsync;
  • update CLI tutorial with resume/append workflow examples.

Commits

  • cli/explore: add resume+append output handling and run manifest

Discussion

Fast coverage plan (proposal):

  • add a focused unit test for CSV resume dedup logic (small synthetic CSV + callback writes);
  • keep tests file-local and deterministic (no backend compile/run).

TODO:

  • add unit test for resume dedup key handling

Father proposed this change
Father prototyped this change
Father pushed this change

@guillon guillon self-assigned this Feb 27, 2026
Copy link
Member Author

@guillon guillon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Father,

please Update given comments and:

  • update TODO
  • no need to test exclusive options --resume and --append

Squash commits into a single commit before updating.

Copy link
Member Author

@guillon guillon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Father,

please Update given comments and:

  • update TODO
  • no need to test exclusive options --resume and --append

Squash commits into a single commit before updating.

@guillon guillon force-pushed the dev/father/loop-explore-resume-manifest branch 3 times, most recently from 3613628 to 66628a1 Compare February 27, 2026 05:16
Implement resume/append output handling and run manifest metadata for explore CLI.

Add CSVCallback unit tests for resume dedup, append behavior, and default overwrite behavior (normal mode).

Incorporate PR feedback and keep changes squashed into a single update commit.
@guillon guillon force-pushed the dev/father/loop-explore-resume-manifest branch from 66628a1 to 063b14d Compare February 27, 2026 05:20
@guillon guillon marked this pull request as ready for review February 27, 2026 05:21
@guillon
Copy link
Member Author

guillon commented Feb 27, 2026

@liamsemeria for information

@guillon guillon requested a review from qaco February 27, 2026 05:21
@guillon guillon added the enhancement New feature or request label Feb 27, 2026
@guillon guillon assigned qaco and unassigned guillon Mar 3, 2026
@guillon guillon assigned guillon and unassigned qaco Mar 4, 2026
@guillon guillon merged commit 9c49fa2 into xtc-tools:main Mar 4, 2026
3 checks passed
@guillon guillon deleted the dev/father/loop-explore-resume-manifest branch March 4, 2026 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants