Skip to content

Search- and selection-driven curation (curate --ids/--from, search --save) - #34

Merged
arpitg1304 merged 1 commit into
mainfrom
feat/curate-explicit-selections
Jul 19, 2026
Merged

Search- and selection-driven curation (curate --ids/--from, search --save)#34
arpitg1304 merged 1 commit into
mainfrom
feat/curate-explicit-selections

Conversation

@arpitg1304

Copy link
Copy Markdown
Owner

What

Closes the seam between semantic search / visual review and curation. forge curate was SQL---where only, so you couldn't curate "the top-K of a search" or a hand-picked set from Studio — those are selections (ranked lists / picked sets), not predicates. Now the selection can also be an explicit --ids list or a --from <selection.json> file.

# search → save a selection → curate it (provenance recorded)
forge search "regrasp after a failed pick" -c ./cat --top 20 --save sel.json
forge curate -c ./cat --from sel.json --label approved

# or an explicit id list
forge curate -c ./cat --ids e1,e2,e3 --label approved

Changes

  • curate() gains an ids selector — composes with --where (intersection) and the dedup policy, and is filtered to episodes that actually exist, so a typo/stale id never creates a dangling label.
  • forge search --save sel.json writes the result ids + provenance ({episode_ids, source}); forge curate --from sel.json applies them and records the source in labeled_by for auditability.
  • Forge Studio's dedup tab gains a "Download decisions" button that exports the same selection file (the rejected picks) — so keep/reject review can write back via forge curate --from, instead of being copy-only.

Why this shape

The two real requests ("top-K of a search", "what I picked in Studio") are explicit selections, so an id/selection-file path fits them natively and unblocks both in one small change. A SQL similarity predicate (the alternative) is a filter, not a ranker, still needs the model outside SQL for text, and does nothing for the hand-pick case — so it's a bigger lift that solves less. Provenance on the label (labeled_by = the search/source) keeps curation auditable despite not being a self-documenting SQL rule.

Doing this before Phase 4 (snapshots) matters: snapshots filter on curation_labels regardless of how they were produced, so search/Studio-curated sets become exportable too, not just SQL-predicate ones.

Testing

5 new tests (curate by ids, id-filtering to existing episodes, ids ∩ where, and the search --save → curate --from round-trip with provenance); Studio template JS re-validated. No regressions.

🤖 Generated with Claude Code

…from, search --save)

Close the seam between semantic search / visual review and curation: `forge
curate` was SQL-`--where` only, so you couldn't curate "the top-K of a search"
or a hand-picked set from Studio. Now the selection can also be an explicit
`--ids` list or a `--from <selection.json>` file.

- curate() gains an `ids` selector — composes with `--where` (intersection) and
  the dedup policy, and is filtered to episodes that actually exist so a typo
  never creates a dangling label.
- `forge search --save sel.json` writes the result ids + provenance
  ({episode_ids, source}); `forge curate --from sel.json` applies them and
  records the source in `labeled_by`. `--ids e1,e2` works too.
- Forge Studio's dedup tab gains a "Download decisions" button that exports the
  same selection file (the rejected picks), so keep/reject review can write back
  via `forge curate --from` instead of being copy-only.

Tests: curate by ids, id-filtering to existing episodes, ids∩where, and the
search --save → curate --from round-trip with provenance. No existing tests
change. Docs: README data-engine section, forge/catalog/README, CHANGELOG.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@arpitg1304
arpitg1304 merged commit df94153 into main Jul 19, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant