Skip to content

fix: screen impossible transcript adoption candidates before SQLite part reads - #1231

Closed
aounakram wants to merge 2 commits into
Martian-Engineering:mainfrom
aounakram:fix/bootstrap-adoption-candidate-filter
Closed

aounakram wants to merge 2 commits into
Martian-Engineering:mainfrom
aounakram:fix/bootstrap-adoption-candidate-filter

Conversation

@aounakram

Copy link
Copy Markdown

fix: screen impossible transcript adoption candidates before SQLite part reads

Summary

In pre-turn bootstrap reconciliation, preserve the legacy adoption planner's one-to-one ambiguity rules while screening pairs that cannot match. The scan now skips a candidate when roles differ, or when identity hashes differ and the stored row has no possible externalized reference and the incoming row has no tool-call IDs for the redaction path. The externalization hint is deliberately case-insensitive and cached once per tail row. Already-anchored historical entries are still compared, because their repeated-content claims can make an unstamped row ambiguous.

This addresses the code-level performance path described in the companion Lossless Claw issue, #1230. It does not claim a complete measured fix for the reported 188–191-second production preparation interval, which has not been CPU-profiled.

Verification

  • Targeted tests: 59 passed across test/batch-dedup.test.ts, test/reconciliation-work.test.ts, test/transcript-anchor-trust-store.test.ts, test/reconcile-plan.test.ts, test/structured-anchor-identity.test.ts, and test/transcript-anchor-audit.test.ts. New tests cover a 1,001-entry history against a 64-row unstamped tail with only one necessary message-parts read, an existing anchor's duplicate-content claim, provenance-backed externalized file/image matches, and provenance-gated host redaction (including a mismatched-call rejection).
  • Isolated synthetic full-bootstrap comparison (10,000 visible entries / 2,000 stored, 50 unstamped tail rows, ~3.3 KB bodies): 4.69 s baseline → 0.71 s candidate, with the same reconcile verdict and imported count. This fixture uses in-memory synthetic data, not live transcript or database contents.
  • git diff --check passed. No package installation or production edit was performed.

Correctness notes

All existing match paths require the same role. Different text identities can still match when the persisted row may contain an externalized reference, or when incoming tool-call IDs permit host-redaction equivalence. Decorated user-message comparison remains in its separate role-specific path. A resolved transcript entry still participates in content-claim counting before it is ruled ineligible to adopt a second row.

The test runner used existing local dependencies without an install. Four additional engine-backed suites could not start because this isolated checkout lacks @earendil-works/pi-coding-agent; repository typecheck also could not complete because Node type definitions and other dev dependencies are absent. The repository-wide release verification suite was not run. CI and maintainer review remain necessary before merge.

AI assistance

The investigation, patch, tests, and report were prepared with AI assistance and manually reviewed against the relevant source paths and test results.

@jalehman

Copy link
Copy Markdown
Contributor

Thanks for the investigation and benchmarks, @aounakram. Closing as superseded: the pre-turn bootstrap reconciliation and legacy adoption planner this screens were removed in #1243, after ingestion moved to a visible-delta cursor (#1238) that reads only new transcript entries each turn (low milliseconds on large sessions). If bootstrap preparation is still slow for you on 1.2.0+, please reopen #1230 or file a fresh issue with timings and we'll profile the new path.

@jalehman jalehman closed this Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants