Skip to content

Add API report completeness validation instruction for review agents #4506

Description

@fullsend-ai-retro

What happened

On PR #4417, the fix agent pushed commit 6e4e53b which included api-report changes. The review agent (run 33420007704) flagged report.api.md as stale — a correct but shallow finding. The human reviewer then discovered the problem was catastrophic: a prior commit had run api-extractor without --tsc, reading stale .d.ts files, which dropped the entire DeltaSyncManager public API surface (6 exported symbols) from the report. Three follow-up commits (88cc6bd, df74548/05ffd1f, b03bb6b) were needed to fix this, including reverting unintended reordering in unrelated packages.

What could go better

The review agent detected the api-report was stale but did not assess its structural completeness. It noticed field-level issues (like the updated optionality mismatch) but missed that entire classes and interfaces were absent from the report. If the review agent had cross-referenced the symbols exported from src/index.ts against the entries in report.api.md, it would have detected the missing DeltaSyncManager, CursorStore, CursorState, ApplyDeltaOptions, DeltaSyncManagerOptions, and InMemoryCursorStore entries. Confidence: high — the diff and human triage comments explicitly document this gap. This is complementary to existing issue #4392 (which covers code/fix agent regeneration); even if agents always regenerate the report, an incorrect regeneration (as happened here) would go undetected without review-side validation.

Proposed change

Add a review-focused instruction to .fullsend/AGENTS.md after the existing changeset guidance. The instruction should tell the review agent: when a PR adds, removes, or renames public exports in any plugin package, verify that report.api.md includes entries for all symbols re-exported from the package's src/index.ts. Flag missing or unexpectedly removed entries as a medium-severity finding. Example instruction text: "When reviewing PRs that change public API surface (exports in src/index.ts), cross-check report.api.md to confirm all exported symbols appear in the report. A report missing exported symbols indicates a broken api-extractor run."

Validation criteria

The next 3 agent-authored or agent-reviewed PRs in this repo that modify public API exports should have the review agent validate api-report completeness against src/index.ts exports. The review agent should flag any missing symbols rather than only noting staleness.


Generated by retro agent from #4417

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions