Skip to content

[bug] extraction index cache never invalidated after writes — stale reads for 30s #28

Description

@kiannidev

Duplicate check

Item Relation
#16 (open) Not duplicate — rebuild /compare diff page UI
No open issue Cache invalidation after disk writes

Summary

listExtractions(), getExtraction(), and getProductDir() memoize results for 30 seconds. invalidateExtractionCache() exists and its docstring says to call it after writing extraction.json, but nothing in the repo calls it. After batch completion, spotfix accept, or sync extract, pages and APIs can return stale or missing extraction data until the TTL expires.


Steps to reproduce

  1. Open a product page that loads via getExtraction("r770").

  2. Write a new extraction.json (batch worker, spotfix accept, or extract-one).

  3. Reload the product page within 30 seconds.

  4. Observe: old extraction values or missing slug in portfolio index until cache TTL elapses.

  5. Confirm no callers:

    rg 'invalidateExtractionCache' --type ts
    # only the definition in lib/extractions.ts

Expected behavior

Any code path that writes extraction.json calls invalidateExtractionCache() so subsequent reads reflect disk state immediately.


Actual behavior

Cache persists for TTL_MS = 30_000 regardless of writes.


Root cause

/** Force a refresh on the next call — invoke after writing an extraction.json. */
export function invalidateExtractionCache(): void {
  _listCache = null;
  _slugIndex = null;
}

Writers that should invalidate but do not:

  • worker/handlers/anthropic-batch.tswriteExtractionJson()
  • lib/pipeline/spotfix.tsatomicWriteJson(extractionPath, ...)
  • scripts/extract-one.tswriteExtractionJson()

Suggested fix

  1. Call invalidateExtractionCache() inside writeExtractionJson() (central fix), or at each write site.
  2. Add unit test: write invalidates cache; next getExtraction() sees new data.

PR scope

~10–20 lines + one cache test.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions