Skip to content

Unified Revalidation#1026

Open
busbyk wants to merge 18 commits intorevalidationfrom
unified-revalidation
Open

Unified Revalidation#1026
busbyk wants to merge 18 commits intorevalidationfrom
unified-revalidation

Conversation

@busbyk
Copy link
Copy Markdown
Collaborator

@busbyk busbyk commented Apr 6, 2026

Make sure to check out #1025 first as this PR is dependent on/based on that as it's first phase.

Description

Replaces the two parallel revalidation subsystems (block reference tracking and relationship reference tracking) with a single unified system that discovers all document references at any nesting depth. This fixes confirmed bugs where events had
wrong block mappings, events were missing from relationship queries entirely, and deeply nested references (e.g., a SponsorsBlock inside a ContentBlock's richText) were invisible to the old system.

The old system required per-collection tracking fields, per-collection population hooks, and hand-maintained config mapping functions. The new system uses Payload's collection config to automatically discover all relationship, upload, and block
references in any document — no manual wiring needed when adding new collections or blocks.

With full on-demand revalidation coverage, the ISR safety-net interval is increased from 600s (10 min) to 3600s (1 hour).

Related Issues

Closes #455

Key Changes

New unified system (3 files):

  • src/utilities/findDocumentsWithReferences.ts — Queries the documentReferences JSON field across all routable collections
  • src/utilities/revalidateDocumentReferences.ts — Orchestrates find → revalidate, replacing both revalidateBlockReferences and revalidateRelationshipReferences
  • src/hooks/revalidateForm.ts — Updated to use unified system

Updated revalidation hooks (8 files):

  • All collection revalidation hooks (revalidatePost, revalidateEvent, revalidateBiography, revalidateMedia, revalidateDocuments, revalidateSponsors, revalidateTag, revalidateTeam, revalidateForm) now call
    revalidateDocumentReferences instead of the old dual revalidateBlockReferences + revalidateRelationshipReferences

Schema changes:

  • Removed blocksInContent from Posts and Events, blocksInHighlightedContent from HomePages

Removed (10 files, ~1,370 lines):

  • revalidateBlockReferences.ts, revalidateRelationshipReferences.ts
  • findDocumentsWithBlockReferences.ts, findDocumentsWithRelationshipReferences.ts
  • getBlocksFromConfig.ts, getRelationshipsFromConfig.ts, extractBlockReferencesFromLexical.ts
  • populateBlocksInContent.ts (Posts + Events copies), populateBlocksInHighlightedContent.ts

ISR interval:

  • Increased from revalidate = 600 to revalidate = 3600 across all 5 routable page components (this has to be statically analyzed so we can't extract this into a constant)

Tests: New tests covering extractDocumentReferences, findDocumentsWithReferences, revalidateDocumentReferences, and revalidateDocument (cycle-safety invariant)

How to test

  1. pnpm tsc — no type errors
  2. pnpm lint — no lint errors
  3. pnpm test — all tests pass
  4. Functional: edit a document in the admin that is referenced inside a block within a richText field on another page → the referencing page should revalidate immediately

Screenshots / Demo video

https://www.loom.com/share/9f6de2da56c24be8976a95223dbe30d7

Migration Explanation

  • 20260406_212933_remove_old_block_tracking_fields — Drops 6 tables: posts_blocks_in_content, events_blocks_in_content, home_pages_blocks_in_highlighted_content + their version tables. Intentional data loss — this data has been fully replaced by the documentReferences field backfilled in a prior migration on the revalidation branch. See _explanation.md alongside the migration.

Future enhancements / Questions

  • Let's monitor for stale content reports for 1-2 weeks post-deploy. If stable, let's consider increasing ISR interval further (4h, 24h).

busbyk and others added 6 commits April 6, 2026 13:49
Discovers collections with a documentReferences field by inspecting
payload.config.collections at runtime instead of hardcoding slugs.
Queries each discovered collection for matching references. Widens
DocumentForRevalidation.collection to string and uses Tenant type.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replaces both revalidateBlockReferences and revalidateRelationshipReferences
with a single function that calls findDocumentsWithReferences then
revalidateDocument for each result. Includes 6 tests covering the
orchestration contract.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ferences

All 9 afterChange hooks (7 reference + 2 routable collections) now call
revalidateDocumentReferences instead of revalidateBlockReferences +
revalidateRelationshipReferences. Biography cascading logic preserved.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The unified documentReferences system fully replaces the old per-collection
block tracking (blocksInContent, blocksInHighlightedContent) and their
associated hooks and query utilities. This removes 10 files and ~1,300 lines
of code that are no longer called.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
On-demand revalidation now has full coverage of deep-nested references,
so the safety-net time-based interval can be relaxed. Extracts the value
into an ISR_REVALIDATE_INTERVAL constant for single-point configuration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@busbyk busbyk self-assigned this Apr 6, 2026
Verifies that revalidateDocument only invalidates Next.js cache
(revalidatePath) and never triggers Payload write operations, which
is the architectural guarantee that circular document references
(A → B → A) cannot cause infinite revalidation loops.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Next.js statically analyzes route config exports and cannot resolve
imported variables. Replace ISR_REVALIDATE_INTERVAL with literal 3600.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 7, 2026

Preview deployment: https://unified-revalidation.preview.avy-fx.org

busbyk and others added 6 commits April 9, 2026 13:45
The members field on Teams is hasMany, so using equals only matches teams
where members is exactly [biographyId]. The contains operator correctly
checks if the array includes the value.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
If a collection has documentReferencesField() but no path mapping in
revalidateDocument, log a warning instead of silently doing nothing.
Also adds early returns to each handled branch and updates the existing
test to verify the warning.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The constant was never imported (Next.js requires static literals for
revalidate exports). Also removes a stale reference to the deleted
getRelationshipsFromConfig utility in a comment.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove references to deleted utilities (revalidateBlockReferences,
findDocumentsWithBlockReferences, etc.) and update the Known Limitations
section since deeply nested richText is now handled by
extractDocumentReferences.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove redundant JSDoc that restates function signatures, obvious
traversal comments, stale references to deleted utilities, and
over-explained test comments. Keeps comments that explain non-obvious
behavior (type widening, cycle safety invariant).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
busbyk and others added 2 commits April 9, 2026 15:57
…nces

Avoids naming collision with the DocumentReference interface exported
from extractDocumentReferences.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
payload.find() defaults to limit 10. Without limit: 0, documents
referenced by more than 10 other documents would silently miss
revalidation for the overflow.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@busbyk busbyk marked this pull request as ready for review April 9, 2026 23:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant