Skip to content

feat(core): tip about VRA when a build uses Layout review mode (PER-9502)#2276

Open
prklm10 wants to merge 3 commits into
masterfrom
feat/PER-9502-vra-layout-tip
Open

feat(core): tip about VRA when a build uses Layout review mode (PER-9502)#2276
prklm10 wants to merge 3 commits into
masterfrom
feat/PER-9502-vra-layout-tip

Conversation

@prklm10

@prklm10 prklm10 commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Emits a single warn-level tip immediately before the Finalized build log whenever any snapshot in a build had Layout review mode enabled (enableLayout, set via global config or per-snapshot).
  • Why: Layout is a legacy review mode; VRA (Visual Review Agent) is the recommended replacement. This surfaces a contextual nudge at build finalization — where users already look — with minimal noise (one line, only when Layout is actually used).
  • Key decisions:
    • Detection flips a per-build flag (build.layoutUsed) in the queue push handler; emission is gated in the end handler, so it logs at most once per process.
    • Covers both standard and parallel builds: each parallel shard runs core's end handler, so cli-build finalize needs no change.
    • Tip copy (renders with the [percy] prefix at normal log level):

      Tip: VRA is Percy's recommended visual review mode — more accurate and adaptable than Layout. Learn more: https://www.browserstack.com/docs/percy/ai-agents/visual-review-agent/overview.

Changes

  • packages/core/src/snapshot.js — detection in push handler + guarded warn in end handler.
  • packages/core/test/snapshot.test.js — 4 new specs.

Testing

  • 4 new specs in snapshot.test.js, all passing:
    • logs the tip before finalizing when a snapshot has layout enabled
    • logs the tip when enableLayout is set globally in config
    • logs the tip only once when multiple snapshots have layout enabled
    • does not log the tip when no snapshot has layout enabled
  • Lint clean (eslint from repo root).
  • Note: the full local core suite shows pre-existing, environment-only failures unrelated to this change — install.test.js (global.__MOCK_IMPORTS__ undefined; ESM loader hook) and one api.test.js server-disabled spec (AggregateError vs ECONNREFUSED, a Node networking quirk). No snapshot/finalize/layout specs fail.

Post-Deploy Monitoring & Validation

  • What to monitor/search
    • Logs: CLI build output for the new line Tip: VRA is Percy's recommended visual review mode.
  • Validation checks (queries/commands)
    • npx percy snapshot with enableLayout: true (global or per-snapshot) → tip appears once just before Finalized build.
    • Same run without any Layout snapshot → tip does NOT appear.
  • Expected healthy behavior
    • Exactly one tip line per build process when Layout is used; none otherwise. Build finalization behavior unchanged.
  • Failure signal(s) / rollback trigger
    • Tip appearing on builds with no Layout usage, duplicate tips within a single process, or any change to Finalized build ordering → revert this commit (log-only change, safe to revert).
  • Validation window & owner
    • Window: first few CLI releases / dogfood builds after merge. Owner: PR author.

Compound Engineered 🤖 Generated with Claude Code


Update: instrumentation of the recommendation

The VRA tip is now also instrumented via a build event so we can measure how often the recommendation is shown:

  • Extended PercyClient#sendBuildEvents to optionally accept the newer top-level params event_name and category (backward compatible — existing callers unchanged; when omitted the API applies its defaults).
  • When the tip is shown, core sends a build event before finalizing the build (so the build is still open), wrapped in try/catch so telemetry can never fail a build:
    • event_name: percy_cli_vra_recommendation_emitted
    • category: percy:cli
    • data: { message: "VRA recommendation shown for a build using Layout review mode" }

Note: category was set to percy:cli (the CLI-side sibling of the percy:app example) since it wasn't specified — easy to change if a different taxonomy is preferred.

Added tests: client sendBuildEvents with event_name/category, and core specs asserting the event is sent (filtered by event_name) only when Layout is used.

Emit a single warn-level tip just before the "Finalized build" log
whenever any snapshot in the build had enableLayout set (via global
config or per-snapshot). Detection flips a per-build flag in the queue
push handler; emission is gated in the end handler, so it logs at most
once per process. Parallel builds surface the tip per shard via core —
cli-build finalize needs no change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@prklm10 prklm10 requested a review from a team as a code owner June 11, 2026 10:59
@prklm10 prklm10 requested review from ninadbstack and pranavz28 June 11, 2026 10:59
@prklm10 prklm10 added the ✨ enhancement New feature or request label Jun 11, 2026
prklm10 and others added 2 commits June 16, 2026 11:39
Emit a percy_cli_vra_recommendation_emitted build event (category
percy:cli) when the VRA tip is shown, so the recommendation can be
instrumented. Extends the client's sendBuildEvents to optionally send
the newer top-level event_name and category params; existing callers
are unaffected (params default and the API applies its own defaults).

The event is sent before finalizing the build and is wrapped so
telemetry can never fail a build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a spec that makes sendBuildEvents reject and asserts the build
still finalizes, exercising the previously-uncovered catch block in
the snapshot queue end handler (restores 100% coverage).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

✨ enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants