Improve Live polling responsiveness and reliability#371
Open
pbakaus wants to merge 1 commit into
Open
Conversation
Restore foreground/background polling as the primary harness architecture, add progressive publication and framework-safe previews, and harden quality and regression coverage. The experimental app-server runtime is intentionally excluded.\n\nPrepared with AI assistance under maintainer direction.
Deploying impeccable with
|
| Latest commit: |
c6ac34b
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://b1bbcabb.impeccable-2rv.pages.dev |
| Branch Preview URL: | https://codex-live-polling-improveme.impeccable-2rv.pages.dev |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit c6ac34b. Configure here.
| if (index === -1) out[body] = true; | ||
| else out[body.slice(0, index)] = body.slice(index + 1); | ||
| } | ||
| return out; |
There was a problem hiding this comment.
Benchmark CLI ignores spaced flags
Medium Severity
The new bench:live script’s parseArgs treats any --name without = as boolean true and never reads the next argv token. Flags like --fixture vite8-react-plain, --output path, or --agent llm therefore resolve to the string "true" instead of the intended value, so runs target the wrong fixture or write to the wrong path.
Reviewed by Cursor Bugbot for commit c6ac34b. Configure here.
| const index = (sorted.length - 1) * p; | ||
| const lower = Math.floor(index); | ||
| const upper = Math.ceil(index); | ||
| return Math.round((sorted[lower] * (1 - (index - lower)) + sorted[upper] * (index - lower)) * 100) / 100; |
There was a problem hiding this comment.
Control bench crashes empty metrics
Low Severity
After the e2e loop, summarize and percentile assume at least one numeric sample. If the metrics JSONL file is missing rows (misconfigured IMPECCABLE_E2E_METRICS_FILE or no recorded timings), sorted[0] and sorted.at(-1) are undefined and the script throws instead of reporting an empty result.
Reviewed by Cursor Bugbot for commit c6ac34b. Configure here.
Greptile SummaryThis PR reworks Live mode around durable polling and progressive preview publication. The main changes are:
Confidence Score: 4/5Mostly safe, with one contained Vue accept-path bug to fix. The polling and publication flow is fenced and covered, but the Vue accept inliner can write accepted source that drops required root attributes.
 T-Rex LogsWhat T-Rex did
Important Files Changed
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+277
to
+280
| function parseStaticAttrs(attrs) { | ||
| const out = new Map(); | ||
| const re = /([A-Za-z_:][\w:.-]*)\s*=\s*(["'])(.*?)\2/g; | ||
| let match; |
There was a problem hiding this comment.
inlineVueComponentAccept uses parseStaticAttrs to merge the original root attributes back into the accepted Vue template, but this parser only records name="value" attributes. When the picked Vue root has required boolean or shorthand attributes such as disabled, required, or v-cloak, Accept drops them if the generated variant omits them, so the accepted source no longer preserves the original component contract.
Context Used: AGENTS.md (source)
Artifacts
Repro: focused Node script for boolean and shorthand Vue root attributes
- Contains supporting evidence from the run (text/javascript; charset=utf-8).
- Keeps the command output available without making the summary code-heavy.
There was a problem hiding this comment.
Pull request overview
This PR substantially reworks Impeccable Live’s polling and publication/accept workflow to improve responsiveness and reduce race conditions, while expanding framework support (Nuxt/Vue component previews) and hardening the detector against inset-shadow “single-edge stripe” regressions. It also adds benchmark and rendered-quality evaluation utilities plus broader regression coverage.
Changes:
- Restore/standardize foreground/background polling behavior (including type-lane filtering), and harden session journaling/state transitions to fence late generation after Accept/Discard.
- Introduce progressive publication primitives (artifact preparation/publish flow) plus accept/source-locking and idempotent accept receipts.
- Add Nuxt/Vue component preview path alongside Svelte component preview, plus new benchmarks/rendered-quality tooling and fixtures/tests for regressions.
Reviewed changes
Copilot reviewed 73 out of 74 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/live-wrap.test.mjs | Adds regression coverage for isolated source previews and safer ambiguity handling. |
| tests/live-vue-component.test.mjs | New Nuxt/Vue component preview tests covering scaffold, accept, publish fencing, and cleanup. |
| tests/live-session-store.test.mjs | Expands session journal/state tests for progressive publication, fencing, and timing telemetry. |
| tests/live-rendered-quality.test.mjs | New unit tests for rendered-quality prompt/context parsing and scoring summaries. |
| tests/live-reference.test.mjs | Tightens authoring contract expectations for Live reference routing and Codex-specific guidance. |
| tests/live-provider-benchmark.test.mjs | New benchmark scoring/aggregation tests for provider comparisons and cleanup gates. |
| tests/live-poll.test.mjs | Adds polling lane normalization and new reply payload fields/agent-reply types. |
| tests/live-inject.test.mjs | Adds Nuxt adapter injection/remove/idempotency tests and conflict behavior. |
| tests/live-generation-preflight.test.mjs | New tests for generation preflight command construction and output shaping. |
| tests/live-event-validation.test.mjs | Adds validation coverage for worker progress/telemetry event types. |
| tests/live-e2e/ui.mjs | Improves E2E UI helpers (action picker selection, richer diagnostics, arrival logic). |
| tests/live-e2e/session.mjs | Extends fixture boot/session orchestration (progressive runs, tracing, install hardening). |
| tests/live-e2e/agents/llm-agent.mjs | Adds progressive-generation guidance/validation and optional spec inclusion for benchmarks. |
| tests/live-e2e-llm-agent.test.mjs | Adds unit tests for progressive prompt guidance and new validation helpers. |
| tests/live-e2e-agent-output.test.mjs | Adds helper test for recognizing expected “late generation” cancellation errors. |
| tests/live-browser-source.test.mjs | Strengthens source-level assertions around poll connectivity, Vue module routing, and progressive checkpoints. |
| tests/live-browser-regression.test.mjs | Expands regression guards for discard behavior, anchor preservation, progressive readiness, Tune param refresh, and component preview gating. |
| tests/live-benchmark.test.mjs | New benchmark library tests for metrics, progressive assembly, and report comparisons. |
| tests/live-accept.test.mjs | Adds isolated source artifact accept/discard tests and accept receipt replay/conflict coverage. |
| tests/framework-fixtures/vite8-sveltekit/files/src/routes/+page.svelte | Updates fixture to include styling for Live UI stability. |
| tests/framework-fixtures/vite8-react-brand-fidelity/gitignore.txt | New fixture ignore rules for brand-fidelity benchmark fixture. |
| tests/framework-fixtures/vite8-react-brand-fidelity/fixture.json | New brand-fidelity fixture configuration including rendered-quality metadata. |
| tests/framework-fixtures/vite8-react-brand-fidelity/files/vite.config.js | New fixture Vite config for predictable dev server behavior. |
| tests/framework-fixtures/vite8-react-brand-fidelity/files/src/styles.css | New fixture design tokens + baseline styling for fidelity checks. |
| tests/framework-fixtures/vite8-react-brand-fidelity/files/src/main.jsx | New fixture entrypoint wiring React app. |
| tests/framework-fixtures/vite8-react-brand-fidelity/files/src/App.jsx | New fixture component for offer-card and ActionLink contract. |
| tests/framework-fixtures/vite8-react-brand-fidelity/files/PRODUCT.md | New fixture product context for Live. |
| tests/framework-fixtures/vite8-react-brand-fidelity/files/package.json | New fixture dependencies for React/Vite. |
| tests/framework-fixtures/vite8-react-brand-fidelity/files/index.html | New fixture HTML shell. |
| tests/framework-fixtures/vite8-react-brand-fidelity/files/DESIGN.md | New fixture design system constraints for fidelity gating. |
| tests/framework-fixtures/README.md | Documents Nuxt fixture and external fixture usage for benchmark tooling. |
| tests/framework-fixtures/nuxt-vite7/fixture.json | Promotes Nuxt fixture to runtime-capable via adapter/plugin approach. |
| tests/framework-fixtures/nuxt-vite7/files/nuxt.config.ts | Updates Nuxt fixture config (including compatibility date). |
| tests/framework-fixtures/nuxt-vite7/files/app/pages/index.vue | New Nuxt app-directory page fixture. |
| tests/framework-fixtures/nuxt-vite7/files/app/app.vue | New Nuxt app root fixture. |
| tests/framework-fixtures/nuxt-vite7/files/app.vue | Removes legacy root app.vue fixture layout. |
| tests/framework-fixtures.test.mjs | Updates fixture assertions for new Live artifacts/receipts/locks and Nuxt plugin adapter behavior. |
| tests/fixtures/antipatterns/astro-inset-shadow-stripe.astro | New fixture for Astro inset-shadow single-edge stripe regression. |
| tests/detect-antipatterns-fixtures.test.mjs | Adds Astro structural CSS fixture assertions for inset-shadow stripe detection. |
| skill/scripts/live/vue-component.mjs | New Nuxt/Vue component preview scaffold/accept/cleanup implementation. |
| skill/scripts/live/source-lock.mjs | New per-source-file locking helper for accept/publish critical sections. |
| skill/scripts/live/source-artifact.mjs | New isolated source preview manifest/session helpers. |
| skill/scripts/live/session-store.mjs | Expands session snapshot model and fences late generation/publication events. |
| skill/scripts/live/poll-lanes.mjs | New event prioritization and selection logic for poll lanes. |
| skill/scripts/live/generation-preflight.mjs | New helper to precompute scaffold metadata via wrap/insert for generation. |
| skill/scripts/live/event-validation.mjs | Adds schema validation for new progress/cleanup events. |
| skill/scripts/live.mjs | Updates Live top-level guidance to emphasize harness-native poll loop setup. |
| skill/scripts/live-wrap.mjs | Adds Vue component previews, isolated source artifact mode, and safer ambiguity refusal behavior. |
| skill/scripts/live-status.mjs | Refactors recovery hint composition. |
| skill/scripts/live-publish.mjs | New CLI wrapper for prepare/publish generation artifacts. |
| skill/scripts/live-poll.mjs | Adds type filtering, accept reply metadata, and lane normalization. |
| skill/scripts/live-inject.mjs | Adds Nuxt adapter support via dev-only client plugin + expanded ignore patterns. |
| skill/scripts/live-accept.mjs | Adds source locking, source-artifact + Vue accept paths, and idempotent accept receipts. |
| skill/agents/impeccable-live-generator.md | New Codex-focused generator agent spec for progressive publication workflow. |
| scripts/test-suites.mjs | Adds new Live benchmark/rendered-quality/Vue tests to suite list. |
| scripts/lib/live-rendered-quality.mjs | New rendered-quality judging prompt/context/parsing utilities. |
| scripts/lib/live-benchmark.mjs | New benchmark metrics and report comparison utilities (atomic vs progressive). |
| scripts/judge-live-rendered.mjs | New CLI to run rendered-quality judging over artifact bundles. |
| scripts/compare-live-benchmarks.mjs | New CLI to compare atomic/progressive benchmark reports against thresholds. |
| scripts/benchmark-live.mjs | New Live benchmark runner for interaction latency and progressive delivery. |
| scripts/benchmark-live-init.mjs | New initialization benchmark runner for cold/warm Live startup. |
| scripts/benchmark-live-control.mjs | New control benchmark runner for E2E-derived latency metrics. |
| package.json | Adds bench:live / bench:live:providers scripts. |
| cli/engine/engines/regex/detect-text.mjs | Adds structural inset-shadow stripe scanning (including Astro <style> blocks). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+1005
to
+1007
| function acceptReceiptPath(cwd, id) { | ||
| return path.join(getLiveDir(cwd), 'accept-receipts', `${id}.json`); | ||
| } |
github-actions
Bot
added
blocked: review threads
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Architecture boundary
The experimental Codex app-server/worker runtime is intentionally excluded from this PR. It remains isolated on
codex/live-app-server-experimentfor future research. This branch keeps polling as the production path and retains only optimizations that are independent of that experiment.Generated root harness/provider output is intentionally omitted under the repository's source-first policy.
Validation
bun run build:browserbun run build:extensionbun run buildbun run testbun run test:live-e2e— 32 tests across 24 runtime suitesPrepared with AI assistance under maintainer direction.
Note
High Risk
Changes core Live accept/source-locking, progressive publication, and multi-framework preview paths where races or marker leakage would corrupt user projects; benchmark scripts are additive but the skill/runtime behavior is production-critical.
Overview
Improves Live responsiveness and safety: Codex delegates generation to
impeccable_live_generatorand documents transactional progressive delivery vialive-publish.mjs(fenced artifacts, publish-first variant 1) while other harnesses keep atomic single-edit by default.live.mdaddsevent.scaffoldreuse, Nuxt/Vue component-preview parity with Svelte, and updated polling/carbonize guidance.live-accept.mjsgains source locks, idempotent accept receipts, and dedicated source-artifact, Vue, and existing Svelte accept/discard paths so Accept can fence concurrent generation.The regex engine adds
scanInsetStripeCssfor chromatic inset box-shadow edge stripes (including Astro<style>blocks), aligned with the generator’s no-stripe rule.New
bench:live/bench:live:providersscripts and shared libs measure init, interaction latency, multi-provider strategies, rendered-quality judging, and atomic vs progressive comparisons; the test suite list expands for benchmarks, publication, Vue, and related Live tests.Reviewed by Cursor Bugbot for commit c6ac34b. Bugbot is set up for automated code reviews on this repo. Configure here.