This repository was archived by the owner on Oct 4, 2026. It is now read-only.
Repository navigation
test(webv2): fix two flaky browser tests (timing- and cursor-dependent) - #92
Merged
Merged
Conversation
…d cursor position Both tests failed intermittently on loaded CI runners, twice blocking PRs in the image-map stack. Neither failure was ever a product bug; both tests were asserting on conditions they had not actually waited for. PreviewCompare had two distinct causes: - The overlay reveals on focus OR hover OR touch press, and the runner's cursor stays wherever the previous test left it. A component that mounts underneath it receives a real `pointerenter`, so it was already revealed before the test began and blur alone would not hide it — the run that exposed this reported `activeElement=BODY` with opacity stuck at 1, i.e. blur had worked and hover was holding it open. The test now clears the hover synthetically (React derives `onPointerLeave` from `pointerout`), which keeps it about focus and independent of pointer position. - It then slept a fixed 350ms and asserted an exact endpoint value. A starved compositor had not finished the transition, so the assertion read the value it started from. Waits now poll until the opacity settles, with a timeout that fails loudly and names the step and last value. Tabs slept 200ms after each of fifteen hovers — three seconds against a 15s test timeout, which is what pushed it over. It now samples until the interaction styles hold still, so each wait is as long as the machine needs and usually far shorter. Verified by running the full browser suite repeatedly: before, these two accounted for most of roughly one failure in three; after, both stayed green across six consecutive runs.
lstein
requested review from
JPPhoto,
Pfannkuchensack and
blessedcoolant
as code owners
August 16, 2026 15:07
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
PreviewCompare.browser.test.tsxandTabs.browser.test.tsxfail intermittently on loaded CI runners.PreviewComparehas now blocked two PRs in the image-map stack (#39 and #41), each needing a manual re-run to get through. Neither failure was ever a product bug — both tests assert on conditions they have not actually waited for.Beyond the wasted re-runs, the real cost is that a chronically red check trains everyone to wave CI through, which is how a genuine failure eventually gets missed.
What was actually wrong
My first theory — "they sleep a fixed interval and assert an exact endpoint" — was only half of it. Instrumenting the failures turned up two independent causes in
PreviewCompare:1. Cursor position leaks between tests. The overlay reveals on
focused || hovered || pressed. The runner's mouse stays wherever the previous test left it, so a component mounting underneath it receives a realpointerenterand is revealed before the test starts — blur then cannot hide it. The diagnostic run showedactiveElement=BODYwith opacity pinned at1: blur had worked, hover was holding it open.Fixed by clearing the hover synthetically (React derives
onPointerLeavefrompointerout), which keeps the test about focus and independent of where the pointer happens to be. Deliberately not by moving the real cursor — I trieduserEvent.hover(document.body)first and it made things worse, since body's centre can be over the component under test.2. Fixed sleeps against real transitions.
waitForMotion()slept 350ms and asserted an exact endpoint. A starved compositor had not advanced the transition, so the assertion read the value it started from — producing the plausible-lookingexpected '1' to be '0'. Waits now poll until opacity settles, and the timeout names the step and the last observed value so the next failure diagnoses itself.Tabswas simpler: 200ms after each of fifteen hovers is three seconds of sleeping against a 15s test timeout, which is what pushed it over the edge. It now samples until the interaction styles hold still.Verification
Full browser suite, run repeatedly on a 16-core machine under load (load average ~6):
pnpm lintand the 5832-test unit suite pass. No production code is touched — only the two test files.Known remaining flakiness (not addressed here)
While measuring, two other browser tests failed intermittently in the same way. They are untouched here to keep this reviewable, and they did not block the stack:
src/platform/ui/hints/FeatureHint.browser.test.tsx— "turns hints off from the card" / "opens the card ... on hover"src/features/generation/ui/shared/ConditioningRebalanceBars.browser.test.tsx— "reports the tap under the pointer"Both look like the same class: hover-driven assertions that depend on real pointer position. Worth a follow-up if they start blocking merges.