Skip to content

test(docs): stop the guest busy resend test from hanging - #181

Merged
oesni merged 3 commits into
mainfrom
fix/guest-busy-test-hang
Oct 24, 2026
Merged

oesni merged 3 commits into
mainfrom
fix/guest-busy-test-hang

Conversation

@oesni

@oesni oesni commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

TestGuestBusyResendKeepsOrder (added in #180) hangs the test (docs) shard until the 10-minute package timeout. Main 44dfd74 fails this way (run 40100896738), and so does every PR whose CI merges main, for example #174. This change makes the test report failures instead of hanging, fixes the failure, and runs the test with the other wall-clock tests.

Only test code and the CI shard script change. Product code does not.

Cause

  • The hang. In the timeout dump every writer had returned, but firstRound was never released. The flood goroutines therefore never stopped, and the writers' own errors were never reported.
  • The failure behind it. Once the hang was gone, CI showed all ten writers getting connection closed after about 45 s. Every update reaches the other nine writers, and guests share 4 MiB of unconfirmed output (LIMITS.publicSendBytes). Parts of 4,000 characters added up to about 4.9 MB, so a worker slowed by the flood closed every writer as a slow consumer before their receipts arrived.

Changes

  • dcb63e2: every writer exit releases its first round once (sync.OnceFunc, deferred), so the flood stops and the test fails with the reader's error instead of hanging.
  • 12d62ef: parts are 1,000 characters, which keeps the fan-out near 1.3 MB. The test moves to the docs_budget shard in scripts/ci-test.sh, which runs the tests that depend on the wall clock.
  • 9d622dc: the test no longer calls t.Parallel(), like the other docs_budget tests, so a local go test of the package runs it on its own as well.

The test still requires at least one busy answer and still checks that every part lands once and in order, so the smaller parts do not let it pass without exercising busy and resend.

Verification

  • CI on 12d62ef: all checks passed (commit-images skipped), including test (docs) and race.
  • Locally, on 9d622dc: the test passed 3 of 3 runs, with 100 busy answers each. scripts/ci-test.sh docs-budget -count=1 passed. go vet ./internal/apps/docs is clean.
  • An independent review of the three commits found no issues.

Note for later

Readers in this test confirm every frame they receive, but writers were still closed as slow consumers while the flood delayed the worker. Several people editing a large document quickly beside heavy visitor traffic could hit the same limit. This PR changes only the test. Whether the shared output budget should behave differently is a separate question.

A writer that returned early through t.Error never called
firstRound.Done, so the flood never stopped and the test ran until the
package timeout without reporting its error.
The test runs ten guest writers against a flood, and its busy answers and
the receipts that keep writers under the guests' 4 MiB output limit depend
on the worker keeping up with the wall clock. Run in parallel with the
rest of the docs shard on a CI runner, the worker fell behind and closed
every writer: first as slow consumers, then with read timeouts. Make it a
serial test in the docs-budget shard, as the other refill-rate tests are,
and report a writer's close reason.
@oesni
oesni force-pushed the fix/guest-busy-test-hang branch from 0c317b2 to 12d62ef Compare October 24, 2026 15:38
It now runs in the docs-budget shard with the other tests that depend on
the wall clock, and like them it no longer calls t.Parallel, so a local
go test of the package runs it alone too.
@oesni
oesni marked this pull request as ready for review October 24, 2026 15:57
@oesni
oesni merged commit 1ce795d into main Oct 24, 2026
21 checks passed
@oesni
oesni deleted the fix/guest-busy-test-hang branch October 24, 2026 16:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant