Repository navigation
TEST Stabilize merge-queue async and browser checks - #3011
Open
Roman Lutz (romanlutz) wants to merge 3 commits into
Open
Roman Lutz (romanlutz) wants to merge 3 commits into
Roman Lutz (romanlutz) wants to merge 3 commits into
Conversation
Expire generator deadlines at the intended await, observe cancellation without watchdog-induced cancellation, and allow CI scheduling slack for coordination. Remove native spinner press-and-hold from single-step browser checks and document the async test patterns. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep upstream setup/send assertions alongside deterministic generator deadlines. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Allow CI scheduling slack before requesting cancellation and use non-cancelling completion waits while preserving rollback, cleanup, and exception assertions. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Merge-queue runs were rejecting unrelated PRs because short async-test watchdogs expired during setup or cleanup. Some watchdogs also sent an extra cancellation to the task under test, changing the cancellation count or original exception the test was meant to verify.
asyncio.Timeoutcontexts at the intended pending await, while checking the configured deadlines, cleanup, and original outcome.finally.inserted.wait()before cancellation was requested. Keep the rollback, connection cleanup, repeated-cancellation, and original-exception assertions intact for both database modes.No production code, production deadlines, coverage thresholds, retries, or required-check settings change. Cancellation, concurrency bounds, and health responsiveness while a database operation remains pending are still asserted. The separate Alembic repair for #2792 is not included.
Merged main at
13cb39257dnormally. The generator-test conflict was resolved by preserving upstream setup/send assertions alongside this PR's controlled deadlines.Tests and Documentation
Validation on Windows with all Python extras installed:
uv run --frozen --no-sync -m pytest -n 4 --dist=loadfile tests\unit --timeout=120 -q: passed, 23,725 tests, 21 skipped, and 84 subtests. Used the documented Makefile-equivalent command becausemakeis unavailable. This full run preceded the later merges of main and SQLite watchdog follow-up.uv run --frozen --no-sync pytest -q -n 4 --dist=loadfile tests\unit\test_async_utils.py tests\unit\memory\test_sqlite_cancellation.py tests\unit\memory\test_async_memory.py tests\unit\backend\test_main.py tests\unit\backend\test_scenario_service.py tests\unit\executor\promptgen\test_target_objective_generator.py tests\unit\scenario\core\test_scenario_partial_results.py --timeout=90: passed, 304 tests on the integrated branch including the SQLite follow-up.uv run --frozen --no-sync pytest -q tests\unit\memory\test_sqlite_cancellation.py -k failed_session_rollback --timeout=90: passed, all 10 reported-path variants; 27 deselected.f5c442d456:uv run --frozen --no-sync pytest -q -n 4 --dist=loadfile tests\unit\test_async_utils.py tests\unit\backend\test_main.py tests\unit\backend\test_scenario_service.py tests\unit\executor\promptgen\test_target_objective_generator.py tests\unit\scenario\core\test_scenario_partial_results.py tests\unit\score\test_scorer_metrics.py tests\unit\score\test_scorer_metrics_io.py --timeout=90: passed, 231 tests and 4 skipped.frontend, with an isolatedE2E_FRONTEND_PORT:uv run --frozen --no-sync cmd.exe /c 'npm exec --no -- playwright test numeric-controls.spec.ts --project=mock --retries=0 --repeat-each=3 --workers=1 --fail-on-flaky-tests': passed, all 24 executions without retries, before the later main merge.frontend:npm exec --no -- eslint e2e\numeric-controls.spec.ts --max-warnings 0andnpm run type-check -- --pretty false: passed before the later main merge.uv run --frozen --no-sync ruff check tests\unit\memory\test_sqlite_cancellation.py tests\unit\executor\promptgen\test_target_objective_generator.py,uv run --frozen --no-sync ruff format --check tests\unit\memory\test_sqlite_cancellation.py tests\unit\executor\promptgen\test_target_objective_generator.py, anduv run --frozen --no-sync ty check tests\unit\memory\test_sqlite_cancellation.py tests\unit\executor\promptgen\test_target_objective_generator.py: passed after the later main merge.uv run --frozen --no-sync python -m build_scripts.validate_docs: passed. Commit-time pre-commit hooks also passed.Updated
doc/contributing/5_unit_tests.md. JupyText was not run because this PR changes no notebooks or JupyText sources. The complete GitHub OS/Python matrix has not been verified on the latest head.