Skip to content

Honor per-task check_timeout_s: the check kill timer was hard-coded at 60s - #91

Open
brandoncordoba wants to merge 2 commits into
NateBJones-Projects:mainfrom
brandoncordoba:fix/check-timeout-s
Open

Honor per-task check_timeout_s: the check kill timer was hard-coded at 60s#91
brandoncordoba wants to merge 2 commits into
NateBJones-Projects:mainfrom
brandoncordoba:fix/check-timeout-s

Conversation

@brandoncordoba

Copy link
Copy Markdown

What

The manifest field check_timeout_s was silently dropped in TaskSpec parsing, so every check ran against the hard-coded 60s CHECK_TIMEOUT_S regardless of what the manifest said. Any check that wraps a real build or test suite (a fix-swarm.py check running a cold xcodebuild build takes 4–8 minutes) was killed at ~60s and the task verdicted TIMEOUT even when the worker's output was correct.

Hit twice in real runs: styled-by-you-picker (2026-07-18) and ipad-hover-sweep (2026-08-02), where all 3 lanes produced correct patches and all 3 falsely verdicted TIMEOUT.

How

  • TaskSpec.from_obj parses and validates check_timeout_s (must be positive, ceiling 3600 — a check is verification, not the work).
  • The value is plumbed into Verifier._run_check's kill timer; the timeout message reports the actual limit used.
  • run --dry-run prints it per task.
  • New lint finding: a check that runs a build/test-shaped tool directly (xcodebuild, cargo test, swift test, pytest, …) at the default timeout gets a nudge to raise it. Matching is command-position only, so tool names inside echo strings or wrapper-script arguments don't fire.
  • The four template kits whose checks wrap builds inside scripts (fix-swarm, repo-feature, test-hardening, doc-swarm) now set check_timeout_s in their skeletons, since lint cannot see inside wrappers.
  • README manifest-fields table documents the field.

Verification

All claims are backed by executed tests (258 pass on macOS, Python 3.13):

  • parse/validate/default behavior, including the 0 and 3601 rejection messages
  • Verifier.verify honoring the task's value (a sleep 5 check killed at 1s)
  • an end-to-end run where the manifest's check_timeout_s governs the kill timer and the state file records check_timed_out
  • the regression shape that bit us: a task built via from_obj whose check outlives a shrunk default but passes because the manifest's raised limit — not the default — governs
  • lint: direct xcodebuild/swift test at the default is flagged; the same command hidden inside a wrapper-script argument is not; a raised value clears the finding

Rebased on current main (includes #84 — an earlier draft of this carried its own StateWriter race fix, dropped in favor of yours).

🤖 Generated with Claude Code

brandoncordoba and others added 2 commits August 3, 2026 00:02
…t 60s

The manifest field was silently dropped in TaskSpec parsing, so checks
wrapping cold builds (ipad-hover-sweep 2026-08-02, styled-by-you-picker
2026-07-18) were killed at the 60s default and honest work landed as
TIMEOUT. Parse and validate the field (positive, ceiling 3600), plumb it
into Verifier._run_check, print it in dry-run, and lint-nudge checks
that run build/test tools directly at the default. Wrapper-check kits
(fix-swarm, repo-feature, test-hardening, doc-swarm) now set it in
their skeletons since lint cannot see inside wrapper scripts.

Regression tests: parsing/validation, the verifier honoring the task's
value, an end-to-end run where the manifest value governs the kill
timer, and a slow check that finishes because the raised limit — not
the shrunk default — governs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Guidance78

Copy link
Copy Markdown

Hit this same wall independently, a check wrapping a real test suite got killed
at 60s while the worker's output was correct, and the retry burned on the same
limit.

Been running an equivalent fix locally since and it's held up. Nothing to add to
the implementation here; the ceiling and the command-position lint matching both
look right to me. Just wanted to confirm the failure is real outside your setup,
in case that helps it get picked up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants