Problem
The CLI's pending-selection predicate parses an existing judge file inside a broad try/except Exception: continue. A truncated JSON cache therefore skips the entire run before rescore_one gets a chance to recover it. The worker already handles invalid cached JSON, but is never scheduled in this case. Verdict files are written directly, so interruption can create this state.
Sources: pending selection, worker cache fallback and direct write.
Verified reproduction
- Create a temporary intercepted run with valid
run-meta.json and data/interception.json.
- Write the incomplete text
{"match": to judge.json.
- Run the real
rescore.main() with --only-batch, --rubric strict, a temporary dummy model configuration and a mocked judge.
- Actual: exit 0, 0 judge calls, summary reports 1 judge error. A normal rerun repeats this behavior unless the user knows to use
--force or delete the cache.
Acceptance criteria
Suggested priority P2: recoverability and unattended batch reliability. #299 handled match: null; this is malformed JSON rejected before that recovery path. #303 concerned batch metadata, not judge cache writes. #313 does not change this predicate.
@Perry2004 please review recovery behavior and the appropriate partial-failure exit status.
Problem
The CLI's pending-selection predicate parses an existing judge file inside a broad
try/except Exception: continue. A truncated JSON cache therefore skips the entire run beforerescore_onegets a chance to recover it. The worker already handles invalid cached JSON, but is never scheduled in this case. Verdict files are written directly, so interruption can create this state.Sources: pending selection, worker cache fallback and direct write.
Verified reproduction
run-meta.jsonanddata/interception.json.{"match":tojudge.json.rescore.main()with--only-batch,--rubric strict, a temporary dummy model configuration and a mocked judge.--forceor delete the cache.Acceptance criteria
Suggested priority P2: recoverability and unattended batch reliability. #299 handled
match: null; this is malformed JSON rejected before that recovery path. #303 concerned batch metadata, not judge cache writes. #313 does not change this predicate.@Perry2004 please review recovery behavior and the appropriate partial-failure exit status.