fix(opencode-review): native OpenAI backend as lead review model (unblock org-wide merge freeze)#350
Conversation
Root cause of the org-wide merge freeze: the required `opencode-review`
check has failed 100% of recent runs across all repos, leaving 12+ PRs
stuck at REVIEW_REQUIRED and every run hanging to the 350-min timeout.
The embedded OpenCode config routed EVERY model in the pool through the
single `github-models` provider (`{env:STRIX_GITHUB_MODELS_TOKEN}` at
https://models.github.ai/inference). GitHub Models rate-limits ("Too many
requests") and caps request bodies at ~4000 tokens (413
tokens_limit_reached), so the reviewer never produced a verdict on any
model and the pool exhausted the whole step without approving.
Fix: give the reviewer a working, un-throttled backend by adding a native
OpenAI provider that hits api.openai.com directly with the org
`OPENAI_API_KEY` secret, and lead the model pool with it.
- opencode.jsonc config: add `provider.openai` (npm @ai-sdk/openai,
baseURL https://api.openai.com/v1, apiKey {env:OPENAI_API_KEY}) with
gpt-5 and gpt-5-mini defined reasoning=true / reasoningEffort=high so
they satisfy assert_opencode_reasoning_effort.py; add "openai" to
enabled_providers.
- OPENCODE_MODEL_CANDIDATES: prepend `openai/gpt-5 openai/gpt-5-mini` as
the lead candidates. The pool passes --model per attempt, so candidate
order (not the config `model` default) is the real lead; the existing
github-models entries are kept unchanged as fallbacks.
- Expose OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} in both env blocks
that invoke opencode with this config (the model-pool step and the
approve/failed-check-diagnosis step) so {env:OPENAI_API_KEY} resolves.
The config `model`/`small_model` defaults are intentionally left on
github-models/deepseek: they are overridden at runtime by the pool's
explicit --model, and static self-tests pin those strings. This unblocks
real reviews without weakening the review gate — if the OpenAI key is
absent or a direct call fails, the pool falls through to the existing
github-models candidates exactly as before.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AxU2xaupAjp912oDNFuWyd
|
Current-head update: bounded the OpenCode model pool so provider/model outages do not loop until the 350-min job timeout. The branch now leads with direct openai/gpt-5-mini, skips direct OpenAI candidates when OPENAI_API_KEY is absent, limits the pool to one full candidate cycle, and sets a 7200s script budget with 2400s per-model timeout. Also fixed the existing Noema URL scheme regression exposed by the full local suite.\n\nLocal verification on c91e0ef:\n- python -m pytest tests -q -> 167 passed, 7 skipped\n- Git Bash: bash -n scripts/ci/run_opencode_review_model_pool.sh -> pass\n- Git Bash: bash -n scripts/ci/test_strix_quick_gate.sh -> pass\n- git diff --check -> pass\n\nRemote checks are still queued during the GitHub Actions hosted-runner start-delay incident. |
|
Current-head update for
Note: |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head51aa55c4b4e545066531ca8b43f35c5176ab9e9d. -
Head SHA:
51aa55c4b4e545066531ca8b43f35c5176ab9e9d -
Workflow run: 29033152279
-
Workflow attempt: 1
Coverage evidence
Coverage Evidence
- Head SHA:
51aa55c4b4e545066531ca8b43f35c5176ab9e9d - Required test evidence: supported repository test suites must pass.
- Required docstring evidence: repository-owned docstring gates must pass when configured; otherwise docstring coverage is advisory.
Python project dependencies (.)
Using CPython 3.12.3 interpreter at: /usr/bin/python3
Creating virtual environment at: .venv
Resolved 17 packages in 124ms
Downloading pygments (1.2MiB)
Downloaded pygments
Prepared 13 packages in 90ms
Installed 13 packages in 12ms
+ attrs==26.1.0
+ click==8.4.2
+ colorama==0.4.6
+ coverage==7.15.0
+ iniconfig==2.3.0
+ interrogate==1.7.0
+ packaging==26.2
+ pluggy==1.6.0
+ py==1.11.0
+ pygments==2.20.0
+ pytest==9.1.1
+ pytest-cov==7.1.0
+ tabulate==0.10.0
- Result: PASS
Python coverage with missing-line report (.)
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-9.1.1, pluggy-1.6.0
rootdir: /home/runner/work/.github/.github/pr-head
configfile: pyproject.toml
plugins: cov-7.1.0
collected 174 items
tests/test_assert_opencode_reasoning_effort.py ........ [ 4%]
tests/test_codeql_pr_workflow_contract.py . [ 5%]
tests/test_noema_review_gate.py ........... [ 11%]
tests/test_opencode_agent_contract.py ............. [ 18%]
tests/test_opencode_review_normalize_output.py ......................... [ 33%]
[ 33%]
tests/test_opencode_workflow_shell_syntax.py . [ 33%]
tests/test_pr_governance_audit_contract.py ... [ 35%]
tests/test_pr_review_fix_scheduler.py ................... [ 46%]
tests/test_pr_review_fix_scheduler_coverage.py .. [ 47%]
tests/test_pr_review_merge_scheduler.py ................................ [ 66%]
............................... [ 83%]
tests/test_render_opencode_prompt_template.py .... [ 86%]
tests/test_required_workflow_queue_contract.py ....... [ 90%]
tests/test_review_execution_contracts.py .. [ 91%]
tests/test_sandboxed_verify.py ......... [ 96%]
tests/test_sandboxed_web_e2e.py ...... [100%]
=============================== warnings summary ===============================
tests/test_assert_opencode_reasoning_effort.py::test_module_entrypoint_success
<frozen runpy>:128: RuntimeWarning: 'scripts.ci.assert_opencode_reasoning_effort' found in sys.modules after import of package 'scripts.ci', but prior to execution of 'scripts.ci.assert_opencode_reasoning_effort'; this may result in unpredictable behaviour
tests/test_render_opencode_prompt_template.py::test_module_entrypoint
<frozen runpy>:128: RuntimeWarning: 'scripts.ci.render_opencode_prompt_template' found in sys.modules after import of package 'scripts.ci', but prior to execution of 'scripts.ci.render_opencode_prompt_template'; this may result in unpredictable behaviour
tests/test_review_execution_contracts.py::test_discovers_package_managers_java_r_json_and_main
<frozen runpy>:128: RuntimeWarning: 'scripts.ci.review_execution_contracts' found in sys.modules after import of package 'scripts.ci', but prior to execution of 'scripts.ci.review_execution_contracts'; this may result in unpredictable behaviour
tests/test_sandboxed_verify.py::test_module_main_entrypoint
<frozen runpy>:128: RuntimeWarning: 'scripts.ci.sandboxed_verify' found in sys.modules after import of package 'scripts.ci', but prior to execution of 'scripts.ci.sandboxed_verify'; this may result in unpredictable behaviour
tests/test_sandboxed_web_e2e.py::test_module_import_and_main_entrypoint
<frozen runpy>:128: RuntimeWarning: 'scripts.ci.sandboxed_web_e2e' found in sys.modules after import of package 'scripts.ci', but prior to execution of 'scripts.ci.sandboxed_web_e2e'; this may result in unpredictable behaviour
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================= 174 passed, 5 warnings in 5.84s ========================
Name Stmts Miss Cover Missing
------------------------------------------------------------------------------
scripts/ci/assert_opencode_reasoning_effort.py 61 0 100%
scripts/ci/noema_review_gate.py 245 1 99% 303
scripts/ci/opencode_review_normalize_output.py 426 0 100%
scripts/ci/pr_review_autofix_context.py 124 0 100%
scripts/ci/pr_review_fix_scheduler.py 195 0 100%
scripts/ci/pr_review_merge_scheduler.py 1220 0 100%
scripts/ci/render_opencode_prompt_template.py 21 0 100%
scripts/ci/review_execution_contracts.py 201 0 100%
scripts/ci/sandboxed_verify.py 108 0 100%
scripts/ci/sandboxed_web_e2e.py 149 0 100%
------------------------------------------------------------------------------
TOTAL 2750 1 99%
Coverage failure: total of 99 is less than fail-under=100
- Result: FAIL (exit 2)
Python docstring coverage advisory
RESULT: PASSED (minimum: 100.0%, actual: 100.0%)
- Result: PASS
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script (3 files)"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script (3 files)"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test (2 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (2 files)"]
R3 --> V3["targeted test run"]
OpenCode Review Overview
Pull request overviewOpenCode reviewed the current-head bounded evidence and found no blocking issues. FindingsNo blocking findings. SummaryApproval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script (3 files)"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script (3 files)"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test (2 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (2 files)"]
R3 --> V3["targeted test run"]
|
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode reviewed the current-head bounded evidence and found no blocking issues.
Findings
No blocking findings.
Summary
Approval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Verification posture: CodeGraph evidence was initialized and bounded current-head evidence reviewed for changed-file evidence including .github/workflows/opencode-review.yml, scripts/ci/noema_review_gate.py, scripts/ci/run_opencode_review_model_pool.sh, scripts/ci/test_strix_quick_gate.sh, tests/test_noema_review_gate.py, and 1 more.
Linter/static: workflow/static review evidence is bounded by the current-head GitHub Checks gate and changed-file evidence.
TDD/regression: coverage execution evidence and focused changed hunks were reviewed from bounded-review-evidence.md.
Coverage: coverage execution evidence reports supported repository test suites passed.
Docstring coverage: coverage execution evidence reports configured repository docstring gates passed or docstring coverage was advisory.
DAG: CodeGraph/source-backed behavior map connects .github/workflows/opencode-review.yml to the affected review, runtime, or workflow path and required checks.
PoC/execution: coverage-evidence job executed on the current head and reported PASS.
DDD/domain: workflow and repository-governance invariants were reviewed against changed files in bounded evidence.
CDD/context: CodeGraph evidence, changed-file history, and focused hunks were reviewed from bounded-review-evidence.md.
Similar issues: changed-file history evidence was reviewed for comparable local precedents.
Claim/concept check: bounded evidence, repository source, current-head workflow evidence, and, where numeric, scientific, statistical, or literature-backed claims are affected, original-paper/formula evidence and parameter-recovery expectations were used for claims.
Standards search: standards and external-source checks are delegated to configured OpenCode web_search/Context7/DeepWiki sources when applicable; no evidence-backed standards blocker is present in bounded evidence.
Compatibility/convention: changed workflow/script conventions, object naming, and reserved-word safety for schema/API/config/code surfaces were checked in bounded evidence.
Breaking-change/backcompat: deployment evidence and changed-file history were checked for backward-compatibility risk.
Performance: changed surfaces were checked for performance risk in bounded evidence.
Developer experience: changed automation, review, test, setup, and maintenance surfaces were checked for helpful or obstructive DX impact in bounded evidence.
User experience: connected user, operator, API, CLI, documentation, review-comment, status-check, rendering, and workflow-reader behavior was checked for contradictions against code, docs, and tests in bounded evidence.
Visual/DOM: Playwright visual, DOM locator, ARIA snapshot, console, and responsive evidence were checked when a web UI surface was present; for non-web surfaces, API/CLI/log/docs/workflow interaction evidence was reviewed instead.
Accessibility/i18n: accessibility, localization, and human-readable text surfaces were checked where UI, CLI, API message, docs, logs, or review text changed.
Supply-chain/license: dependency, package, model, container, and external-tool changes were checked in bounded evidence.
Packaging: package, build, test, lint, and security contracts were checked in bounded evidence.
Security/privacy: workflow-token, review-gate, and repository-automation security/privacy boundaries were checked in bounded evidence.
- Result: APPROVE
- Reason: Coverage and docstring evidence PASS; workflow changes introduce a safe, bounded native OpenAI backend and fix the stalled review pool without exposing secrets.
- Head SHA:
8dd1def6a83234c283b47290631578b212e7d6ee - Workflow run: 29046404348
- Workflow attempt: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script (3 files)"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script (3 files)"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test (2 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (2 files)"]
R3 --> V3["targeted test run"]
1 participant
Org-wide merge freeze
The required
opencode-reviewcheck is failing 100% of recent runs across all repos. 12+ open PRs (3 repos) are stuck atREVIEW_REQUIRED, and every review run hangs to the 350-minute step timeout. No PR in the org can merge.Root cause
The embedded OpenCode config in
.github/workflows/opencode-review.ymlrouted every model in the pool through the singlegithub-modelsprovider:GitHub Models rate-limits (
Too many requests) and caps request bodies at ~4000 tokens (413 tokens_limit_reached). So no model in the candidate pool ever produced a review verdict — the pool churned through every fallback and exhausted the whole step without approving. The gate never opens.Fix
Give the reviewer a working, un-throttled backend by adding a native OpenAI provider (hits
api.openai.comdirectly using the orgOPENAI_API_KEYsecret) and making it the lead of the model pool. GitHub Models entries are kept unchanged as fallbacks.opencode.jsoncconfig: addedprovider.openai—npm: @ai-sdk/openai,baseURL: https://api.openai.com/v1,apiKey: {env:OPENAI_API_KEY}, withgpt-5andgpt-5-mini(reasoning: true,reasoningEffort: high, matchingassert_opencode_reasoning_effort.py). Added"openai"toenabled_providers.OPENCODE_MODEL_CANDIDATES: prependedopenai/gpt-5 openai/gpt-5-minias the lead candidates. The pool passes--modelexplicitly per attempt, so candidate order is the real lead — not the configmodeldefault. The full github-models list follows unchanged.OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}to both env blocks that invoke opencode with this config (the model-pool step and the approve / failed-check-diagnosis step) so{env:OPENAI_API_KEY}resolves.Why the config
model/small_modeldefaults were left on github-models/deepseekThey are overridden at runtime by the pool's explicit
--model, and static self-tests pin those exact strings. Changing them would break tests for zero runtime benefit. The "lead" is controlled entirely by the candidate list, which is what this PR reorders.Does not weaken the gate
This only changes which backend the reviewer talks to — the review contract, approve gate, agents, and permissions are untouched. If
OPENAI_API_KEYis absent or a direct call fails, the pool falls through to the existing github-models candidates exactly as today (at worst one wasted lead attempt before fallback).Cost implication
PR reviews will now consume the org OpenAI API key (billed to
api.openai.com) for the leadgpt-5/gpt-5-miniattempts on every reviewed PR, instead of the free-but-broken GitHub Models quota. This is the tradeoff that makes reviews actually complete. Reviewers should confirm this billing is acceptable before merge.Validation
python3 -c "import yaml; yaml.safe_load(...)"→ yaml okpython3 -m json.tool→ valid JSON;provider.openaipresent with correct baseURL/apiKey/models;enabled_providers = [openai, github-models].assert_opencode_reasoning_effort.py→ all candidates pass (incl.openai/gpt-5,openai/gpt-5-mini).test_strix_quick_gate.sh: every substring it pins that was already passing still passes; the full-order candidate asserts it "fails" on were already failing on the pre-change file (that test is out of sync and is not the merge gate).Needs review before merge
This gates all org merges. Please verify: (1) the org
OPENAI_API_KEYsecret is available to this workflow'spull_request_targetcontext, (2) the OpenAI billing cost is acceptable, and (3)gpt-5/gpt-5-miniare the intended model ids for the org's OpenAI account. Do not merge until confirmed.🤖 Generated with Claude Code