[codex] Fail OpenCode approve when review publish fails#272
Open
seonghobae wants to merge 13 commits into
Open
[codex] Fail OpenCode approve when review publish fails#272seonghobae wants to merge 13 commits into
seonghobae wants to merge 13 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the OpenCode review workflow to fail the APPROVE gate when publishing the approving GitHub pull review fails, removing the prior “rate-limit soft-pass” behavior that could leave branch protection requirements unsatisfied.
Changes:
- Remove the rate-limit detection/soft-pass path for APPROVE pull-review publication failures.
- Fail the workflow when pull-review publication fails (so the PR cannot appear “approved” without an actual GitHub approval review).
- Tighten the shell regression gate to assert the “skipped approval” success messaging is absent and the fail-closed message remains.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| scripts/ci/test_strix_quick_gate.sh | Updates regression assertions to ensure approval publication cannot be treated as success and that fail-closed messaging remains. |
| .github/workflows/opencode-review.yml | Removes rate-limit soft-pass logic for APPROVE review publication and fails the job when the pull review cannot be published. |
Comments suppressed due to low confidence (1)
.github/workflows/opencode-review.yml:2903
- On pull-review publication failure, the script emits
::endgroup::unconditionally, butemit_review_body_to_action_logonly opens a::group::forREQUEST_CHANGESandINLINE_COMMENT_PUBLISH_FAILED. ForAPPROVEfailures this can produce an unmatchedendgroupin Actions logs. Gate theendgroupemission on the same event types (or move group closing intoemit_review_body_to_action_log).
printf '::error::OpenCode could not publish the pull review for head %s, so the review state was not changed.\n' "$HEAD_SHA"
echo "::endgroup::"
exit 1
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Pull request overview
OpenCode reviewed the current-head mergeability evidence and changed-file flow before approval, then found merge conflicts on the affected path.
Findings
1. HIGH Merge Conflict Guidance - Resolve the PR branch against the latest base branch
- Problem: GitHub reports mergeStateStatus
DIRTYfor this pull request. - Root cause: Branch
codex/fail-approve-publish-without-reviewcannot be merged cleanly intomain; the changed-file flow below shows which review/runtime path is blocked by the conflict. - Fix: Merge or rebase the latest
mainintocodex/fail-approve-publish-without-review, resolve conflict markers in the PR branch, rerun the focused checks, and push the same branch. - Repair commands:
gh pr checkout 272 --repo ContextualWisdomLab/.github
git fetch origin main
git merge --no-ff origin/main # or: git rebase origin/main
git status --short
# resolve files, then git add <resolved-files>
# merge path: git commit
# rebase path: git rebase --continue
git push origin HEAD:codex/fail-approve-publish-without-review
# rebase path only: git push --force-with-lease origin HEAD:codex/fail-approve-publish-without-review- Regression test: Keep OpenCode approval gated on mergeability so model-output failures cannot approve a conflicted PR.
Merge Conflict Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> Conflict["Merge conflict blocks this path"]
Conflict --> V1["actionlint plus required checks"]
Evidence --> S2["CI script: test_strix_quick_gate.sh"]
S2 --> I2["review and security gate shell path"]
I2 --> Conflict["Merge conflict blocks this path"]
Conflict --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_agent_contract.py"]
S3 --> I3["regression suite"]
I3 --> Conflict["Merge conflict blocks this path"]
Conflict --> V3["targeted test run"]
- Result: REQUEST_CHANGES
- Reason: mergeStateStatus is
DIRTY; mergeable isCONFLICTING. - Head SHA:
a392b9f769f8f77b9ba5eb75d591fea6e0ba9e02 - Workflow run: 28503462405
- Workflow attempt: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> Conflict["Merge conflict blocks this path"]
Conflict --> V1["actionlint plus required checks"]
Evidence --> S2["CI script: test_strix_quick_gate.sh"]
S2 --> I2["review and security gate shell path"]
I2 --> Conflict["Merge conflict blocks this path"]
Conflict --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_agent_contract.py"]
S3 --> I3["regression suite"]
I3 --> Conflict["Merge conflict blocks this path"]
Conflict --> V3["targeted test run"]
Contributor
OpenCode Review Overview
Pull request overviewOpenCode cannot approve yet because required coverage evidence did not pass. Review outcome1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
Coverage evidenceCoverage Evidence
Python project dependencies (.)
Python coverage with missing-line report (.)
Python docstring coverage advisory
Coverage Decision
Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script: test_strix_quick_gate.sh"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script: test_strix_quick_gate.sh"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_agent_contract.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_opencode_agent_contract.py"]
R3 --> V3["targeted test run"]
|
Contributor
Author
|
@copilot resolve the merge conflicts in this pull request |
Contributor
Resolved the merge conflicts and merged |
Contributor
There was a problem hiding this comment.
Pull request overview
OpenCode exhausted the configured model pool without a usable current-head review conclusion. This is not approval evidence, so the PR is blocked until a source-backed review can establish approval sufficiency or identify concrete fixes.
Findings
1. HIGH review evidence:1 - OpenCode could not establish approval sufficiency
- Problem: every configured model path failed to produce a usable current-head control block.
- Root cause: model execution, timeout, export, normalization, or approval-gate validation did not complete after exponential retry across the configured model pool.
- Impact: approving from deterministic check state alone would miss PR-intent mismatches, missing files, edge-case bugs, robustness gaps, UX/DX regressions, security issues, and CodeGraph-backed base/head flow changes.
- Fix: rerun OpenCode after model availability recovers, or update the PR with the missing files, tests, docs, generated artifacts, and verification evidence needed for a source-backed review conclusion.
- Regression test: keep the approval gate posting REQUEST_CHANGES, not APPROVE or check-only failure, when no model produces a valid current-head review.
Summary
- Result: REQUEST_CHANGES
- Reason: coverage-evidence passed and peer GitHub Checks completed without failures, but no model produced a valid review control block.
- Deterministic evidence checked but not used for approval: current-head changed-file evidence (.github/workflows/opencode-review.yml, scripts/ci/test_strix_quick_gate.sh, tests/test_opencode_agent_contract.py); coverage-evidence result success; peer checks from statusCheckRollup excluding this OpenCode check.
- Model outcome: model_pool=exhausted; selected_model=none.
- Head SHA:
063685c7c0be54ebb55ec946c34a0739be552c6f - Workflow run: 28512066276
- Workflow attempt: 2
No PR approval was posted because model-output failure is not evidence that the PR has no blockers.
Inline comment note: OpenCode could not find an added RIGHT-side diff line for this PR, so the model-exhaustion blocker is attached to the PR review body instead of a file line.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script: test_strix_quick_gate.sh"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script: test_strix_quick_gate.sh"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_agent_contract.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_opencode_agent_contract.py"]
R3 --> V3["targeted test run"]
Contributor
Author
@copilot 고칩시다. re-kick을 하든, exponential retry를 하든.. 이렇게 하면 뭐 나더러 어쩌라고.. |
Contributor
반영했습니다. 모델 풀 결과가 |
Contributor
There was a problem hiding this comment.
Pull request overview
OpenCode exhausted the configured model pool without a usable current-head review conclusion. This is not approval evidence, so the PR is blocked until a source-backed review can establish approval sufficiency or identify concrete fixes.
Findings
1. HIGH review evidence:1 - OpenCode could not establish approval sufficiency
- Problem: every configured model path failed to produce a usable current-head control block.
- Root cause: model execution, timeout, export, normalization, or approval-gate validation did not complete after exponential retry across the configured model pool.
- Impact: approving from deterministic check state alone would miss PR-intent mismatches, missing files, edge-case bugs, robustness gaps, UX/DX regressions, security issues, and CodeGraph-backed base/head flow changes.
- Fix: rerun OpenCode after model availability recovers, or update the PR with the missing files, tests, docs, generated artifacts, and verification evidence needed for a source-backed review conclusion.
- Regression test: keep the approval gate posting REQUEST_CHANGES, not APPROVE or check-only failure, when no model produces a valid current-head review.
Summary
- Result: REQUEST_CHANGES
- Reason: coverage-evidence passed and peer GitHub Checks completed without failures, but no model produced a valid review control block.
- Deterministic evidence checked but not used for approval: current-head changed-file evidence (.github/workflows/opencode-review.yml, scripts/ci/test_strix_quick_gate.sh, tests/test_opencode_agent_contract.py); coverage-evidence result success; peer checks from statusCheckRollup excluding this OpenCode check.
- Model outcome: model_pool=exhausted; selected_model=none.
- Head SHA:
1224db9dd75437f42e2b98f38107622a23d7cd31 - Workflow run: 28513538624
- Workflow attempt: 2
No PR approval was posted because model-output failure is not evidence that the PR has no blockers.
Inline comment note: OpenCode could not find an added RIGHT-side diff line for this PR, so the model-exhaustion blocker is attached to the PR review body instead of a file line.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script: test_strix_quick_gate.sh"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script: test_strix_quick_gate.sh"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_agent_contract.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_opencode_agent_contract.py"]
R3 --> V3["targeted test run"]
Contributor
Author
@copilot 모델 부족이면 계속 exponential sleep 해서라도 반드시 성공시키자 |
Contributor
반영했습니다. 모델 풀 결과가 |
Contributor
There was a problem hiding this comment.
Pull request overview
OpenCode exhausted the configured model pool without a usable current-head review conclusion. This is not approval evidence, so the PR is blocked until a source-backed review can establish approval sufficiency or identify concrete fixes.
Findings
1. HIGH review evidence:1 - OpenCode could not establish approval sufficiency
- Problem: every configured model path failed to produce a usable current-head control block.
- Root cause: model execution, timeout, export, normalization, or approval-gate validation did not complete after exponential retry across the configured model pool.
- Impact: approving from deterministic check state alone would miss PR-intent mismatches, missing files, edge-case bugs, robustness gaps, UX/DX regressions, security issues, and CodeGraph-backed base/head flow changes.
- Fix: rerun OpenCode after model availability recovers, or update the PR with the missing files, tests, docs, generated artifacts, and verification evidence needed for a source-backed review conclusion.
- Regression test: keep the approval gate posting REQUEST_CHANGES, not APPROVE or check-only failure, when no model produces a valid current-head review.
Summary
- Result: REQUEST_CHANGES
- Reason: coverage-evidence passed and peer GitHub Checks completed without failures, but no model produced a valid review control block.
- Deterministic evidence checked but not used for approval: current-head changed-file evidence (.github/workflows/opencode-review.yml, scripts/ci/test_strix_quick_gate.sh, tests/test_opencode_agent_contract.py); coverage-evidence result success; peer checks from statusCheckRollup excluding this OpenCode check.
- Model outcome: model_pool=exhausted; selected_model=none.
- Head SHA:
3c3bf6dc619682f82b284ca760009ac154d55d13 - Workflow run: 28528202916
- Workflow attempt: 1
No PR approval was posted because model-output failure is not evidence that the PR has no blockers.
Inline comment note: OpenCode could not find an added RIGHT-side diff line for this PR, so the model-exhaustion blocker is attached to the PR review body instead of a file line.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script: test_strix_quick_gate.sh"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script: test_strix_quick_gate.sh"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_agent_contract.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_opencode_agent_contract.py"]
R3 --> V3["targeted test run"]
Contributor
There was a problem hiding this comment.
Pull request overview
OpenCode exhausted the configured model pool without a usable current-head review conclusion. This is not approval evidence, so the PR is blocked until a source-backed review can establish approval sufficiency or identify concrete fixes.
Findings
1. HIGH review evidence:1 - OpenCode could not establish approval sufficiency
- Problem: every configured model path failed to produce a usable current-head control block.
- Root cause: model execution, timeout, export, normalization, or approval-gate validation did not complete after exponential retry across the configured model pool.
- Impact: approving from deterministic check state alone would miss PR-intent mismatches, missing files, edge-case bugs, robustness gaps, UX/DX regressions, security issues, and CodeGraph-backed base/head flow changes.
- Fix: rerun OpenCode after model availability recovers, or update the PR with the missing files, tests, docs, generated artifacts, and verification evidence needed for a source-backed review conclusion.
- Regression test: keep the approval gate posting REQUEST_CHANGES, not APPROVE or check-only failure, when no model produces a valid current-head review.
Summary
- Result: REQUEST_CHANGES
- Reason: coverage-evidence passed and peer GitHub Checks completed without failures, but no model produced a valid review control block.
- Deterministic evidence checked but not used for approval: current-head changed-file evidence (.github/workflows/opencode-review.yml, scripts/ci/test_strix_quick_gate.sh, tests/test_opencode_agent_contract.py); coverage-evidence result success; peer checks from statusCheckRollup excluding this OpenCode check.
- Model outcome: model_pool=exhausted; selected_model=none.
- Head SHA:
3c3bf6dc619682f82b284ca760009ac154d55d13 - Workflow run: 28541870020
- Workflow attempt: 1
No PR approval was posted because model-output failure is not evidence that the PR has no blockers.
Inline comment note: OpenCode could not find an added RIGHT-side diff line for this PR, so the model-exhaustion blocker is attached to the PR review body instead of a file line.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script: test_strix_quick_gate.sh"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script: test_strix_quick_gate.sh"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_agent_contract.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_opencode_agent_contract.py"]
R3 --> V3["targeted test run"]
Contributor
There was a problem hiding this comment.
Pull request overview
OpenCode exhausted the configured model pool without a usable current-head review conclusion. This is not approval evidence, so the PR is blocked until a source-backed review can establish approval sufficiency or identify concrete fixes.
Findings
1. HIGH review evidence:1 - OpenCode could not establish approval sufficiency
- Problem: every configured model path failed to produce a usable current-head control block.
- Root cause: model execution, timeout, export, normalization, or approval-gate validation did not complete after exponential retry across the configured model pool.
- Impact: approving from deterministic check state alone would miss PR-intent mismatches, missing files, edge-case bugs, robustness gaps, UX/DX regressions, security issues, and CodeGraph-backed base/head flow changes.
- Fix: rerun OpenCode after model availability recovers, or update the PR with the missing files, tests, docs, generated artifacts, and verification evidence needed for a source-backed review conclusion.
- Regression test: keep the approval gate posting REQUEST_CHANGES, not APPROVE or check-only failure, when no model produces a valid current-head review.
Summary
- Result: REQUEST_CHANGES
- Reason: coverage-evidence passed and peer GitHub Checks completed without failures, but no model produced a valid review control block.
- Deterministic evidence checked but not used for approval: current-head changed-file evidence (.github/workflows/opencode-review.yml, scripts/ci/test_strix_quick_gate.sh, tests/test_opencode_agent_contract.py); coverage-evidence result success; peer checks from statusCheckRollup excluding this OpenCode check.
- Model outcome: model_pool=exhausted; selected_model=none.
- Head SHA:
3c3bf6dc619682f82b284ca760009ac154d55d13 - Workflow run: 28552741347
- Workflow attempt: 1
No PR approval was posted because model-output failure is not evidence that the PR has no blockers.
Inline comment note: OpenCode could not find an added RIGHT-side diff line for this PR, so the model-exhaustion blocker is attached to the PR review body instead of a file line.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script: test_strix_quick_gate.sh"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script: test_strix_quick_gate.sh"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_agent_contract.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_opencode_agent_contract.py"]
R3 --> V3["targeted test run"]
Contributor
There was a problem hiding this comment.
Pull request overview
OpenCode exhausted the configured model pool without a usable current-head review conclusion. This is not approval evidence, so the PR is blocked until a source-backed review can establish approval sufficiency or identify concrete fixes.
Findings
1. HIGH review evidence:1 - OpenCode could not establish approval sufficiency
- Problem: every configured model path failed to produce a usable current-head control block.
- Root cause: model execution, timeout, export, normalization, or approval-gate validation did not complete after exponential retry across the configured model pool.
- Impact: approving from deterministic check state alone would miss PR-intent mismatches, missing files, edge-case bugs, robustness gaps, UX/DX regressions, security issues, and CodeGraph-backed base/head flow changes.
- Fix: rerun OpenCode after model availability recovers, or update the PR with the missing files, tests, docs, generated artifacts, and verification evidence needed for a source-backed review conclusion.
- Regression test: keep the approval gate posting REQUEST_CHANGES, not APPROVE or check-only failure, when no model produces a valid current-head review.
Summary
- Result: REQUEST_CHANGES
- Reason: coverage-evidence passed and peer GitHub Checks completed without failures, but no model produced a valid review control block.
- Deterministic evidence checked but not used for approval: current-head changed-file evidence (.github/workflows/opencode-review.yml, scripts/ci/test_strix_quick_gate.sh, tests/test_opencode_agent_contract.py); coverage-evidence result success; peer checks from statusCheckRollup excluding this OpenCode check.
- Model outcome: model_pool=exhausted; selected_model=none.
- Head SHA:
3c3bf6dc619682f82b284ca760009ac154d55d13 - Workflow run: 28555964097
- Workflow attempt: 1
No PR approval was posted because model-output failure is not evidence that the PR has no blockers.
Inline comment note: OpenCode could not find an added RIGHT-side diff line for this PR, so the model-exhaustion blocker is attached to the PR review body instead of a file line.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script: test_strix_quick_gate.sh"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script: test_strix_quick_gate.sh"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_agent_contract.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_opencode_agent_contract.py"]
R3 --> V3["targeted test run"]
Contributor
There was a problem hiding this comment.
Pull request overview
OpenCode exhausted the configured model pool without a usable current-head review conclusion. This is not approval evidence, so the PR is blocked until a source-backed review can establish approval sufficiency or identify concrete fixes.
Findings
1. HIGH review evidence:1 - OpenCode could not establish approval sufficiency
- Problem: every configured model path failed to produce a usable current-head control block.
- Root cause: model execution, timeout, export, normalization, or approval-gate validation did not complete after exponential retry across the configured model pool.
- Impact: approving from deterministic check state alone would miss PR-intent mismatches, missing files, edge-case bugs, robustness gaps, UX/DX regressions, security issues, and CodeGraph-backed base/head flow changes.
- Fix: rerun OpenCode after model availability recovers, or update the PR with the missing files, tests, docs, generated artifacts, and verification evidence needed for a source-backed review conclusion.
- Regression test: keep the approval gate posting REQUEST_CHANGES, not APPROVE or check-only failure, when no model produces a valid current-head review.
Summary
- Result: REQUEST_CHANGES
- Reason: coverage-evidence passed and peer GitHub Checks completed without failures, but no model produced a valid review control block.
- Deterministic evidence checked but not used for approval: current-head changed-file evidence (.github/workflows/opencode-review.yml, scripts/ci/test_strix_quick_gate.sh, tests/test_opencode_agent_contract.py); coverage-evidence result success; peer checks from statusCheckRollup excluding this OpenCode check.
- Model outcome: model_pool=exhausted; selected_model=none.
- Head SHA:
3c3bf6dc619682f82b284ca760009ac154d55d13 - Workflow run: 28558954038
- Workflow attempt: 1
No PR approval was posted because model-output failure is not evidence that the PR has no blockers.
Inline comment note: OpenCode could not find an added RIGHT-side diff line for this PR, so the model-exhaustion blocker is attached to the PR review body instead of a file line.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script: test_strix_quick_gate.sh"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script: test_strix_quick_gate.sh"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_agent_contract.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_opencode_agent_contract.py"]
R3 --> V3["targeted test run"]
Contributor
There was a problem hiding this comment.
Pull request overview
OpenCode exhausted the configured model pool without a usable current-head review conclusion. This is not approval evidence, so the PR is blocked until a source-backed review can establish approval sufficiency or identify concrete fixes.
Findings
1. HIGH review evidence:1 - OpenCode could not establish approval sufficiency
- Problem: every configured model path failed to produce a usable current-head control block.
- Root cause: model execution, timeout, export, normalization, or approval-gate validation did not complete after exponential retry across the configured model pool.
- Impact: approving from deterministic check state alone would miss PR-intent mismatches, missing files, edge-case bugs, robustness gaps, UX/DX regressions, security issues, and CodeGraph-backed base/head flow changes.
- Fix: rerun OpenCode after model availability recovers, or update the PR with the missing files, tests, docs, generated artifacts, and verification evidence needed for a source-backed review conclusion.
- Regression test: keep the approval gate posting REQUEST_CHANGES, not APPROVE or check-only failure, when no model produces a valid current-head review.
Summary
- Result: REQUEST_CHANGES
- Reason: coverage-evidence passed and peer GitHub Checks completed without failures, but no model produced a valid review control block.
- Deterministic evidence checked but not used for approval: current-head changed-file evidence (.github/workflows/opencode-review.yml, scripts/ci/test_strix_quick_gate.sh, tests/test_opencode_agent_contract.py); coverage-evidence result success; peer checks from statusCheckRollup excluding this OpenCode check.
- Model outcome: model_pool=exhausted; selected_model=none.
- Head SHA:
3c3bf6dc619682f82b284ca760009ac154d55d13 - Workflow run: 28515588277
- Workflow attempt: 2
No PR approval was posted because model-output failure is not evidence that the PR has no blockers.
Inline comment note: OpenCode could not find an added RIGHT-side diff line for this PR, so the model-exhaustion blocker is attached to the PR review body instead of a file line.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script: test_strix_quick_gate.sh"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script: test_strix_quick_gate.sh"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_agent_contract.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_opencode_agent_contract.py"]
R3 --> V3["targeted test run"]
Merge latest main into codex/fail-approve-publish-without-review to resolve the reported merge conflict (mergeStateStatus DIRTY). Kept the PR's fail-closed intent (removed rate-limit soft-pass, gated the review log ::endgroup:: on events that opened a group, added model-pool re-kick) and took main's evolved model-pool timeout/candidate assertions where they superseded the branch snapshot. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C2Mig78TxShv22uBcr7pCo
Contributor
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head35464ebfb5cb12e169c8af9e7b02e170980a5e04. -
Head SHA:
35464ebfb5cb12e169c8af9e7b02e170980a5e04 -
Workflow run: 28859251297
-
Workflow attempt: 1
Coverage evidence
Coverage Evidence
- Head SHA:
35464ebfb5cb12e169c8af9e7b02e170980a5e04 - Required test evidence: supported repository test suites must pass.
- Required docstring evidence: repository-owned docstring gates must pass when configured; otherwise docstring coverage is advisory.
Python project dependencies (.)
Using CPython 3.12.3 interpreter at: /usr/bin/python3
Creating virtual environment at: .venv
Resolved 17 packages in 128ms
Downloading pygments (1.2MiB)
Downloaded pygments
Prepared 13 packages in 148ms
Installed 13 packages in 12ms
+ attrs==26.1.0
+ click==8.4.2
+ colorama==0.4.6
+ coverage==7.15.0
+ iniconfig==2.3.0
+ interrogate==1.7.0
+ packaging==26.2
+ pluggy==1.6.0
+ py==1.11.0
+ pygments==2.20.0
+ pytest==9.1.1
+ pytest-cov==7.1.0
+ tabulate==0.10.0
- Result: PASS
Python coverage with missing-line report (.)
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-9.1.1, pluggy-1.6.0
rootdir: /home/runner/work/.github/.github/pr-head
configfile: pyproject.toml
plugins: cov-7.1.0
collected 174 items
tests/test_assert_opencode_reasoning_effort.py ........ [ 4%]
tests/test_codeql_pr_workflow_contract.py . [ 5%]
tests/test_noema_review_gate.py .......F... [ 11%]
tests/test_opencode_agent_contract.py .....F...F... [ 18%]
tests/test_opencode_review_normalize_output.py ......................... [ 33%]
[ 33%]
tests/test_opencode_workflow_shell_syntax.py . [ 33%]
tests/test_pr_governance_audit_contract.py ... [ 35%]
tests/test_pr_review_fix_scheduler.py ................... [ 46%]
tests/test_pr_review_fix_scheduler_coverage.py .. [ 47%]
tests/test_pr_review_merge_scheduler.py ................................ [ 66%]
............................... [ 83%]
tests/test_render_opencode_prompt_template.py .... [ 86%]
tests/test_required_workflow_queue_contract.py ....... [ 90%]
tests/test_review_execution_contracts.py .. [ 91%]
tests/test_sandboxed_verify.py ......... [ 96%]
tests/test_sandboxed_web_e2e.py ...... [100%]
=================================== FAILURES ===================================
_______________ test_call_llm_handles_configuration_and_verdicts _______________
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7fb3d8655e50>
def test_call_llm_handles_configuration_and_verdicts(monkeypatch):
pr = make_pr()
monkeypatch.delenv("NOEMA_LLM_API_URL", raising=False)
monkeypatch.delenv("NOEMA_LLM_API_KEY", raising=False)
assert noema.call_llm("owner/repo", 1, pr, "diff", False) is None
monkeypatch.setenv("NOEMA_LLM_API_URL", "file:///etc/passwd")
monkeypatch.setenv("NOEMA_LLM_API_KEY", "secret")
> with pytest.raises(ValueError, match="must start with http:// or https://"):
E AssertionError: Regex pattern did not match.
E Expected regex: 'must start with http:// or https://'
E Actual message: 'URL scheme must be http or https'
tests/test_noema_review_gate.py:209: AssertionError
----------------------------- Captured stdout call -----------------------------
Noema LLM review unavailable: NOEMA_LLM_API_URL or NOEMA_LLM_API_KEY is not configured.
___________ test_workflow_provisions_sandbox_tool_and_reviewer_agent ___________
def test_workflow_provisions_sandbox_tool_and_reviewer_agent():
"""Guard the runtime OpenCode workspace, not only repo-local config."""
workflow = Path(".github/workflows/opencode-review.yml").read_text(
encoding="utf-8"
)
assert "code-reviewer-prompt.md" in workflow
assert "sandboxed_verify.py" in workflow
assert "sandboxed_web_e2e.py" in workflow
assert "review_execution_contracts.py" in workflow
assert "SANDBOXED_VERIFY_RESULT" in workflow
assert "SANDBOXED_WEB_E2E_RESULT" in workflow
assert "Docker Compose, devcontainer, Nix, or temporary package-install sandbox" in workflow
assert "scientific, statistical, simulation" in workflow
assert "skewed true" in workflow
assert "object naming" in workflow
assert "connected code paths, rendering paths" in workflow
assert "CHECK_LOOKUP_GH_TOKEN" in workflow
assert "retrying with workflow github token" in workflow
assert 'review_write_token="$GH_TOKEN"' in workflow
assert 'review_write_token="$OPENCODE_APP_TOKEN"' in workflow
assert 'review_write_token="$CHECK_LOOKUP_GH_TOKEN"' in workflow
assert 'review_write_token="${OPENCODE_APP_TOKEN:-$GH_TOKEN}"' not in workflow
assert "Review execution contracts" in workflow
assert "Accessibility/i18n:" in workflow
assert "Supply-chain/license:" in workflow
assert "Packaging:" in workflow
assert 'gsub("`"; "\'")' not in workflow
assert 'gsub("`"; "'")' in workflow
assert '"code-reviewer"' in workflow
assert workflow.count('"reasoningEffort": "high"') >= 10
assert '"task": "allow"' in workflow
assert 'cat >"$prompt_file" <<EOF' not in workflow
assert 'cat >"$prompt_file" <<\'EOF\'' not in workflow
assert "Run OpenCode PR Review model pool" in workflow
assert "opencode_review_model_pool" in workflow
assert "run_opencode_review_model_pool.sh" in workflow
assert "rekick_model_pool_on_exhaustion" in workflow
assert "format('pr-{0}', github.event.pull_request.number)" in workflow
assert "format('pr-{0}-{1}', github.event.pull_request.number, github.event.pull_request.head.sha)" not in workflow
assert "OPENCODE_MODEL_CANDIDATES" in workflow
model_pool_runner = Path("scripts/ci/run_opencode_review_model_pool.sh").read_text(encoding="utf-8")
assert "assert_reasoning_effort_for_candidate" in model_pool_runner
assert "assert_opencode_reasoning_effort.py" in model_pool_runner
assert "--config opencode.jsonc" in model_pool_runner
reasoning_effort_guard = Path("scripts/ci/assert_opencode_reasoning_effort.py").read_text(encoding="utf-8")
assert 'options.reasoningEffort=high' in reasoning_effort_guard
assert 'variants.high.reasoningEffort=high' in reasoning_effort_guard
assert "deepseek/deepseek-r1" in reasoning_effort_guard
assert "--config \"$OPENCODE_REVIEW_WORKDIR/opencode.jsonc\"" in workflow
assert 'timeout --kill-after=15s "${export_timeout_seconds}s" opencode export' in model_pool_runner
assert "session export did not complete within %ss" in model_pool_runner
assert "Follow the complete review contract" in model_pool_runner
assert "packet-first entry point" in model_pool_runner
assert "Current-head evidence packet" in model_pool_runner
assert "not a generic model-exhaustion message" in model_pool_runner
assert "is_context_overflow_failure" in model_pool_runner
assert "tokens_limit_reached" in model_pool_runner
assert "skipping remaining attempts for this model" in model_pool_runner
assert "approve_low_risk_review_fallback_after_model_exhaustion" not in workflow
assert "changed_file_is_low_risk_review_fallback" not in workflow
assert "approve_central_review_process_fallback" not in workflow
assert "opencode.jsonc | \\" in workflow
assert "scripts/ci/run_opencode_review_model_pool.sh | \\" in workflow
assert "tests/test_opencode_agent_contract.py | \\" in workflow
assert "ContextualWisdomLab/appguardrail:scripts/ci/collect_org_security_failures.py" in workflow
assert "ContextualWisdomLab/appguardrail:.github/workflows/org-security-failure-collector.yml" in workflow
assert "ContextualWisdomLab/appguardrail:tests/test_org_security_failure_collector.py" in workflow
assert "appguardrail org-security failure collector" in workflow
assert 'max_changed_count=3' in workflow
assert "changed_count\" -gt \"$max_changed_count\"" in workflow
assert "steps.central_review_process_fallback_scope.outputs.eligible != 'true'" not in workflow
assert workflow.index("Detect central review-process scope") < workflow.index(
"Initialize CodeGraph index for OpenCode"
)
assert "CENTRAL_REVIEW_PROCESS_FALLBACK_ELIGIBLE" in workflow
assert "CENTRAL_REVIEW_PROCESS_FALLBACK_SCOPE_LABEL" in workflow
assert "model pool was intentionally skipped" not in workflow
assert "deterministic fallback" not in workflow
assert "production source 또는 package manifest 변경이 없습니다" not in workflow
assert "request_changes_for_coverage_evidence_failure" in workflow
assert '"## Review outcome"' in workflow
assert '"## Check outcome"' not in workflow
assert "publish REQUEST_CHANGES when coverage-evidence blocker states" in workflow
> assert re.search(r"opencode-review-target:[\s\S]{0,240}timeout-minutes: 360", workflow)
E assert None
E + where None = <function search at 0x7fb3da494220>('opencode-review-target:[\\s\\S]{0,240}timeout-minutes: 360', 'name: Required OpenCode Review\n\non:\n pull_request_target:\n types: [opened, synchronize, reopened, ready_for_r... The scheduled and PR-event scheduler paths remain authoritative.\\n\' "$GH_REPOSITORY" "$base_branch"\n fi\n')
E + where <function search at 0x7fb3da494220> = re.search
tests/test_opencode_agent_contract.py:286: AssertionError
__________ test_merge_scheduler_uses_escalating_mutation_credentials ___________
def test_merge_scheduler_uses_escalating_mutation_credentials():
"""Guard immediate merge/update execution credentials for central scheduling."""
workflow = Path(".github/workflows/pr-review-merge-scheduler.yml").read_text(
encoding="utf-8"
)
assert "id-token: write" in workflow
assert "Exchange OpenCode app token for scheduler mutations" in workflow
assert "secrets.PR_REVIEW_MERGE_TOKEN" in workflow
assert "secrets.OPENCODE_APPROVE_TOKEN" in workflow
assert "steps.scheduler_app_token.outputs.token" in workflow
assert "SCHEDULER_READ_TOKEN: ${{ github.token }}" in workflow
assert "SCHEDULER_MUTATION_TOKEN_SOURCE" in workflow
> assert 'default: "-1"' in workflow
E assert 'default: "-1"' in 'name: Required PR Review Merge Scheduler\n\non:\n push:\n branches: [main, develop, master]\n pull_request_targe... args+=(--no-update-branches)\n fi\n python3 scripts/ci/pr_review_merge_scheduler.py "${args[@]}"\n'
tests/test_opencode_agent_contract.py:421: AssertionError
=============================== warnings summary ===============================
tests/test_assert_opencode_reasoning_effort.py::test_module_entrypoint_success
<frozen runpy>:128: RuntimeWarning: 'scripts.ci.assert_opencode_reasoning_effort' found in sys.modules after import of package 'scripts.ci', but prior to execution of 'scripts.ci.assert_opencode_reasoning_effort'; this may result in unpredictable behaviour
tests/test_render_opencode_prompt_template.py::test_module_entrypoint
<frozen runpy>:128: RuntimeWarning: 'scripts.ci.render_opencode_prompt_template' found in sys.modules after import of package 'scripts.ci', but prior to execution of 'scripts.ci.render_opencode_prompt_template'; this may result in unpredictable behaviour
tests/test_review_execution_contracts.py::test_discovers_package_managers_java_r_json_and_main
<frozen runpy>:128: RuntimeWarning: 'scripts.ci.review_execution_contracts' found in sys.modules after import of package 'scripts.ci', but prior to execution of 'scripts.ci.review_execution_contracts'; this may result in unpredictable behaviour
tests/test_sandboxed_verify.py::test_module_main_entrypoint
<frozen runpy>:128: RuntimeWarning: 'scripts.ci.sandboxed_verify' found in sys.modules after import of package 'scripts.ci', but prior to execution of 'scripts.ci.sandboxed_verify'; this may result in unpredictable behaviour
tests/test_sandboxed_web_e2e.py::test_module_import_and_main_entrypoint
<frozen runpy>:128: RuntimeWarning: 'scripts.ci.sandboxed_web_e2e' found in sys.modules after import of package 'scripts.ci', but prior to execution of 'scripts.ci.sandboxed_web_e2e'; this may result in unpredictable behaviour
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/test_noema_review_gate.py::test_call_llm_handles_configuration_and_verdicts - AssertionError: Regex pattern did not match.
Expected regex: 'must start with http:// or https://'
Actual message: 'URL scheme must be http or https'
FAILED tests/test_opencode_agent_contract.py::test_workflow_provisions_sandbox_tool_and_reviewer_agent - assert None
+ where None = <function search at 0x7fb3da494220>('opencode-review-target:[\\s\\S]{0,240}timeout-minutes: 360', 'name: Required OpenCode Review\n\non:\n pull_request_target:\n types: [opened, synchronize, reopened, ready_for_r... The scheduled and PR-event scheduler paths remain authoritative.\\n\' "$GH_REPOSITORY" "$base_branch"\n fi\n')
+ where <function search at 0x7fb3da494220> = re.search
FAILED tests/test_opencode_agent_contract.py::test_merge_scheduler_uses_escalating_mutation_credentials - assert 'default: "-1"' in 'name: Required PR Review Merge Scheduler\n\non:\n push:\n branches: [main, develop, master]\n pull_request_targe... args+=(--no-update-branches)\n fi\n python3 scripts/ci/pr_review_merge_scheduler.py "${args[@]}"\n'
================== 3 failed, 171 passed, 5 warnings in 6.11s ===================
- Result: FAIL (exit 1)
Python docstring coverage advisory
RESULT: PASSED (minimum: 100.0%, actual: 100.0%)
- Result: PASS
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script: test_strix_quick_gate.sh"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script: test_strix_quick_gate.sh"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_agent_contract.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_opencode_agent_contract.py"]
R3 --> V3["targeted test run"]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Verification
Context
pg-erd-cloud #406 has an OpenCode APPROVE overview but no approving pull review, so branch protection still blocks merge. The required workflow must not hide that state.