Skip to content

5CtptaYtdmyQe9suv9KGE3qhaacizqF3j4FxHQJAwrGWguxs reliability-hybrid agent: correctness + integration gates - #1558

Open
dataCenter430 wants to merge 1 commit into
unarbos:mainfrom
dataCenter430:miner/reliability-hybrid
Open

dataCenter430 wants to merge 1 commit into
unarbos:mainfrom
dataCenter430:miner/reliability-hybrid

Conversation

@dataCenter430

Copy link
Copy Markdown
Contributor

Adds a pre-final correctness gate (undefined-name / missing-import detection for Python + React hooks) and an integration gate flagging orphaned new files, plus v20 prompt/ranking refinements.

agent.py only — +364/-15.
Scope guard, pyflakes smoke, stdlib-only imports verified locally.

@github-actions

Copy link
Copy Markdown

OpenRouter PR Judge

Verdict: PASS
Model: anthropic/claude-opus-4.7
Threshold: 70

Score Value
Overall 78
Real edit 80
Safety 95
Scope 90
Contract 95

Summary

This PR adds two new mechanical refinement gates to the solve loop: a Python AST-based undefined-name check and a React-hook missing-import check, plus an orphaned-new-file integration advisory. It also adds a 'soft nudge' mid-loop prompt fired earlier and more gently than the existing hail-mary, tweaks the mid-loop fraction from 0.55 to 0.50, expands the stopword/generic-hook lists for identifier extraction, adds a type-aware cue to the self-check prompt, and widens the recently-observed-paths window. The solve() signature, return shape, and validator boundaries are preserved.

Static Checks

  • No static findings.

Judge Reasons

  • Adds concrete new mechanisms (_check_correctness, _check_undefined_names_py, _check_react_hook_imports, _check_orphaned_new_files) wired into the refinement gate chain with a new MAX_CORRECTNESS_FIX_TURNS counter.
  • Soft-nudge path is a new mid-loop behavior with its own gating constants, not just prompt wording.
  • Prompt changes (type-aware cue, integration advisories, correctness-fix prompt) are paired with real mechanism changes that produce the inputs to those prompts.
  • No new third-party imports; uses stdlib ast/re/subprocess; no new env vars; no sampling params; no forbidden hosts.
  • solve() signature and return dict shape unchanged; DANGEROUS_PATTERNS not weakened; api_base/api_key routing preserved.
  • Existing # MINER-EDITABLE / # VALIDATOR CONTRACT markers and structure remain readable; identifiers are descriptive.

Risks

  • goodhart (minor): the new self-check 'type-aware cue' and prompt text could be read as judge-flattering language, but it's paired with real gate logic, not prose-only.

Required Changes

  • No required changes returned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant