Skip to content

benchmarks: add nonclosed projection image construction - #324

Merged
morluto merged 11 commits into
mainfrom
agent/harbor-nonclosed-projection-image
Aug 2, 2026
Merged

benchmarks: add nonclosed projection image construction#324
morluto merged 11 commits into
mainfrom
agent/harbor-nonclosed-projection-image

Conversation

@yuelgrace1810-ops

Copy link
Copy Markdown
Collaborator

Summary

Adds a Harbor regression benchmark for constructing and certifying a closed subspace whose orthogonal projection has nonclosed image.

  • Source: CounterMath, pinned revision d4e9f8ca877552f4491a9c2d52e0d230c0fca620, test row 201
  • License: CC-BY-SA-4.0
  • Quality score: 91/100
  • Family: Regression
  • Difficulty: Hard (provisional)
  • Primary objective: infinite-dimensional counterexample construction

Benchmark design

The task uses H = ℓ² ⊕ ℓ², the diagonal operator T(x)_n = x_n/n, its graph M, and projection onto the second summand. A valid submission must provide exact truncation witnesses converging to y_n = 1/n, a quantitative tail bound, and the forced-preimage argument showing that y is not in the image.

The shortcut audit rejects theorem-name citation, finite-prefix-only evidence, and finite-dimensional substitutes. The verifier independently checks canonical rational data, twelve exact prefixes, the convergence bound, divergence of the forced preimage norms, evidence bindings, and the COMPUTED assurance ceiling.

Validation

  • make harbor-check: 456 passed
  • Ruff lint/format: passed
  • Task-specific verifier and adversarial tests: passed as part of Harbor checks
  • Ownership/count, manifest, schema, and digest checks: passed
  • git diff --check: passed
  • Local Docker execution was unavailable; GitHub Actions provides the containerized Oracle lane

Assurance boundary

The verifier establishes the frozen sequence construction and its exact finite certificates. It does not claim a proof-assistant-verified theorem about arbitrary Hilbert spaces, so the result remains COMPUTED, not VERIFIED.

@morluto
morluto marked this pull request as ready for review August 2, 2026 06:53

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5631b5ddc9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread benchmarks/datasets/agent-workflow-v1/nonclosed-projection-image/instruction.md Outdated

morluto commented Aug 2, 2026

Copy link
Copy Markdown
Owner

@codex

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: 5631b5ddc9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

morluto and others added 3 commits August 2, 2026 15:22
Bump the agent-workflow-v1 dataset version from 1.1.0 to 1.2.0 to
satisfy the suite version-bump planner gate introduced by PR #328.
The dataset manifest and task digests are unchanged because the
version field lives in suite.toml, not in the generated dataset.toml.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Update the README version sentence from 1.1.0 to 1.2.0 to match the
suite.toml version bump.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ection task

Address #324 review threads on the nonclosed-projection-image workflow task.

- Instruction: drop the prescriptive "use an ell2 diagonal-operator graph"
  directive so the prompt stays strategy-free and states only the
  counterexample requirements and evidence links.
- Verifier: replace literal Oracle-string comparisons with generic validation
  of any bounded positive diagonal weights whose square-summable limit y and
  non-square-summable forced preimage x satisfy y_n = w_n x_n, with recomputed
  partial norms and a structured tail bound proving convergence. The review's
  alternative (weights 1/n^2, limit 1/n, preimage n) now passes instead of
  being a false negative.
- Evidence: require a result-bound RESULT_JSON marker plus a PROOF_JSON block
  with a substantive per-obligation argument (boundedness, closedness, range
  identification, convergence, absent preimage), so a token-only evidence file
  no longer earns full evidence validity.
- Add regressions covering the reference and alternate witnesses, the
  token-only and missing-obligation evidence attacks, and corrupted witness
  rejections; refresh the dataset manifest, verifier checksum label, and
  research manifest digests.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 61acb1b6a9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fe5847343c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread research/evaluations/capability-workflow-v1/gap-ledger.json Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2120abbe8e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b5853e153d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 008a3561dc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@morluto
morluto merged commit 5bfce33 into main Aug 2, 2026
27 checks passed
@morluto
morluto deleted the agent/harbor-nonclosed-projection-image branch August 2, 2026 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants