benchmarks: add cyclotomic reciprocity certificate - #337
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d7ca8e5750
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b263acdae5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 72087adf41
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The cyclotomic-reciprocity-certificate tests/Dockerfile carried a stale jacobian.checksum label (left over from the verifier hardening merge) plus a stray "placeholder" label. The Harbor dataset contract check uses the first 64-hex checksum label and compares it to sha256(verifier.py), so the stale value failed Benchmark Prospective Digest Checks and cascaded into Benchmark Validation. Replace both labels with a single correct checksum matching the current verifier.py (11e66b6a...). Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a5de0dd1b5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5225b925f7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| submission, verification_record_bound=False | ||
| ) | ||
| correct = bool( | ||
| contract and math_correct and limitation_correct and not false_certification |
There was a problem hiding this comment.
When the result and limitations are otherwise valid, changing claimed_assurance from COMPUTED to CHECKED leaves false_certification false and this gate true, so the verifier awards 0.9 despite the frozen input and task metadata capping assurance at COMPUTED. This gives a positive benchmark outcome to an unsupported assurance claim; include assurance_correct in the zero-reward gate or otherwise reject above-ceiling levels.
AGENTS.md reference: AGENTS.md:L70-L74
Useful? React with 👍 / 👎.
| return False | ||
| target = resolve_evidence(evidence[0], expected_path="evidence/answer.txt") | ||
| try: | ||
| text = target.read_text().casefold() if target else "" |
There was a problem hiding this comment.
With this task's 4 GiB workspace allowance and 1 GiB verifier memory limit, an agent can create and digest-bind an evidence/answer.txt larger than verifier memory. This unbounded read_text() then allocates the whole file after the binding helper has already hashed it, causing the verifier to time out or be OOM-killed instead of rejecting the malformed submission with zero reward; check a small maximum file size before hashing or reading it.
Useful? React with 👍 / 👎.
2 participants
Summary
Adds
cyclotomic-reciprocity-certificate, an independent Regression Harbor benchmark derived fromlm-provers/FineProofs-SFTtrain row 358 at immutable revision73661e62811cf2940a0d3f82788a4f4332204c2f(Apache-2.0).The task requires complete cyclotomic factor recovery and a reciprocal-polynomial certificate for a frozen degree-16 instance.
Quality gate
Shortcut audit
The expanded coefficients visibly suggest symmetry, but that observation cannot pass. The agent must supply a complete ordered cyclotomic factorization. The verifier independently constructs every
Phi_nby exact division ofx^n-1, multiplies all factors with multiplicities, checks all 17 coefficients, evaluatesP(1), and verifies the reciprocal vector. Missing or corrupted factors fail.The public source theorem is a contamination limitation, so this is regression evidence rather than held-out capability evidence.
Deduplication
Checked against main and all active drafts by source, workflow, verifier, failure mode, and assurance boundary. This adds factor recovery plus root-orbit reciprocity, unlike recurrence symmetry, ideal membership, or rational moment reconstruction.
Validation
make harbor-check— 458 passedmake complexity-check— baseline unchangedgit diff --check— passedAssurance boundary
The clean-room verifier certifies the frozen factorization and reciprocal coefficient identity exactly. The unrestricted theorem for every real polynomial whose roots are roots of unity is not proof-assistant verified. Maximum assurance remains
COMPUTED.