Repository navigation
studyrun: power gate, planted truth, autochecks, forecasts and spec/2 - #436
Merged
Merged
Conversation
Adds studyrun.spec/2, verdict paths for keys that hold a dot, a power gate, planted-truth fitting, autochecks for limits, constants and post hoc files (studyrun audit, exit 4), forecast scoring and ext_sha256 in new locks. The four new modules and verdict.py land byte-identical to the upgrade's carry-over list. __init__, __main__, ledger, prereg, spec and stage were three-way merged against the pre-upgrade backup, because the Flywheel copy had diverged: canonical-record/1 hashing, the lock's canonical_form, and ledger scope keys stay. ext_sha256 is hashed under the lock's own canonical form, so a lock written by the prototype code still verifies. tests/test_studyrun_upgrade.py carries the upgrade's 23 tests with Flywheel imports. A compatibility sweep over 570 studyrun.spec/1 specs gives identical check problems, plan hashes and re-judged verdicts on Flywheel main and on this branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
studyrun now refuses five more ways a study can look sound and not be: an unreadable verdict path, a gate too weak to bind, a planted truth with the wrong sign, a limit or constant changed after the fact, and forecasts written after the data. Old specs and locks run and verify as before.
What it adds
a\.bor pass a list path. An unresolved path names the dotted key and stays UNTESTABLE.power.py): the lock runs the locked verdict rules on simulated runs and records each verdict's pass rate with a Wilson interval and a binding flag. A path that resolves in no simulated run refuses the lock.planted.py): fits sign and relative size of lagged terms on a calibration panel, or takes a cited fit, and refuses a truth world that plants a sign against it or drops a measured term.autocheck.py,studyrun audit, exit 4): limit lines reread against their cited spans, constants derived by non-study code, post hoc files sealed by hash with their checkpoint.forecast.py): Brier and log score per verdict, interval coverage and miss factor per quantity, intoresult.jsonand a ledger line.studyrun.spec/2: the same shape as /1, with forecasts for every verdict, power, planted, limits and constants required at lock.ext_sha256in new locks freezes power, planted and forecasts besideplan_sha256.docs/STUDYRUN.mddocuments each one.Where it came from, and how it was merged
The upgrade was made on 2026-11-02 in the private prototype the Flywheel copy came from, with a carry-over list of files and sha256 prefixes. Every source file matched that list before the move.
autocheck.py,forecast.py,planted.py,power.pyandverdict.pyland byte-identical to the list.__init__,__main__,ledger,prereg,specandstagehad diverged in Canonical-record spec, shared stats vectors and studyrun #433 (canonical-record/1 hashing, the lock'scanonical_form, ledger scope keys), so the list says merge by hand. I three-way merged each against the upgrade's pre-change backup: 7 conflicts: four import blocks, the CLI usage text, the lock line andplan_sha256. Flywheel's behaviour is kept in every conflict.ext_sha256(spec, form)hashes under the lock's owncanonical_form, asplan_sha256already does, so a lock written by the prototype code verifies here.docs/provenance/studyrun-upgrade-2026-11-02.jsonlists every file's source and both sha256 values.Tests
tests/test_studyrun_upgrade.py: the upgrade's 23 tests, imports changed toharness.studyrunandtests.test_studyrun. They include a sign-flipped plant that must be refused, an underpowered verdict that must be non-binding, an unescaped dotted key that must stay UNTESTABLE, a removed limit line, a changed constant, an edited post hoc file, an unregistered post hoc file, exact Brier and log scores, spec/2 refusal and the CLI.compat_sweep.py, through a shim that importsharness.studyrun, over all 570studyrun.spec/1specs in the private workspace, once on Flywheel main and once on this branch. Check problems, plan hashes and every re-judged verdict are identical for all 570; 551 have a stored scored result. The same 3 specs error under both, as the upgrade notes record (two compare a string with a number, one uses==).docs/STUDYRUN.md. CI is the check of record.Follow-up
power.pybrings one more Wilson interval. It should test againsttests/data/stats_vectors.jsonlike the Holm and bounds code; not done here.Not merged; left for review.
🤖 Generated with Claude Code