|
| 1 | +# Cumulative PR budget and Git ledger |
| 2 | + |
| 3 | +Refs [#215](https://github.com/InferMatrixLab/InferRSI/issues/215). |
| 4 | +`tools/skill_eval_budget.py` implements pure accounting transitions. |
| 5 | +`tools/skill_eval_budget_git.py` implements the GitHub Git-data adapter using the |
| 6 | +existing authenticated `GitHub.request` client and injected `GITHUB_TOKEN`. |
| 7 | +The adapter never loads credentials or executes PR code. Tests use mocks only. |
| 8 | + |
| 9 | +## Accounting |
| 10 | + |
| 11 | +- Key: immutable repository ID plus PR number; never branch name, repository |
| 12 | + name, head revision, workflow run or label epoch. Renames, pushes, retries and |
| 13 | + label removal/reapplication cannot create a fresh PR allowance. |
| 14 | +- The requested ceiling is USD 200 per PR, represented as 200,000,000 integer USD |
| 15 | + microdollars. Trusted pricing calculations must round worst-case costs upward. |
| 16 | + Floats, booleans, negative costs and zero-cost reservations are rejected. |
| 17 | +- A trusted frozen-plan digest covers all revisions, mapping, exam, provider, |
| 18 | + pricing, retries, request/token limits and other cost-relevant inputs. The core |
| 19 | + receives its independently computed worst-case total; it does not derive cost |
| 20 | + or prove that the plan producer is trustworthy. |
| 21 | +- Reserve the entire plan before execution. Charge = audited legacy obligation |
| 22 | + plus **every reservation**, including cancelled, failed, interrupted, completed |
| 23 | + and ambiguous executions. Actual cost is evidence only; there are no refunds. |
| 24 | +- An exact plan/run/cost binding is reserve-idempotent. Reusing a plan with a |
| 25 | + different run, or a run with a different plan/cost, fails closed. This first |
| 26 | + version deliberately does not support rerunning an identical plan. A distinct |
| 27 | + authorized attempt must have a distinct frozen identity and a fresh reservation. |
| 28 | + Run identity includes workflow run attempt, not just workflow run ID. |
| 29 | + |
| 30 | +```text |
| 31 | +verified genesis → reserved → claimed → finished |
| 32 | + └───→ cancelled |
| 33 | +``` |
| 34 | + |
| 35 | +Claim is non-idempotent: at most one process can obtain a successful claim CAS. |
| 36 | +No takeover, lease expiry, automatic retry or reset exists. A process crash after |
| 37 | +claim burns capacity even when no request happened. `claim_once` returns only on |
| 38 | +an explicit, durable CAS success for that invocation. Its return is **not** a |
| 39 | +replayable execution credential. The trusted caller must execute at most once, |
| 40 | +respect all frozen bounds (including provider retries), and not expose a result |
| 41 | +that another worker can reuse to execute again. An actual-cost overrun is rejected |
| 42 | +and requires external quarantine/reconciliation before further paid activity; |
| 43 | +preventing overruns depends on the future bounded executor. |
| 44 | + |
| 45 | +## Genesis is an audited migration, never a missing-file default |
| 46 | + |
| 47 | +`Genesis(evidence, legacy_charge)` requires an explicit audit reference and an |
| 48 | +explicit integer charge. Zero is acceptable only when the authorized audit proves |
| 49 | +no earlier paid obligations. Include settled legacy spend and conservative upper |
| 50 | +bounds for all outstanding/ambiguous legacy runs; quiesce or reconcile older |
| 51 | +writers before migration. Unknown obligations block genesis. If the obligations |
| 52 | +exceed the ceiling, initialization fails and the PR is ineligible. |
| 53 | + |
| 54 | +Constructing this value is not authorization or cryptographic proof. The separately authorized |
| 55 | +trusted initializer must verify the audit and atomically create only for a key |
| 56 | +that has **never** had a ledger. There is intentionally no genesis initializer in the core or Git adapter. Missing/corrupt existing state, HTTP 404, expired artifacts, empty |
| 57 | +workflow history and label reapplication are never permission to initialize. |
| 58 | + |
| 59 | +## Atomic storage contract |
| 60 | + |
| 61 | +`AtomicStore.read_verified(key)` returns a `Snapshot(history_token, ledger)` only |
| 62 | +from authenticated, durable, current history. `decode` validates schema and local |
| 63 | +invariants, **not provenance, rollback resistance or history completeness**. |
| 64 | +The backend retains an independently configured genesis/checkpoint anchor that |
| 65 | +survives ledger ref deletion. A ledger's own revision cannot detect its reset. |
| 66 | +See the precise rollback trust boundary below. |
| 67 | +Missing, corrupt, untrusted or discontinuous history must raise and block work. |
| 68 | + |
| 69 | +`compare_and_swap(snapshot, successor)` must atomically append exactly one |
| 70 | +successor against the current immutable history token, using |
| 71 | +`validate_successor` as well as key/lineage checks. All writers use the same CAS |
| 72 | +boundary. A matching body already stored by another request is **not** this |
| 73 | +caller's successful commit. On conflict, reread verified state and recompute; |
| 74 | +never overwrite a newer charge. On unknown write outcome, do not execute. |
| 75 | + |
| 76 | +Reservation writes can reconcile by rereading the exact binding and charge. |
| 77 | +Claim writes cannot reconcile into a new execution grant: an observed `claimed` |
| 78 | +entry may already have executed. Duplicate/ambiguous claims stay blocked. If a |
| 79 | +verified read still shows `reserved`, a new CAS can compete safely only against |
| 80 | +that current token; a delayed earlier write must conflict with the same boundary. |
| 81 | +Terminal evidence retries are idempotent only when identical; no terminal update |
| 82 | +changes charged capacity. |
| 83 | + |
| 84 | +## Git-data adapter and trusted bootstrap |
| 85 | + |
| 86 | +Each PR uses exactly `refs/heads/skill-eval-budget/<repository_id>/<pr_number>`. |
| 87 | +A trusted `LedgerAnchor` supplies immutable repository/PR identity, expected |
| 88 | +`genesis_commit`, and independently retained `checkpoint_commit`. These values |
| 89 | +must come from reviewed root configuration, **never PR files, artifacts, labels, |
| 90 | +comments, caller-selected refs or ledger contents**. The API repository ID is |
| 91 | +checked on every verified read; repository renaming does not refresh capacity. |
| 92 | + |
| 93 | +Before activation an authorized operator must quiesce previous paid writers, |
| 94 | +audit settled **and outstanding/ambiguous** legacy obligations, and create a |
| 95 | +genesis commit with exactly `ledger.json` holding revision-zero state and the |
| 96 | +explicit `Genesis(evidence, legacy_charge)`. The genesis may be parentless or have |
| 97 | +exactly one explicitly audited existing-main parent when the authorized Git API |
| 98 | +requires a parent. Its tree must still contain **only** `ledger.json`; do not |
| 99 | +inherit the main tree. Pin the complete genesis commit SHA and checkpoint in |
| 100 | +trusted root configuration. That immutable SHA binds the genesis parent as well |
| 101 | +as its state. Parent ancestry before genesis is provenance, not ledger history: |
| 102 | +verification stops at the pinned audited genesis and never interprets main commits |
| 103 | +as budget transitions. Merge genesis commits are rejected. All commits after |
| 104 | +genesis still require exactly one parent and one valid accounting transition. An empty ledger is valid only after an explicit no-prior-spend |
| 105 | +audit; unknown obligations block bootstrap. Bootstrap/ref creation is deliberately |
| 106 | +not automated by this patch and no missing-ledger fallback exists. If the PR had |
| 107 | +an earlier ledger, recover/reconcile that history rather than creating new zero |
| 108 | +capacity. Unknown/reset history requires operator investigation, not retry-reset. |
| 109 | + |
| 110 | +Reads fetch the exact ref, every single-parent commit and exact ledger tree/blob |
| 111 | +back to pinned genesis. They check all accounting transitions and require both |
| 112 | +the configured checkpoint and the last tip observed by that store instance to |
| 113 | +be ancestors. Missing refs/objects, merges, extra tree entries, corruption, |
| 114 | +discontinuous transitions, divergent roots, rollback behind the checkpoint, |
| 115 | +identity changes, racing reads and excessive history all block execution. |
| 116 | +History verification is bounded to 10,000 commits and 4 MB per ledger; reaching |
| 117 | +bounds fails closed, never truncates accounting or resets capacity. |
| 118 | + |
| 119 | +Writes create an immutable tree and a single-parent child of the verified tip, |
| 120 | +then PATCH the existing ref with `force: false`. The |
| 121 | +[GitHub reference API](https://docs.github.com/en/rest/git/refs#update-a-reference) |
| 122 | +requires a fast-forward when force is false. Under append-only writers, two |
| 123 | +children of the same tip cannot both become the next tip. A random nonce makes |
| 124 | +even identical state transitions different commits, preventing two identical |
| 125 | +claims from receiving successful idempotent updates for a shared commit SHA. |
| 126 | +Stale snapshots return conflict; failed/uncertain writes return unknown, including |
| 127 | +permission/rules rejection. No automatic PATCH retry or claim reconciliation |
| 128 | +confers execution permission. Unreachable trees/commits after a crash do not |
| 129 | +change accounting; a committed reservation/claim remains permanently charged. |
| 130 | +No branch creation, force update, deletion or repository-settings API is used. |
| 131 | + |
| 132 | +### Trust boundary: not protection against malicious administrators |
| 133 | + |
| 134 | +Git refs are mutable by sufficiently privileged principals. The security model |
| 135 | +requires trusted root configuration/code and repository administrators, plus no |
| 136 | +out-of-band force writes/deletion/recreation of the ledger ref. Use independently |
| 137 | +administered append-only restrictions where available; this implementation does |
| 138 | +not change settings or bypass rules. A static genesis pin detects a replaced |
| 139 | +root but **cannot detect rollback to a valid later ancestor across new processes**. |
| 140 | +A trusted checkpoint detects rollback before that checkpoint, and process memory |
| 141 | +detects rollback behind a tip already seen in that process. Neither is an external |
| 142 | +monotonic witness. The tests explicitly demonstrate this limitation. |
| 143 | + |
| 144 | +Absolute rollback resistance against malicious repository administrators would |
| 145 | +require separately authorized, independently durable monotonic storage or an |
| 146 | +independently administered enforcement boundary. Do not claim this Git branch |
| 147 | +provides it. Retain and advance reviewed checkpoints as audit evidence; never |
| 148 | +lower or replace a checkpoint to make a failed verification pass. This document |
| 149 | +does not authorize any settings changes, migration, or remote writes. |
| 150 | + |
| 151 | +## Verification and integration boundary |
| 152 | + |
| 153 | +`pytest tests/test_skill_eval_budget.py tests/test_skill_eval_budget_git.py` |
| 154 | +covers cumulative limits, permanent reservations, legacy charges, duplicate |
| 155 | +plan/run bindings, malformed state, reservation/claim races, unique claim commit |
| 156 | +identities, crashes/lost responses, rules rejection, history corruption/deletion, |
| 157 | +checkpoint rollback detection and its explicit administrator limitation. The |
| 158 | +mock Git API enforces non-force fast-forward semantics. CPU tests do not prove |
| 159 | +provider billing or live GitHub rules/permission compatibility. |
| 160 | + |
| 161 | +The adapter does not create a plan, grant label authority, run a provider or |
| 162 | +reconcile actual billing. The trusted controller must reserve and claim before |
| 163 | +executing, revalidate admission, enforce all worst-case plan bounds including |
| 164 | +retries, and keep credentials away from untrusted PR code. A returned claim is |
| 165 | +process-local permission for one execution, not a reusable workflow output. |
| 166 | +Live activation still requires the audited genesis/root configuration and |
| 167 | +successful permitted writes; absent prerequisites fail closed. |
0 commit comments