You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 03a6a62
Browse filesBrowse the repository at this point in the historyBrowse files
skill-eval: split the four-file family by function owner and drop the residual (S01)
Move skill-eval diagnostic, engineering, evaluation, and improvement
functions into Benchmark rules, Harness execution/sandbox, and Apps
composition. Delete the harness.skill_evaluation to knowledge_operations
residual and its gate registration. Public entry signatures and trial-record
bytes stay the same under the same fixtures.
usage_totals remains the H maintenance implementation. SelectionPolicy
thresholds move to Benchmark as a whole; choose stays in H. #338 leftover
infra.artifacts call sites outside this family are not taken into S01
(pre-R01 H blank pack). overall isolation_complete stays false.
Clean-history candidate: this commit transplants the accepted S01 tree
onto current main 57f8e8a without replaying the local H02 / #397 working
stack. The resulting tree is identical to 8b4a0cf (ordinary merge of
9218391 and 57f8e8a). Rollback remains at that original SHA. This
commit is a local candidate only and is not a rewrite of the published
branch.
Refs #410
Agent-assisted-by: Claude Code / claude-fable-5-2
Agent-assisted-by: Grok Bot
Signed-off-by: GUOGUO <xwdong1998@163.com>
Copy file name to clipboardExpand all lines: docs/contracts/current-design.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -153,7 +153,7 @@ Decision: see [`schema-and-semantics.md`](schema-and-semantics.md). Draft 2020-1
153
153
154
154
## D018 — first-wave gate policy detail
155
155
156
-
Decision: [`gate-policy-runstore-and-first-prs.md`](gate-policy-runstore-and-first-prs.md) §B: exact seven leaf→module allowances, shape checks, transitive-closure checks, the registered residual `harness/skill_evaluation.py → knowledge_operations → knowledge.*` edge, the staged isolation test, and the rule that a non-empty allowlist keeps `overall isolation_complete=false`.
156
+
Decision: [`gate-policy-runstore-and-first-prs.md`](gate-policy-runstore-and-first-prs.md) §B: exact seven leaf→module allowances, shape checks, transitive-closure checks, the registered residual `harness/skill_evaluation.py → knowledge_operations → knowledge.*` edge, the staged isolation test, and the rule that a non-empty allowlist keeps `overall isolation_complete=false`. The skill-evaluation split (#410) later removed the registered residual edge and its exemption.
Copy file name to clipboardExpand all lines: docs/contracts/function-owners.md
+7Lines changed: 7 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -21,3 +21,10 @@ Status: **Proposed**. Author decisions under the user's delegation, based on the
21
21
|`catalog.repository_pack`/`REPO_TYPES`| D020 neutral raw-data technical leaf; target interpretation stays H | K does not expose the pack reader publicly; `infra` changes only its same-leaf import; seven K bodies/root entries/tests move together |
22
22
23
23
Any later split of a mixed function is accepted by real pytest/host/wheel/record comparison in the corresponding PR; none of this delays today's contract or gate-policy start. Existing external public composed entries keep their original semantics and paths per D018/D022, and an owner change cannot be used to delete an old promise.
24
+
25
+
## Implemented rows
26
+
27
+
The skill-evaluation split (#410) implements the `skill_evaluation`, `skill_eval_diagnostic`, `skill_eval_engineering` and `improvement` rows. H keeps `harness/skill_evaluation.py`, `harness/artifact_sandbox.py` and `harness/improvement.py`; B holds `benchmark/skill_evaluation.py`, `benchmark/skill_eval_diagnostic.py`, `benchmark/skill_eval_engineering.py` and `benchmark/improvement.py`; Apps composes them in `apps/skill_evaluation.py`, `apps/skill_eval_engineering.py` and `apps/improvement.py`. Two points follow the tree as it stands after #338, not the row text:
28
+
29
+
-`usage_totals` stays in `harness/usage.py`, because `harness/optimization.py` consumes it for budget accounting. B `assessment` receives the baseline and candidate totals from Apps and does not import H.
30
+
-`SelectionPolicy` holds only the two thresholds, so it lives whole in B. H `choose` receives the assessments, the validity block and the composition flag; it holds no threshold and no policy class.
Copy file name to clipboardExpand all lines: docs/contracts/gate-policy-runstore-and-first-prs.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -39,7 +39,7 @@ The symbol inventory is AST-derived and does not claim to have found every third
39
39
1. The default cross-core prohibition stays. Only the seven leaf→exact-K-module edges above may serve as migration-compatibility exceptions; a module prefix does not authorize adjacent new modules.
40
40
2. Shape checks allow only a module docstring, a `future` import, the listed `ImportFrom` lines (alias only as `name as name`), and an `__all__` constant string list. No business definitions, other imports, assignment side effects, or dynamic module hooks.
41
41
3. The static graph resolves absolute/relative imports, `from package import name`, and known export edges, and builds the transitive closure. Any target K/B/H core API entry that reaches another core, Apps, or a legacy facade fails. The D020 neutral technical leaf is not a legacy facade and must be independently verified to contain only raw data reading/data constants and no core imports; "only indirectly, so it passes" is not allowed. Target H entry points are the new `harness.api`/`execution_service`/`run_store`; the current mixed `harness` does not automatically count as an isolated core.
42
-
4. The existing `harness/skill_evaluation.py → knowledge_operations → knowledge.*` edge is a registered **exact residual legacy dependency**: retained, and it keeps `overall isolation_complete=false`. It is not erased from the report and no new/expanded H bypass is permitted. It is not an exception available to a target H new entry.
42
+
4. The existing `harness/skill_evaluation.py → knowledge_operations → knowledge.*` edge is a registered **exact residual legacy dependency**: retained, and it keeps `overall isolation_complete=false`. It is not erased from the report and no new/expanded H bypass is permitted. It is not an exception available to a target H new entry. The skill-evaluation split (#410) later removed this edge and its exemption; [`../module-layout.md`](../module-layout.md#first-wave-compatibility-exceptions-proposed) records the current rule.
43
43
5.`test_clean_scan_is_not_isolation` becomes a staged-model test. It does not permanently assert that all three package directories are absent. It verifies that `overall` is still false when any target package, any compat exception, or an old H→model/infra transitive residual is missing. Green is not manufactured by deleting assertions.
44
44
6. A non-empty allowlist forces `overall isolation_complete=false`; "new core business closure isolated" is reported as a separate verifiable result and cannot impersonate isolation of the whole historical namespace.
45
45
7. New Apps entries call core public APIs only and never pass through old root shells; cores never import Apps. If a dynamic import is genuinely needed later, it must declare exact static edges and get extra review; in the first wave all compatibility shells prohibit it.
Copy file name to clipboardExpand all lines: docs/contracts/isolation-errors-and-cli.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -42,7 +42,7 @@ Only the later resource slice K02 moves Knowledge assets to `knowledge/data`, an
42
42
43
43
### 4. Gate and sequencing
44
44
45
-
P0b must accept the policy of **exactly seven leaves + catalog's two-target symbol routes + a business-free technical leaf + infra importing the neutral leaf directly** before K01 is allowed. Apart from root catalog's second, neutral-leaf export, the six leaves export only their same-named K modules; catalog's K export is also limited to the named symbols. No new K entry may reach the legacy facade / H / B / Apps. The new infra path may not reach K / H / B / Apps. The old `harness.skill_evaluation` → root KO → new K path is registered residue; whole-isolation is false, and it is not given to new H to use.
45
+
P0b must accept the policy of **exactly seven leaves + catalog's two-target symbol routes + a business-free technical leaf + infra importing the neutral leaf directly** before K01 is allowed. Apart from root catalog's second, neutral-leaf export, the six leaves export only their same-named K modules; catalog's K export is also limited to the named symbols. No new K entry may reach the legacy facade / H / B / Apps. The new infra path may not reach K / H / B / Apps. The old `harness.skill_evaluation` → root KO → new K path is registered residue; whole-isolation is false, and it is not given to new H to use. The skill-evaluation split (#410) later removed that path.
46
46
47
47
There must be a synthetic graph that does not depend on whether the real target directories already exist: allow the exact leaves; reject adjacent K targets; reject K → old shell → K; reject new H → shell → K; check that Apps → shell does not meet the new Apps rules; reject a hidden core import inside `_repo_pack_data`; infra → root catalog → K must be caught as a negative case, and infra → the neutral leaf must pass. A non-empty allowlist or residue makes overall false, even if a local new-core graph is isolated. The scan scope is explicitly the `src` transfer graph. `tools` / `tests` are checked separately as registered consumers; it must not be said that the existing gate already scans them. Undeclared dynamic edges fail closed.
Copy file name to clipboardExpand all lines: docs/module-layout.md
+15-12Lines changed: 15 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -65,6 +65,9 @@ Several flat modules remain shared seams until their slice starts.
65
65
|`harness/optimization_session.py`, `harness/project.py`, `harness/task_protocol.py`| Shared preparation, execution and retained-result assessment | CLI supplies task resources and knowledge access; ordinary project lifecycle does not import Benchmark |
66
66
|`harness/optimization.py`, `harness/candidates.py`, `harness/usage.py`| Shared project and packaged-task lifecycle | Injected execution/assessment; declared file/directory candidates, selection direction and exact/up-to rounds |
67
67
|`harness/agent_sessions.py`| Native host handoff records | Freezes requests and retains native IDs and exported observations; the initiating Agent calls host tools |
68
+
|`harness/skill_evaluation.py`, `harness/artifact_sandbox.py`, `harness/improvement.py`, `harness/atomic_json.py`| Skill-evaluation host adapter, trial execution and retention, sandbox replay, search progression | Verdict rules and task definitions arrive as injected values; no Model, Infra, Knowledge, Benchmark or Apps import |
69
+
|`benchmark/skill_evaluation.py`, `benchmark/skill_eval_diagnostic.py`, `benchmark/skill_eval_engineering.py`, `benchmark/improvement.py`| Skill-evaluation criteria, task definitions and acceptance rules | Pure judgments over retained facts; no Harness import and no process start |
70
+
|`apps/skill_evaluation.py`, `apps/skill_eval_engineering.py`, `apps/improvement.py`| Composed skill-evaluation entries | Join Knowledge reads, Harness execution and Benchmark rules behind the signatures that `harness/` held before the split |
68
71
|`knowledge_operations.py`, `knowledge_store.py`, `catalog.py`, `research.py`, `skill_catalog.py`, `_skill_selection.py`, `_source_snapshot.py`| Static compatibility shells since K01; bodies in `knowledge/`| Interfaces unchanged |
69
72
|`harness/`| Mixed legacy tree, not target isolation | Not redefined |
70
73
|`evaluation.py`, `model/validators/`| Benchmark-leaning checks | Not moved |
@@ -132,8 +135,7 @@ A clean result means no new cross-core import of those names. It does not mean
132
135
the modules are isolated. Isolation stays false while any one of these holds:
133
136
`knowledge/`, `benchmark/`, or `apps/` is absent or has no scannable
134
137
module; the compatibility allowlist
135
-
is non-empty; the registered residual `harness.skill_evaluation` → root
136
-
`knowledge_operations` exists; or `harness/**` still imports Model or Infra,
138
+
is non-empty; or `harness/**` still imports Model or Infra,
137
139
directly or through another module. The three packages may appear in stages.
138
140
A clean graph for the new cores is reported separately as "new core business
139
141
closure isolated" and never sets overall `isolation_complete` to true.
@@ -191,8 +193,8 @@ The leaf's own import rule and `REPO_TYPES` object identity govern it.
191
193
-**Closures.** A Knowledge module reaches nothing outside Knowledge except
192
194
the data leaf and `inferrsi._json_v1`, the canonical encoder in the P0a
193
195
design. Benchmark modules and every `inferrsi.harness.*` module reach no
194
-
other core, no Apps module, and no old root leaf, except the registered
195
-
residual below. Apps reaches no old root leaf. `inferrsi.infra`
196
+
other core, no Apps module, and no old root leaf. Apps reaches no old root
197
+
leaf. `inferrsi.infra`
196
198
reaches no Knowledge, Harness, Benchmark, or Apps module, so `infra` → root
197
199
`catalog` → Knowledge fails while `infra` → the data leaf passes.
198
200
-**New H entries.** The registered entry modules are
@@ -203,14 +205,15 @@ The leaf's own import rule and `REPO_TYPES` object identity govern it.
203
205
paths from labels. Any `inferrsi.harness.*` module that exists is still
204
206
checked against the closure rule above. The current mixed `harness` tree is
205
207
not automatically an isolated core.
206
-
-**Residual.** The registered `harness.skill_evaluation` → root
207
-
`knowledge_operations` edge stays listed and keeps overall isolation false.
208
-
A second direct harness importer of that target fails, and a new H entry
209
-
may not use the residual even through `skill_evaluation`. A mixed non-entry
210
-
harness module may still reach that target only through the residual
211
-
importer; that existing leak is not a new exception. The exemption is
212
-
reachability after removing only that exact residual edge: another path
213
-
to the same target is still a violation, regardless of traversal order.
208
+
-**No residual.** The skill-evaluation split (#410) moved `export_bundle` to
209
+
Apps and removed the registered `harness.skill_evaluation` → root
210
+
`knowledge_operations` edge together with its exemption. Every harness
211
+
module is walked on the full graph. A harness module that reaches an old
212
+
root leaf fails, and so does each harness module that reaches the leaf
213
+
through it. The report lists every remaining `harness` → Model or Infra
214
+
path in `harness_model_infra`; the split modules `harness.skill_evaluation`,
215
+
`harness.improvement`, `harness.artifact_sandbox` and `harness.atomic_json`
216
+
have none.
214
217
215
218
The manifest records the base commit, each old leaf's file hash at that
216
219
commit, export names, audited definitions, consumer evidence, catalog's two
0 commit comments