Skip to content

Commit bc49433

Browse files
committed
knowledge: preserve notices and clarify speculative counters
Package extensionless third-party notices and assert their installed attribution. Distinguish model-stop/output-limit exclusions from later stop-string cuts in accepted-proposal counters; reconcile the assertion inventory. Agent-assisted-by: Codex / gpt-6.1-sol [effort=xhigh] (approved_arex_author; review fixes) Signed-off-by: WeiQing Chen <530634352@qq.com>
1 parent b4b26cb commit bc49433

7 files changed

Lines changed: 23 additions & 11 deletions

File tree

‎pyproject.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ where = ["src"]
2323
namespaces = false
2424

2525
[tool.setuptools.package-data]
26-
inferrsi = ["data/**/*.json", "data/**/*.md", "data/**/LICENSE", "data/**/scripts/*.py"]
26+
inferrsi = ["data/**/*.json", "data/**/*.md", "data/**/LICENSE", "data/**/THIRD_PARTY_NOTICES", "data/**/scripts/*.py"]
2727

2828
[project.optional-dependencies]
2929
skills = ["PyYAML>=6.0"]

‎src/inferrsi/data/skill_library/repositories/splash/sub-skills/qualification/references/verification-contract.md‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -63,11 +63,14 @@ cancellation or E2E correctness. Evidence:
6363
- A native cycle normally produces eight draft rows: one anchor and seven
6464
proposals. Remaining output budget and verify-row choice can reduce the
6565
verified/retained portion. Unverified proposals are not acceptance failures.
66-
- `draft_n` counts proposals actually verified. `draft_n_accepted` counts the
67-
accepted proposals that the response's policy retains, with exclusions for
68-
stop/output-limit and thinking-end replacement described by the API contract.
69-
Do not divide by every proposal produced internally and call that the same
70-
metric. Retain the definitions with the raw counter values.
66+
- `draft_n` counts proposals actually verified. `draft_n_accepted` counts those
67+
verification accepted, excluding accepted proposals past the output limit or
68+
from a model stop token onward. A later server-side `stop`-string cut does
69+
**not** subtract those proposals: they still count. Thinking-budget end tokens
70+
substituted for draft proposals count as neither proposed nor accepted.
71+
Do not divide by every internally produced proposal, or recalculate accepted
72+
counts from only the final visible string. Retain these definitions with the
73+
raw counters; visible retained output and verification acceptance differ.
7174
- `metrics.speculation.cycles` and `1 + accepted / cycles` describe speculative
7275
yield. They omit preparation, prefill, queueing and the cost of the draft,
7376
target verifier, state/cache work and corrections. More accepted tokens can

‎tests/check_installed.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,11 @@
4545
skill_index = search_knowledge(repo="inferrsi", limit=100)
4646
assert len(skill_index) == 61, "Missing installed skill abstracts"
4747
assert all(record["index_type"] == "packaged-skill" for record in skill_index)
48+
splash_notices = files("inferrsi").joinpath(
49+
"data/skill_library/repositories/splash/THIRD_PARTY_NOTICES"
50+
)
51+
assert splash_notices.is_file(), "Splash third-party notices are missing from the wheel"
52+
assert "The ggml authors" in splash_notices.read_text(encoding="utf-8")
4853
indexed = KnowledgeReader().read("inferrsi.ascend.skills.ascend_operator_profiling")
4954
assert "data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md" in indexed.content
5055
case_path = (

‎tests/fixtures/repo_to_skill/splash/reports/self-refine/evals.json‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@
4949
{
5050
"id": 4,
5151
"prompt": "Splash now reports more accepted draft tokens per cycle, so I want to announce a service speedup. Both runs used the same model name, but one used --verify-rows auto and the other all, and I only saved API predicted_per_second. Explain whether this proves anything, define the counters, and plan a fair comparison without starting a server. I can supply four rounds of lower-is-better samples for offline summarization.",
52-
"expected_output": "The response routes to qualification and distinguishes produced, verified, accepted and retained rows.; The response explains that draft_n counts verified proposals rather than all produced proposals, and that accepted tokens per cycle do not prove net speed or quality.; The response identifies auto versus all verify-row policy as a changed control, including timing/history dependence and possible greedy or seeded output variation.; The response explains that API intervals exclude initial admission and the decode rate excludes the first emission, so client E2E latency and complete task quality are still required.; The response requires same target/draft and prepared-byte identities, hardware/power/cache/concurrency/precision controls and frozen quality/performance gates before comparison.; The response uses or references bundled summarize_rounds.py for descriptive ABBA samples without inventing a PASS threshold, executing an engine or claiming reproduction.",
52+
"expected_output": "The response routes to qualification and distinguishes produced, verified, accepted and retained rows.; The response explains that draft_n counts verified proposals rather than all produced proposals, and that accepted tokens per cycle do not prove net speed or quality.; The response identifies auto versus all verify-row policy as a changed control, including timing/history dependence and possible greedy or seeded output variation.; The response explains that API intervals exclude initial admission and the decode rate excludes the first emission, so client E2E latency and complete task quality are still required.; The response requires same target/draft and prepared-byte identities, hardware/power/cache/concurrency/precision controls and frozen quality/performance gates before comparison.; The response uses or references bundled summarize_rounds.py for descriptive ABBA samples without inventing a PASS threshold, executing an engine or claiming reproduction.; The response distinguishes model-stop/output-limit exclusions from later stop-string truncation, whose accepted proposals still count; it does not recompute acceptance from visible output alone.",
5353
"files": [
5454
"sub-skills/qualification/SKILL.md",
5555
"sub-skills/qualification/references/verification-contract.md",
@@ -62,7 +62,8 @@
6262
"The response identifies auto versus all verify-row policy as a changed control, including timing/history dependence and possible greedy or seeded output variation.",
6363
"The response explains that API intervals exclude initial admission and the decode rate excludes the first emission, so client E2E latency and complete task quality are still required.",
6464
"The response requires same target/draft and prepared-byte identities, hardware/power/cache/concurrency/precision controls and frozen quality/performance gates before comparison.",
65-
"The response uses or references bundled summarize_rounds.py for descriptive ABBA samples without inventing a PASS threshold, executing an engine or claiming reproduction."
65+
"The response uses or references bundled summarize_rounds.py for descriptive ABBA samples without inventing a PASS threshold, executing an engine or claiming reproduction.",
66+
"The response distinguishes model-stop/output-limit exclusions from later stop-string truncation, whose accepted proposals still count; it does not recompute acceptance from visible output alone."
6667
]
6768
},
6869
{
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
11
# Content self-refine
22

33
The root author reviewed every complete lane output and helper, all case assertions and source-script decisions. Concrete state/format/memory/measurement contracts, root/sibling routing, internal links and native gaps are present. Added the safe root request helper link only after its file existed. Fixed verdict/noise thresholds were removed from the adapted statistics helper; budget syntax is explicitly not memory fit. Source body limits and protocol-versus-native implementation boundaries remain explicit. Fresh independent runtime-only usability grading is separately pending.
4+
5+
Independent review found and source-checked a counter distinction: accepted proposals cut by a later stop string still count, unlike model-stop/output-limit exclusions. The reference and assertion now state this exactly. A wheel audit also found extensionless THIRD_PARTY_NOTICES absent; packaging now explicitly includes that basename and the installed check asserts its retained attribution. Source notices are unchanged.

‎tests/fixtures/repo_to_skill/splash/test-cases/index.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,12 @@
55
| integration/memory-cache-concurrency-and-quality | integration | 4 |
66
| integration/prefix-safety-and-benchmark-attribution | integration | 4 |
77
| root/offline-request | splash | 3 |
8-
| sub-skills/qualification/acceptance-is-not-net-speed | qualification | 6 |
8+
| sub-skills/qualification/acceptance-is-not-net-speed | qualification | 7 |
99
| sub-skills/qualification/precision-and-ane-control | qualification | 6 |
1010
| sub-skills/qualification/same-machine-round-plan | qualification | 6 |
1111
| sub-skills/runtime-planning/gguf-format-and-draft-compatibility | runtime-planning | 7 |
1212
| sub-skills/runtime-planning/memory-cap-is-not-only-weights | runtime-planning | 7 |
1313
| sub-skills/serving-cache/persistent-cache-after-model-change | serving-cache | 7 |
1414
| sub-skills/serving-cache/same-prefix-different-cache-salt | serving-cache | 7 |
1515

16-
10 cases; 57 assertions. Every focused route has two difficult synthetic cases, with source-native anchors; qualification also has a model-identity control case. Two integrated source-backed scenarios cross cache, memory and qualification; these test guidance, not actual native execution.
16+
10 cases; 58 assertions. Every focused route has two difficult synthetic cases, with source-native anchors; qualification also has a model-identity control case. Two integrated source-backed scenarios cross cache, memory and qualification; these test guidance, not actual native execution.

‎tests/fixtures/repo_to_skill/splash/test-cases/sub-skills/qualification/acceptance-is-not-net-speed/assertions.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@
2020
"The response identifies auto versus all verify-row policy as a changed control, including timing/history dependence and possible greedy or seeded output variation.",
2121
"The response explains that API intervals exclude initial admission and the decode rate excludes the first emission, so client E2E latency and complete task quality are still required.",
2222
"The response requires same target/draft and prepared-byte identities, hardware/power/cache/concurrency/precision controls and frozen quality/performance gates before comparison.",
23-
"The response uses or references bundled summarize_rounds.py for descriptive ABBA samples without inventing a PASS threshold, executing an engine or claiming reproduction."
23+
"The response uses or references bundled summarize_rounds.py for descriptive ABBA samples without inventing a PASS threshold, executing an engine or claiming reproduction.",
24+
"The response distinguishes model-stop/output-limit exclusions from later stop-string truncation, whose accepted proposals still count; it does not recompute acceptance from visible output alone."
2425
]
2526
}

0 commit comments

Comments
 (0)