From 062d870897aae54e03448f866501a88420f87b4c Mon Sep 17 00:00:00 2001 From: carl Date: Mon, 17 Aug 2026 10:08:15 +0800 Subject: [PATCH 1/3] =?UTF-8?q?feat:=20[=20sources=20]=20=E6=AC=A1?= =?UTF-8?q?=E7=B4=9A=E4=BD=90=E8=AD=89=E4=BE=86=E6=BA=90=E8=B7=AF=E5=BE=91?= =?UTF-8?q?=EF=BC=9A=E4=BF=A1=E4=BB=B6=E6=91=98=E9=8C=84=E8=88=87=E8=A3=9C?= =?UTF-8?q?=E5=85=85=E8=BC=89=E9=AB=94?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 有些規範性資訊只存在於供應商的信件裡 —— 金鑰怎麼取得、測試環境在哪、 上線前要做什麼。先前只有兩條路:不放進 pipeline,資訊變成 missing;或讓 agent 讀進擷取結果,於是 provenance 宣稱一條主張有來源支撐,而那個來源 不存在於任何 manifest 條目。第二條讓報告說謊,而且沒有任何機制會發現。 新增第三條:人工摘錄成 Markdown,經 `import-supplementary-note` 匯入並 寫出綁定內容的 `.source.json` sidecar。manifest 據此把來源標成 `authority: supplementary`。它填得了 missing、支撐不了 explicit_support, 與正式文件衝突時正式文件勝。形狀取自 `rendered_url.py` —— 那條路徑要解 的問題完全相同。 等級放在來源層而非主張層:不可重新取得是載體固有的性質,對它承載的每 一條主張都成立;放主張層等於把同一判斷重複 N 次,並給了 N 次判錯的機會。 也刻意不併進 `derived_support`,那個關係指的是推理距離,混用之後沒有人 分得出一個 derived_support 是哪一種。 三處會咬人的交互作用,都有回歸測試: - `source_guard.source_violations` 在唯一文件時整個跳過,所以它問的是 `sole_normative_source()` —— 否則加一份摘錄會讓一份單一手冊的 run 在 建立目錄前被拒。但 `classify_item` 問的仍是 `sole_source()`(含次級), 因為它的 fallback 是「把無法解析的 locator 歸給唯一的文件」,而摘錄 就是第二份文件。兩者拆開之前,一條寫著「供應商信件」的引用會被記成 manual.md 支持 —— 正是這個功能要防的事,由它的修正重新引進。 - sidecar 讀取端 fail closed:缺席才是 normative,讀不動不是。宣告綁在 `source_file` 與 `imported_sha256` 上,否則一個兩行的複製 sidecar 就能 把正式手冊降級,而降級後它會整份退出新鮮度指紋。 - shadow/strict 撤掉次級佐證的支撐提案與證據引用。filename-only 引用本 來就降級,暴露面是 v1 精確證據 —— 它擁有自己宣告的 claim path,會把 摘錄以與手冊同等的身分寫進 Core candidate。撤掉而非改標 insufficient, 因為 Core 禁止 runtime 提出那個關係:那是它驗證後的結論。 其他:`record-fingerprint` 排除次級佐證,並對零可重取得來源 fail loud (空指紋不會失敗,它會永遠回報新鮮);逐條 warning 級 `SUPPLEMENTARY_SUPPORT` 點名只靠次級佐證成立的 plan item,含 `schemas[].field_evidence[]`;計分歸 source grounding。 接受的破口寫進 ADR 0010:摘錄是人寫的,`excerpted_by` 買到的是可追責, 不是可驗證。這是唯一帶著這個代價的來源類別。 規格裡「衝突時正式文件勝且不產生 SOURCE_CONFLICT」目前做不到 —— `source_conflicts[]` 是自由文字、沒有逐來源歸屬,要做得擴充擷取 schema。 記在 ADR 的 Not decided here。 Closes #102 🤖 Generated with Claude Code --- AGENTS.md | 11 +- README.md | 29 ++ ...carriers-are-accountable-not-verifiable.md | 112 +++++ loop_apidoc/agentcli/assemble.py | 5 +- loop_apidoc/agentcli/source_guard.py | 4 +- loop_apidoc/cli.py | 46 +- loop_apidoc/freshness/record.py | 14 +- loop_apidoc/manifest/builder.py | 4 +- loop_apidoc/manifest/models.py | 12 + loop_apidoc/manifest/scanner.py | 70 ++- loop_apidoc/plan/classify.py | 48 +- loop_apidoc/score/evaluate.py | 3 + loop_apidoc/shadow/bridge.py | 72 ++- loop_apidoc/supplementary_note.py | 140 ++++++ loop_apidoc/validate/authority.py | 94 ++++ loop_apidoc/validate/models.py | 1 + loop_apidoc/validate/validator.py | 2 + .../reference/assemble-and-correction.md | 4 +- .../test_assemble_supplementary_authority.py | 409 ++++++++++++++++++ tests/test_cli_freshness.py | 10 +- tests/test_cli_import_supplementary_note.py | 260 +++++++++++ tests/validate/test_models.py | 4 + 22 files changed, 1334 insertions(+), 20 deletions(-) create mode 100644 docs/adr/0010-supplementary-carriers-are-accountable-not-verifiable.md create mode 100644 loop_apidoc/supplementary_note.py create mode 100644 loop_apidoc/validate/authority.py create mode 100644 tests/agentcli/test_assemble_supplementary_authority.py create mode 100644 tests/test_cli_import_supplementary_note.py diff --git a/AGENTS.md b/AGENTS.md index c0d1f9fd..86fc53b3 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -106,29 +106,30 @@ The CLI commands include source acquisition, quality, assembly, analysis, and Fo | `loop_apidoc/domain/conformance.py` + `loop_apidoc/core/conformance.py` + `loop_apidoc/core/conformance_policy.py` | strict (`extra="forbid"`) immutable authority/scope models, pure `ContractConformance` assessment/proposal/composition, and the separate deterministic observation-target/routing policy. Status observations must bind the same operation's response-status path; response field/type observations must bind a matching field name/type path in a schema referenced by that operation's response. Only confirms/contradicts count as assessed; inconclusive/out-of-scope remain untested/open. Proposal/composition bind the complete Normative release digest. Stale amendments reflect verifiable release/contract/source/policy/approval-time drift; expiry and inapplicability remain separate. Free-text revalidation triggers are review declarations, not an external automatic trigger contract. | | `loop_apidoc/feedback/` | passive normalized-JSON adapter behind `feedback assess`/`propose`/`submit`/`review`/`approve`/`compose`/`current`/`provider-erratum`: fail-loud approved-base/bundle loading, deterministic ungoverned reports/previews, Foundry-backed case submission/write-once non-approval review/exact-scope approval and query, and a digest-verified non-mutating erratum handoff. `feedback review` accepts immutable cases with or without proposals, requires reviewer identity/version, timezone-aware decision time no earlier than observation completion or an existing proposal's creation, `rejected|needs_evidence`, and a corrective route other than `closed_no_change`/`amendment_proposal`. Governed persistence deterministically rejects sensitive field names and obvious email/phone/national-ID/SSN/passport/Luhn-valid payment-card values; low-entropy PII is omitted, never hashed. `feedback current` requires timezone-aware `--at`, rejects not-yet-effective/expired/stale/non-current-base Effective assets, verifies all three bounded Effective artifacts, and returns bounded-validity/discrepancy counters including unresolved contradictions. It performs no provider network I/O. `loader.py`, `report.py`, `erratum.py`, and the CLI adapter are explicit read/write I/O exits; governed writes are delegated to `foundry/`. | | `loop_apidoc/privacy.py` | single pure deterministic policy for sensitive field/value detection and display redaction shared by ungoverned feedback reports, governed Foundry persistence, and the pre-agent source-risk gate. The patterns are separated by what the match proves, not by where it is used: `SECRET_MATERIAL` (self-evidencing), `CREDENTIAL_REFERENCE` (a credential is being named — real value in a governed payload, almost always a placeholder in a source document), `CONTACT_PII`, `PII_VALUE`, and `PAYMENT_CARD_CANDIDATE` + `is_payment_card_number` (never used without the Luhn check, or long order/merchant IDs become cards). `find_sensitive_value` unions them, so the split alone leaves governed behaviour unchanged; only `source_risk/` grades them by severity. Two pattern bodies did change governed detection and are deliberate: the card candidate no longer spans a line break (it previously joined adjacent numeric fields into cards nobody wrote), and the contact-PII domain is now a bounded label structure rather than `[^\s@]+\.[^\s@]+` — the unbounded form is quadratic over a whole document and would let one source file stall the pre-agent gate. | -| `loop_apidoc/manifest/` | scan local sources + build `manifest.json` (`.doc` is detected as `word-legacy` and reported **unsupported**: OOXML validation/rendering does not apply to an OLE compound file and `preprocess` converts only `.pdf`/`.docx`, so calling it supported would be a claim the pipeline cannot honour — `.docx` is unaffected) (`scanner.py` excludes non-spec furniture via `DEFAULT_EXCLUDES` + `--exclude` globs → `status: ignored`, never source evidence; `builder.py` verifies coverage-matched rendered provenance and skips only that matched origin probe) | +| `loop_apidoc/manifest/` | scan local sources + build `manifest.json`. `LocalSource.authority` is `normative` (the default — every source that predates this field is a formal document, so the default is a fact rather than a compatibility concession) or `supplementary`, taken from the source's `.source.json` sidecar; an absent, unreadable, or authority-less sidecar means `normative`. The level is declared by the sidecar rather than a CLI flag (a one-shot declaration that does not travel with the corpus) or a directory convention (relative_path is also the citation key, so moving a file would silently change its level). (`.doc` is detected as `word-legacy` and reported **unsupported**: OOXML validation/rendering does not apply to an OLE compound file and `preprocess` converts only `.pdf`/`.docx`, so calling it supported would be a claim the pipeline cannot honour — `.docx` is unaffected) (`scanner.py` excludes non-spec furniture via `DEFAULT_EXCLUDES` + `--exclude` globs → `status: ignored`, never source evidence; `builder.py` verifies coverage-matched rendered provenance and skips only that matched origin probe) | | `loop_apidoc/url_catalog.py` | reproducible URL navigation catalog behind `catalog-url`/`select-url`: `fetch_catalog` (bounded HTTP GET of *one* entry page, size-capped, `CatalogFetchError`), `build_catalog` (parse only sidebar/nav lists into `CatalogNode`s — entry-page fragments kept as `anchor` section identities; links are recorded, never followed), `select_catalog` (pure filter by branch/term/URL; widens nothing, fetches nothing) | | `loop_apidoc/url_corpus.py` | token-efficient cached URL corpus behind `cache-url-pages`/`cache-url-entry`/`related-url-pages`: `cache_catalog_pages` (fetch each catalog URL once — anchors of the same document become `sections` — and write content-addressed `raw/.html` + `body/.txt`; failures become `status: fetch_failed` entries, not exceptions; an un-rendered SPA shell probes only same-origin `/swagger.json`, `/openapi.json`, `/v3/api-docs`, and `/api-doc/v3/sections`; only JSON with an `openapi` or `swagger` root field is stored as a separate corpus source, while failed/non-spec/generic-JSON responses are silently not recorded and the CLI warns of the shell count on stderr), `extract_page_metadata` (pure: title/headings/body/internal links/`action:`+error-code entities from `
`; an error-code entity needs a cue word — `error`/`code`/`錯誤`/`代碼`/`狀態` — within a tight window of the digits, and a year-shaped value is never one, so page scoring is not fed by dates, amounts, and rate limits), `find_related_pages` (pure evidence-based scoring — same branch / in-out links / shared entities — returning candidate cards without loading body text) | | `loop_apidoc/gitbook_llms.py` | deterministic GitBook `llms.txt` filtering/cache with safe path preservation, URL sidecars, and coverage | | `loop_apidoc/markdown_drafts/` | separate non-authoritative, line-cited Markdown endpoint/table/example drafts; never alters `source_facts` validation. Its scanner deliberately diverges from `source_facts/markdown.py` — each is looser where its own job needs it, neither is uniformly stricter, and they must not be unified, since a change made for draft output would silently move a fail-closed gate (ADR 0009, pinned in `tests/source_facts/test_scanner_divergence.py`) | | `loop_apidoc/extraction_scaffold/` | pure projection of Markdown drafts into review-only extraction-shaped inventory/endpoint JSON; `write.py` is this feature's sole atomic output exit, and agents must copy/review output before it is used as real extraction | | `loop_apidoc/html_snapshot.py` | `normalize-html-snapshot`: `html_to_markdown` (pure: readable main-document text, no invented content; `colspan`/`rowspan` expand into a rectangular grid — the spanning cell's text stays in its own column and the covered columns are left blank so a group-title row stays distinguishable from a parameter row, while `rowspan` carries the text down its own column; out-of-range/non-numeric spans count as 1, overlapping spans discard the whole table, a multi-row `thead` merges into one header row, and a nested table renders as its own table instead of appending its rows to the enclosing one — a misaligned parameter table becomes a source fact nobody wrote) + `normalize_html_snapshot` (writes the Markdown and a `.source.json` sidecar binding it to the raw file's URL + sha256) | +| `loop_apidoc/supplementary_note.py` | `import-supplementary-note`: imports a hand-written excerpt of supplier correspondence (email, chat, or a spreadsheet re-saved as a Markdown table) as a **supplementary** source. Shape is taken from `rendered_url.py` because the problem is the same — a file with no verifiable origin buys traceability with mandatory provenance fields, a timezone-aware timestamp, a file digest, a versioned sidecar, and fail-closed read-side validation. Supplementary sources may be cited and may fill `missing`; they never support `explicit_support`, and the normative document wins any disagreement. **The accepted breach:** an excerpt is written by a person who may err or over-read, and the pipeline cannot tell. `excerpted_by` buys accountability, not verifiability — that is the one real cost of this path and the only source class that carries it. | | `loop_apidoc/rendered_url.py` | first-class offline browser-rendered URL import: validates original/canonical URL, timezone-aware capture metadata, capture method, safe immutable destinations, and SHA-256; writes the unchanged HTML/Markdown source, versioned provenance sidecar, and `fetched_rendered` coverage. Its read-side verifier binds coverage → sidecar → manifest local source and fails closed on any mismatch before an origin probe. | | `loop_apidoc/source_risk/` | deterministic pre-agent gate behind `inspect-source-risk`: `models.py` (versioned `SourceRiskReport`, findings and coverage), `inspect.py` (bounded read of manifest-bound UTF-8 Markdown/HTML/OpenAPI JSON/YAML; fixed rules, a 1,000-entry report cap with fail-closed `SR-FINDINGS-TRUNCATED`, a separate 500-entry warning budget whose overflow is the warning-severity `SR-WARNINGS-TRUNCATED`, and stable `source_binding_digest`; PDF/Word, invalid UTF-8, over-`max_bytes`, and other unscannable pending sources are blockers), `loader.py` (fail-loud schema/ruleset/verdict/manifest/source-binding verification plus deterministic reinspection of current bytes), `report.py` (`source-risk-report.{json,zh-TW.md}`; findings never echo matched payloads). The ruleset covers both directions: whether a source can **manipulate** the agent (Unicode tag, bidi override, control characters, instruction-override text) and whether it **leaks material to** the agent. The leak rules reuse `privacy.py`'s deterministic patterns, and only self-evidencing material blocks: `SR-SECRET-VALUE` (blocker) is PEM private-key blocks and JWTs, whose structure is the evidence. `SR-CREDENTIAL-REFERENCE` stays a warning because a competent API document documents `Authorization: Bearer `, and whether the value is real cannot be settled from the text — blocking it would reject nearly every qualifying source, and a gate that always needs a waiver is not a gate. `SR-CONTACT-PII`, `SR-PII-VALUE`, and `SR-PAYMENT-CARD` are warnings; card candidates are Luhn-validated and the card schemes' published test numbers are excluded, since a payment document necessarily documents them. A candidate never spans a line break — joining two adjacent numeric fields produces a Luhn-valid card roughly one time in ten, and that card was never written by anyone — but every other Unicode space still separates one, because PDF extraction yields U+00A0 and zh-TW documents yield U+3000. Exit 0/1/2 means pass/reject/input error; source bytes are never rewritten. | | `loop_apidoc/source_quality/` | pre-extraction source quality gate behind `assess-sources --source-risk`: `models.py` (`QualityObservation`/`QualityFinding`, `FindingSeverity`, verdict `pass`/`reject` + `SourceDiffReport`; report embeds a verified `SourceRiskReport`; blocker observations may carry explicit HTTP(S) `required_source_refs`), `loader.py` (read side: manifest, agent-written observations JSON, and a completed assessment dir — `SourceQualityInputError`), `assess.py` (`assess_source_quality`, pure: manifest usability + observations + source-risk audit → findings; any blocker ⇒ `reject`; rejected reports aggregate a bounded ordered/de-duplicated `required_source_refs` list without fetching it), `diff.py` (`build_source_diff`, pure manifest-vs-manifest added/removed/changed), `report.py` (`write_reports` → `source-quality-report.{json,zh-TW.md}` + `source-diff.{json,md}`). `assemble` requires `--source-quality`, re-loads the reports, rejects missing/stale/mismatched embedded audits and internally inconsistent reports (strict `extra="forbid"` models; `verdict` must be `reject` exactly when a blocker finding exists, and `required_source_refs` must equal the ordered de-duplicated union from those blockers), and copies a passing pair into the run-dir's `source-quality/`. | -| `loop_apidoc/freshness/` | cheap scheduled freshness gate behind `record-fingerprint`/`check-freshness`/`check-freshness-batch`: `models.py` (`SourceKind`/`SourceStatus`/`FreshnessVerdict`, `SourceSignal`/`FingerprintEntry`/`SourceFingerprint`, `SourceResult`/`FreshnessReport`, `Watchlist`/`WatchlistItem`, `BatchItemStatus`/`BatchItemResult`/`BatchReport`, `EXIT_CODES` verdict→exit-code map, `FreshnessInputError`), `signals.py` (pure `hash_bytes`/`file_signal`/OpenAPI-version detection + `classify` comparing an observed signal to a baseline entry; network `fetch_url_signal` — OpenAPI-URL sources compare `info.version`, HTML uses ETag/Last-Modified then body sha256 — writes nothing), `record.py` (`build_fingerprint` reads a completed run's manifest + URL coverage into a `SourceFingerprint`, `write_fingerprint` — WRITE exit, refuses overwrite without `force`), `check.py` (`check_freshness` orchestration: reads the baseline fingerprint, re-derives each source's current signal via `signals.py`, aggregates into a `FreshnessReport`; writes nothing), `batch.py` (`load_watchlist` — fail-loud parse of a `freshness-watchlist.json`; `scan_watchlist` — fans `check_freshness` over each watchlist item, capturing per-item errors into that item's `BatchItemResult` rather than aborting the batch, and aggregates into a `BatchReport`; writes nothing), `report.py` (`write_reports` → `freshness-report.{json,md}` — WRITE exit; also `render_batch_markdown`/`write_batch_reports` → `freshness-scan.{json,md}` — WRITE exit) | +| `loop_apidoc/freshness/` | cheap scheduled freshness gate behind `record-fingerprint`/`check-freshness`/`check-freshness-batch`: `models.py` (`SourceKind`/`SourceStatus`/`FreshnessVerdict`, `SourceSignal`/`FingerprintEntry`/`SourceFingerprint`, `SourceResult`/`FreshnessReport`, `Watchlist`/`WatchlistItem`, `BatchItemStatus`/`BatchItemResult`/`BatchReport`, `EXIT_CODES` verdict→exit-code map, `FreshnessInputError`), `signals.py` (pure `hash_bytes`/`file_signal`/OpenAPI-version detection + `classify` comparing an observed signal to a baseline entry; network `fetch_url_signal` — OpenAPI-URL sources compare `info.version`, HTML uses ETag/Last-Modified then body sha256 — writes nothing), `record.py` (`build_fingerprint` reads a completed run's manifest + URL coverage into a `SourceFingerprint`, omitting supplementary sources — the whole gate rests on a source being re-obtainable and digest-comparable, which an excerpt of correspondence is not, so including it would only yield a permanently meaningless verdict and pollute the batch scan; `write_fingerprint` — WRITE exit, refuses overwrite without `force`), `check.py` (`check_freshness` orchestration: reads the baseline fingerprint, re-derives each source's current signal via `signals.py`, aggregates into a `FreshnessReport`; writes nothing), `batch.py` (`load_watchlist` — fail-loud parse of a `freshness-watchlist.json`; `scan_watchlist` — fans `check_freshness` over each watchlist item, capturing per-item errors into that item's `BatchItemResult` rather than aborting the batch, and aggregates into a `BatchReport`; writes nothing), `report.py` (`write_reports` → `freshness-report.{json,md}` — WRITE exit; also `render_batch_markdown`/`write_batch_reports` → `freshness-scan.{json,md}` — WRITE exit) | | `loop_apidoc/governance/` | bounded source-governance trigger behind `governance-scan`: `scan.py` (`build_governance_report`, pure classification of a batch scan into `no_action`/`review_required`/`attention_required`), `models.py` (typed trigger contract), `report.py` (`write_reports` → `governance-trigger.{json,md}` — WRITE exit). It never re-extracts, generates, imports, or approves a contract. | | `loop_apidoc/agentcli/` | `assemble.py` (assemble agent-written JSON → plan→generate→validate, `AssembleInputError` / `RunDirectoryCollisionError`), `strict.py` (blocking exact-evidence legacy→Core candidate adapter; never approves, publishes, or mutates Foundry), `input_schema.py` (typed pydantic guards, including optional v1 exact-evidence references, operational `applies_to[]`, and typed transport/amount-direction/idempotency/line-currency semantics), `evidence.py` (read-side v1 evidence materialization/digest verifier plus pure claim-path verification shared by both entry points), `source_guard.py` (pure boundary checks for the three schema contracts a subagent can't infer: `endpoints[].path` must start with `/`, each extraction file's `source` citations must name a manifest source — per-file scope, so a partially-citing file is left to validation's per-entry `SOURCE_UNVERIFIED` — and null-path endpoints must carry a `summary`), `cross_file.py` (pure cross-file invariants: endpoint files ↔ inventory — count, identity multiset (`(method, path)`, or `(method, summary)` for null-path webhooks), no duplicates, `schema_ref`/`security[]` resolution, `endpoints[].server` → `environments[].name` resolution, operational applicability → existing operation/field resolution, and typed semantic operation refs → existing operations), `gate.py` (`check_extraction`, the single pure aggregator `assemble` and `verify-extraction` both call — also folds in `source_facts`' semantic completeness and deferral checks, taking the `FactIndex` as an argument), `verify.py` (the `verify-extraction` shell: build manifest + load extraction + pure gate + evidence verifier; writes nothing), `extraction.py` (convert `inventory.json` into plan stage answers), `preprocess.py` (PDF/DOCX→markdown orchestration with full DOCX preflight before batch writes) | | `loop_apidoc/docx_normalization.py` + `docx_{models,validation,render,publish}.py` | stable DOCX facade plus bounded, fail-closed OOXML validation, deterministic rendering, and staged Markdown/`.source.json` publication with rollback on reported write failures; package validation scans every Word XML part for active DDE fields, markup alternatives, merged cells, and external content without executing or resolving relationships | | `loop_apidoc/adapters/fragments.py` | read-side I/O exit that materializes exact page/line/section/table-cell/JSON Pointer fragments from source artifacts; fragment digests use normalized fragment content, not whole-document bytes | -| `loop_apidoc/shadow/` | opt-in legacy/Core compatibility sidecar: `models.py` (mode, diagnostics, comparison, summaries), `bridge.py` (pure manifest/plan → evidence/support proposals/metadata; a v1 exact reference owns its declared claim path while filename-only legacy citations degrade to `insufficient`/unverified), `runner.py` (in-memory deterministic verification and evidence-aware projections through validate only), `report.py` (successful `core/*.json` plus `core/projections/`, or safe `core/error.json`; this package's only file-I/O exit) | +| `loop_apidoc/shadow/` | opt-in legacy/Core compatibility sidecar: `models.py` (mode, diagnostics, comparison, summaries), `bridge.py` (pure manifest/plan → evidence/support proposals/metadata; a v1 exact reference owns its declared claim path while filename-only legacy citations degrade to `insufficient`/unverified. A supplementary carrier's support proposal and evidence reference are **withdrawn** before Core sees them: filename-only citations already degrade on their own, so the exposure was the exact reference, which would otherwise carry an excerpt into a Core candidate with the manual's standing. Withdrawal rather than relabelling, because the Core model forbids a runtime proposing `insufficient` — that is Core's conclusion after verification, not a runtime assertion), `runner.py` (in-memory deterministic verification and evidence-aware projections through validate only), `report.py` (successful `core/*.json` plus `core/projections/`, or safe `core/error.json`; this package's only file-I/O exit) | | `loop_apidoc/source_facts/` | deterministic source-fact inventory feeding the semantic completeness gate (issue #14): `models.py` (`EndpointFact`/`SourceFacts`/`FactIndex`, `by_identity()` keeping only the **intersection** when several sources document one `(METHOD, path)` — an overview index table or a deprecated v1 section would otherwise widen the requirement past what the extraction was right to ignore; ambiguity fails open), `markdown.py` (`scan_markdown`, pure, and deliberately not shared with the draft scanner — ADR 0009: endpoint declarations, parameter-table field names — only tables whose first header cell is name-like, with nested-row decoration stripped and group-label rows skipped — and fenced example-block counts, fence-aware so code samples never leak facts; a closing fence must carry no info string (CommonMark), and when the scan ends inside a fence `SourceFacts.unclosed_fence_line` records where it opened so the `SOURCE_FACTS_UNSCANNED` warning names the line instead of listing possible causes — the strict rule is never relaxed into guessing a fence shut, ADR 0008. **Scope limit:** only well-structured Markdown yields facts; a flattened HTML-to-text dump yields none and the gate is a no-op on it — an accepted trade-off, since guessing structure would manufacture false facts and a false fact blocks a correct extraction — the cost is disclosed at runtime as `SOURCE_FACTS_UNSCANNED`, and `collect.py` is named in ADR 0007's falsification condition), `collect.py` (`collect_facts`, the package's only read: manifest-named Markdown sources → `FactIndex`; unreadable sources are skipped, since manifest coverage already reports them), `gate.py` (`source_fact_violations`, pure: for every extracted endpoint that matches a fact by `(METHOD, path)`, a documented field absent from every structural position — and not named in `missing[]` — or a documented example with an empty `examples[]` is a violation; no match ⇒ no judgement), `deferral.py` (`deferral_violations`, pure: rejects placeholder answers like "requires further extraction"/「需進一步擷取」 outside `missing[]`). `markdown.py` also recognises **error-code tables** into per-source `ErrorCodeFact`s, and `FactIndex.documented_error_codes()` unions them across sources into the documented error-code floor — union, not the `by_identity()` intersection, because different documents tabulate different code sets rather than competing accounts of one thing (ADR 0005). Recognition is strict: an unambiguous code header, or a generic one (`代碼`/`code`/`status code`) corroborated by an enclosing error section; at least two columns; and one malformed data row discards the whole table rather than lowering the floor silently. **`gate.py` deliberately does not consume the floor** — only focus directives are judged against it. Error codes are a document-level shared catalogue with no endpoint to match against, so an unscoped requirement would hit integrations that correctly implement only part of a provider; requiring exhaustiveness stays something a requester asks for (ADR 0006). That is a decision, not an oversight, and `source_facts/gate.py` is named in ADR 0006's falsification condition | | `loop_apidoc/focus/` | requester-authored extraction focus directives: `models.py` (strict `extra="forbid"` `FocusDirective`/`FocusResponse` contracts — `kind` is the sole determinant of severity, `intent` the sole determinant of anchor type, and the only two outcomes are `satisfied`/`not_found`; there is deliberately no "not applicable", since whether a directive applies is the requester's judgement), `loader.py` (this package's only read exit: parses `focus.json` and `/focus-response.json`, `FocusInputError`), `gate.py` (pure: directive↔response correspondence, intent↔anchor-type agreement, anchor resolution against the extraction, and the requirement that a `not_found` answer account for every readable manifest source; also projects anchor evidence for the shared exact-evidence verifier rather than verifying it separately), `fields.py`/`codes.py` (pure anchor vocabularies built on the shared field-name and typed error-catalogue readers), `report.py` (this package's only write exit: `/focus/focus-report.{json,zh-TW.md}`). Structural violations fold into `agentcli/gate.py`, so they fail before a run directory exists. The documented error-code floor is judged outside this package, in `validate/focus.py` (`omitted_error_codes`, shared by the `assemble` issue and the `verify-extraction` forecast so the two cannot disagree); `codes.py` stays the anchor vocabulary that resolves a reported code against the typed catalogue and is deliberately not the floor's source. Focus material never reaches provenance, the score, or Foundry (ADR 0004). | | `loop_apidoc/agentcli/identity.py` | the one definition of an endpoint's cross-file identity key (`METHOD /path`, or `METHOD (webhook) ` when a webhook's path is null), shared by the cross-file invariants and the focus anchor resolver | | `loop_apidoc/extraction/` | shared models + utilities (models, stages, questions, store, jsonblock) used by the agent extraction | -| `loop_apidoc/plan/` | normalization plan + source-match classification, including typed `transport[]` / `amount_direction[]` / `idempotency[]` / `line_currency_policy[]`; `claim_projection.py` is the pure, shared legacy-plan → material-claim projection used by the v1 gate and shadow bridge | +| `loop_apidoc/plan/` | normalization plan + source-match classification. `sole_source()` counts **normative** documents only: it is what lets a citation naming a section rather than a file stay attributable, and `source_guard.source_violations` is skipped on the same condition — so counting a supplementary excerpt would make adding one refuse the whole run at the boundary, before a run directory exists. A supplementary carrier supplements a corpus; it never redefines what that corpus is. Also including typed `transport[]` / `amount_direction[]` / `idempotency[]` / `line_currency_policy[]`; `claim_projection.py` is the pure, shared legacy-plan → material-claim projection used by the v1 gate and shadow bridge | | `loop_apidoc/generate/` | OpenAPI / Markdown / `review.html` / provenance / always-written `integration-contract.json` generation (`integration.py` carries source-backed typed domain semantics, operational rules, and integration mechanics; `review.py` builds the offline manual-review page; `handoff.py`'s `build_handoff` emits the derived `handoff/` pack — `integration-tasks.md` / `postman_collection.json` / `sdk-hints.json` — from OpenAPI + plan + integration, duplicating no schema) | -| `loop_apidoc/validate/` | structure / completeness / consistency / no-speculation checks + report; `coverage.py` also flags supported/readable sources with zero material citations, while `response_contract.py` reports successful path responses with no usable schema fields and computes response-contract metrics. `fact_coverage.py` (pure) makes the semantic completeness gate's no-op visible: `agentcli/fact_coverage.py` computes one per-source projection (endpoint facts scanned, facts matching an extracted endpoint identity) from the `FactIndex` and extraction `assemble`/`verify-extraction` already hold — it lives in `agentcli/` so `validate/` never depends on the extraction gate's vocabulary, and a source with zero facts, zero matches, or an unread tail behind a refused closing fence (`unclosed_fence_line`, ADR 0008 — reported even when earlier facts matched, since a half-read source is the more dangerous shape) becomes a warning-severity `SOURCE_FACTS_UNSCANNED` issue scored under source grounding. `validate_outputs` takes the projection, never the `FactIndex`; no projection means the check was not evaluated, so `validate_run_dir` never emits it. `verify-extraction` forecasts the same projection without entering `--json` or changing its exit code (ADR 0007) | +| `loop_apidoc/validate/` | structure / completeness / consistency / no-speculation checks + report. `authority.py` (pure) names **each** plan item whose only support is a supplementary carrier as a warning-severity `SUPPLEMENTARY_SUPPORT`, scored under source grounding. Deliberately per-item rather than one run-level warning: `SOURCE_FACTS_UNSCANNED` already proved that a run-level warning goes to background noise, and "the only basis for this normative claim is an email" must not end up there. The plan item's status stays `supported` — `unverified` means the citation does not resolve to a manifest source, while an excerpt resolves fine, so reusing it would make that code's remedy ("re-read the affected scope") advise something impossible. Also `coverage.py` also flags supported/readable sources with zero material citations, while `response_contract.py` reports successful path responses with no usable schema fields and computes response-contract metrics. `fact_coverage.py` (pure) makes the semantic completeness gate's no-op visible: `agentcli/fact_coverage.py` computes one per-source projection (endpoint facts scanned, facts matching an extracted endpoint identity) from the `FactIndex` and extraction `assemble`/`verify-extraction` already hold — it lives in `agentcli/` so `validate/` never depends on the extraction gate's vocabulary, and a source with zero facts, zero matches, or an unread tail behind a refused closing fence (`unclosed_fence_line`, ADR 0008 — reported even when earlier facts matched, since a half-read source is the more dangerous shape) becomes a warning-severity `SOURCE_FACTS_UNSCANNED` issue scored under source grounding. `validate_outputs` takes the projection, never the `FactIndex`; no projection means the check was not evaluated, so `validate_run_dir` never emits it. `verify-extraction` forecasts the same projection without entering `--json` or changing its exit code (ADR 0007) | | `loop_apidoc/run/` | run-id generation, result/status models, and persisting the plan into the run dir | | `loop_apidoc/diff/` | run-to-run version diff: `loader.py` (load a completed run-dir's artifacts, `DiffInputError`), `compare.py` (classify changes across `openapi.yaml` / `integration-contract.json` / `provenance.json` / `validation/report.json` / `manifest.json` into `breaking` / `additive` / `changed` / `source_only`), `models.py` (`DiffFinding` / `DiffImpact` / `DiffReport`), `report.py` (render + write `diff/report.{json,md}`) | | `loop_apidoc/review/` | local single-user Foundry review workbench behind `review`: `workflow.py` opens/imports a candidate, compares it with the current asset (or a baseline), persists only structured `review/decision.json` handoff, and approves on an explicit human action; `binding.py` fingerprints the reviewed artifacts and rejects stale decisions; `web.py` is a loopback-only, token-protected standard-library UI adapter. It never calls a model or replaces deterministic validation. | diff --git a/README.md b/README.md index dc37d10d..6f35ad18 100644 --- a/README.md +++ b/README.md @@ -430,6 +430,35 @@ versioned provenance sidecar；格式錯誤與任何輸出碰撞都會拒絕。 `fetched_rendered` 結果讓 `manifest`／`assemble` 使用通過驗證的本機快照，不再探測受保護 origin；URL、路徑、method 或 digest 不一致則 fail closed。 +### 次級佐證：供應商補充說明 + +有些規範性資訊只存在於供應商的信件或通訊軟體訊息裡——金鑰怎麼取得、測試環境位址、 +上線前必須完成的事——而供應商往往沒有打算把它寫進文件。這類內容可以人工摘錄成 +Markdown 後匯入，標記為**次級佐證**： + +```bash +uv run loop-apidoc import-supplementary-note \ + --input ./notes/sandbox-key.md \ + --from "engineer@provider.example" \ + --received-at "2026-08-16T10:00:00+08:00" \ + --subject "測試環境金鑰取得方式" \ + --excerpted-by "carl" \ + --sources ./sources +``` + +指令寫出摘錄本身與一份 `.source.json` sidecar，記錄出處、收到時間、摘錄者與 SHA-256； +`manifest` 讀 sidecar 把該來源標成 `authority: supplementary`，沒有 sidecar 的來源一律是 +`normative`。供應商另外給的欄位對照試算表，另存為 Markdown 表格後走同一條路徑。 + +次級佐證做得到的事：被引用、填補 `missing`、讓一條主張成立。做不到的事：與正式文件 +衝突時勝出、與正式文件在報告裡混為一談。凡是只靠次級佐證才成立的主張，都會在驗證 +報告裡以 warning 級的 `SUPPLEMENTARY_SUPPORT` **逐條**點名（不是一則籠統的 run 層警告）， +並計入文件品質分數的 source grounding。這類來源也不會進入 `record-fingerprint` 的指紋—— +新鮮度比對的前提是來源可被重新取得，而一封信沒有 URL、沒有版本。 + +**要接受的代價**：摘錄是人寫的，摘錄者可能寫錯或過度解讀，pipeline 無法分辨。 +sidecar 記下摘錄者是為了**可追責**，不是可驗證。這是這條路徑與其他所有來源的本質差異。 + 若 URL 本身就是 Swagger 2.0 或 OpenAPI 3.x JSON／YAML，請先把它固定為本機來源，而不是 走 HTML 導覽流程： diff --git a/docs/adr/0010-supplementary-carriers-are-accountable-not-verifiable.md b/docs/adr/0010-supplementary-carriers-are-accountable-not-verifiable.md new file mode 100644 index 00000000..e005b122 --- /dev/null +++ b/docs/adr/0010-supplementary-carriers-are-accountable-not-verifiable.md @@ -0,0 +1,112 @@ +--- +status: accepted +--- + +# Supplementary carriers buy accountability, not verifiability + +Some normative information about a supplier's API exists only in correspondence — how to obtain +a sandbox key, where the test environment lives, what must be done before go-live, what a +merchant parameter actually means. It is not in any document, and the supplier often has no +intention of putting it there. The same is true of the field-mapping spreadsheet that arrives +alongside the manual. + +Until now there were two options and both were bad. Leave it out: the information becomes +`missing`, the integration contract has a hole, but the report is honest. Or let the agent read +it and write it into the extraction: the contract is complete, and `provenance.json` now claims +a normative claim has source support that exists in no manifest entry. The second option makes +the report lie, and nothing in the pipeline would ever notice. + +## The decision + +Correspondence excerpts and re-saved spreadsheets enter as a third thing: a **supplementary** +source. They may be cited, may fill `missing`, and may make a claim stand. They may not outrank +a formal document, and they may not be indistinguishable from one in a report. + +The level lives on the source, not on the claim. Being unre-obtainable is a property the carrier +has, and it holds for every claim that carrier ever supports; putting it on the claim would +restate the same judgement N times and give N chances to get it wrong. It is also deliberately +not folded into `derived_support` — that relationship means *inferential distance*, and mixing +carrier credibility into it would leave nobody able to tell which of the two a +`derived_support` meant. + +The dividing line is re-obtainability, not medium. A supplier engineer's email is written by the +supplier, exactly as the PDF is; medium is not what makes it weaker. What makes it weaker is +that `freshness/` compares SHA-256 to detect drift and `governance/` triggers re-review from +that comparison, and an email has no URL, no version, and no second fetch. Anything that +`check-freshness` cannot periodically re-walk is supplementary, whatever it is made of. + +## The breach we are accepting + +An excerpt is written by a person. That person can transcribe wrongly, compress away a +condition, or read more into a sentence than it said — and no part of the pipeline can tell. +Every other source class in this repository is verifiable: the bytes are hashed, the fragment is +addressable, and a reader can go back to the original and check. This one is not. + +`excerpted_by` is therefore not a verification mechanism. It records **who to ask** when a claim +turns out to be wrong. That is strictly weaker than what every other source offers, it is the +entire cost of this path, and it is the reason a supplementary source can never reach +`explicit_support` no matter how well written the excerpt is. + +We accept it because the alternative is worse in a specific way. Excluding the carrier does not +make the information stop existing — it makes it arrive through an unrecorded channel, because +an engineer who knows how the sandbox key is obtained will type it into the extraction whether +or not there is a legitimate path for it. A recorded weak source beats an unrecorded one +masquerading as a strong one. + +## What follows from it, and what does not + +Two questions that looked like one turn out to be different, and they get different functions. + +`source_guard` skips `source_violations` entirely when attribution is unambiguous, and it asks +`sole_normative_source()` — supplementary carriers ignored. Otherwise adding one excerpt to a +single-manual corpus would refuse the entire run at the input boundary, before a run directory +exists. Supporting material must never cost the manual its run. + +`classify_item` asks `sole_source()`, which counts every usable document including supplementary +ones. Its fallback attributes an unresolvable locator to the one document present, and that +licence rests entirely on there being only one. An excerpt is a second document. Were it +excluded here, a citation reading "供應商信件 2026-08-16" would be recorded as `supported` by +`manual.md` — a claim attributed to a document that never made it, and `SUPPLEMENTARY_SUPPORT` +would stay silent because the citation now names the manual. That is precisely the failure this +distinction exists to prevent, reintroduced by the fix for it. + +The asymmetry is the point: skipping a boundary check defers reporting to per-entry validation, +while attributing a locator asserts something about a document's contents. Only the first stays +safe once a second document exists. With an excerpt in the corpus, an unresolvable locator is +ambiguous and stays `UNVERIFIED` until the agent cites precisely. + +A claim resting only on a supplementary source is named individually, as a warning-severity +`SUPPLEMENTARY_SUPPORT` at that plan item, and not as one run-level warning. +`SOURCE_FACTS_UNSCANNED` already demonstrated where run-level warnings end up: nine benchmark +cases carry it permanently and it now needs a paragraph of documentation explaining how to tell +its three causes apart. "The only basis for this normative claim is an email" must not become +background noise. + +The plan item stays `supported`. `unverified` means the citation does not resolve to a manifest +source; an excerpt resolves fine — it has a file, a digest, and a sidecar. Reusing that status +would make its remedy ("re-read the affected scope and add the citation") advise something that +cannot be done, because there is nothing to re-read. + +In `shadow` and `strict`, a supplementary carrier's support proposal and evidence reference are +withdrawn before the Core sees them. Filename-only legacy citations already degrade on their +own, so the exposure was the v1 exact reference: it owns its declared claim path and would +otherwise carry an excerpt into a Core candidate with the manual's standing. The withdrawal is +deliberately not a relabelling to `insufficient` — the Core model forbids a runtime from +proposing that, because insufficiency is Core's conclusion after verification, not a runtime +assertion. Removing the proposal lets Core reach that conclusion itself. + +**Not decided here:** what happens when a supplementary source and a formal document disagree. +The intended rule is that the formal document wins without raising `SOURCE_CONFLICT`, since the +resolution is deterministic and needs no human judgement. The deterministic layer cannot enforce +it today: `source_conflicts[]` is free text declared by the extraction agent with no per-source +attribution, so nothing in `validate/` can tell which side of a conflict is supplementary. +Enforcing it requires extending the extraction schema, which is a separate change. Until then +the rule lives as guidance to the extraction agent. + +**Falsified if:** a supplementary source becomes indistinguishable from a normative one at any +point where a claim's support is judged. Concretely, this decision no longer holds when +`loop_apidoc/agentcli/source_guard.py` stops asking `sole_normative_source()`, when +`loop_apidoc/validate/authority.py` stops naming supplementary-only claims individually, when +`loop_apidoc/shadow/bridge.py` lets a supplementary citation reach `explicit_support`, when +`loop_apidoc/freshness/record.py` fingerprints a supplementary source, or when +`loop_apidoc/manifest/models.py` drops `SourceAuthority`. diff --git a/loop_apidoc/agentcli/assemble.py b/loop_apidoc/agentcli/assemble.py index b0784a4e..e9090ea3 100644 --- a/loop_apidoc/agentcli/assemble.py +++ b/loop_apidoc/agentcli/assemble.py @@ -31,7 +31,8 @@ from loop_apidoc.extraction.stages import QueryKind from loop_apidoc.extraction.store import ExtractionStore from loop_apidoc.generate.writer import generate_outputs -from loop_apidoc.manifest.builder import ManifestInputError, build_manifest +from loop_apidoc.manifest.builder import build_manifest +from loop_apidoc.manifest.scanner import ManifestScanError from loop_apidoc.manifest.models import Manifest, UrlSource from loop_apidoc.plan.builder import build_normalization_plan from loop_apidoc.plan.integration import build_integration_contract @@ -302,7 +303,7 @@ def run_assemble_pipeline( excludes=excludes, url_coverage=url_coverage, ) - except ManifestInputError as exc: + except ManifestScanError as exc: raise AssembleInputError(str(exc)) from exc from loop_apidoc.source_risk import SourceRiskInputError, verify_source_risk_report diff --git a/loop_apidoc/agentcli/source_guard.py b/loop_apidoc/agentcli/source_guard.py index 80545982..adedc273 100644 --- a/loop_apidoc/agentcli/source_guard.py +++ b/loop_apidoc/agentcli/source_guard.py @@ -17,7 +17,7 @@ from typing import Any from loop_apidoc.manifest.models import Manifest -from loop_apidoc.plan.classify import match_manifest_source, sole_source +from loop_apidoc.plan.classify import match_manifest_source, sole_normative_source # Inventory sections whose entries each carry a `source`, per # reference/extraction-schemas.md. @@ -150,7 +150,7 @@ def source_violations( `plan.classify.classify_item` already treats such citations as supported. Checking here would reject inputs the pipeline accepts. """ - if sole_source(manifest) is not None: + if sole_normative_source(manifest) is not None: return [] inventory_scope = [ diff --git a/loop_apidoc/cli.py b/loop_apidoc/cli.py index 571add44..51426707 100644 --- a/loop_apidoc/cli.py +++ b/loop_apidoc/cli.py @@ -89,6 +89,7 @@ def manifest( sources.relative_to(sources_root).as_posix() if sources.is_file() else None ) from loop_apidoc.manifest.builder import ManifestInputError + from loop_apidoc.manifest.scanner import ManifestScanError from loop_apidoc.preparation.coverage import CoverageInputError, load_coverage try: @@ -104,7 +105,7 @@ def manifest( excludes=tuple(exclude), url_coverage=parsed_coverage, ) - except (CoverageInputError, ManifestInputError) as exc: + except (CoverageInputError, ManifestScanError) as exc: typer.echo(f"manifest error: {exc}", err=True) raise typer.Exit(code=2) from exc if selected_relative_path is not None: @@ -374,6 +375,49 @@ def snapshot_openapi_url_command( ) +@app.command(name="import-supplementary-note") +def import_supplementary_note_command( + input: Path = typer.Option( + ..., "--input", exists=True, readable=True, help="人工摘錄的 Markdown" + ), + received_from: str = typer.Option( + ..., "--from", help="誰給的：寄件者、通訊軟體帳號或提供者" + ), + received_at: str = typer.Option( + ..., "--received-at", help="含時區的 ISO-8601 收到時間" + ), + excerpted_by: str = typer.Option(..., "--excerpted-by", help="摘錄者"), + sources: Path = typer.Option(..., "--sources", help="不可變本機來源目錄"), + subject: str | None = typer.Option(None, "--subject", help="信件主旨或標題"), + filename: str | None = typer.Option( + None, "--filename", help="來源檔名；預設沿用輸入檔名" + ), +) -> None: + """匯入供應商補充說明的人工摘錄，標記為次級佐證。""" + from loop_apidoc.supplementary_note import ( + SupplementaryNoteError, + import_supplementary_note, + ) + + try: + result = import_supplementary_note( + input, + received_from=received_from, + received_at=received_at, + excerpted_by=excerpted_by, + sources=sources, + subject=subject, + filename=filename, + ) + except (SupplementaryNoteError, OSError) as exc: + typer.echo(f"import-supplementary-note error: {exc}", err=True) + raise typer.Exit(code=2) from exc + typer.echo( + f"次級佐證已寫入 {result.source_path}；SHA-256 {result.sha256}；" + f"provenance 已寫入 {result.provenance_path}" + ) + + @app.command(name="import-rendered-url") def import_rendered_url_command( input: Path = typer.Option( diff --git a/loop_apidoc/freshness/record.py b/loop_apidoc/freshness/record.py index 9742ccd2..149fcefc 100644 --- a/loop_apidoc/freshness/record.py +++ b/loop_apidoc/freshness/record.py @@ -13,7 +13,7 @@ SourceSignal, ) from loop_apidoc.freshness.signals import fetch_url_signal -from loop_apidoc.manifest.models import Manifest, ProcessingStatus +from loop_apidoc.manifest.models import Manifest, ProcessingStatus, SourceAuthority from loop_apidoc.preparation.coverage import CoverageInputError, ResultStatus, load_coverage _USABLE_URL_STATUSES = {ResultStatus.FETCHED, ResultStatus.FETCHED_RENDERED} @@ -55,6 +55,10 @@ def build_fingerprint( for src in manifest.local_sources: if src.status is not ProcessingStatus.PENDING: continue + # 次級佐證不可重新取得 —— 沒有 URL、沒有版本。納入指紋只會讓 + # watchlist 對它永遠給出無意義的判定,並污染批次掃描的結論。 + if src.authority is SourceAuthority.SUPPLEMENTARY: + continue entries.append(FingerprintEntry( id=src.relative_path, kind=SourceKind.LOCAL_FILE, @@ -93,6 +97,14 @@ def build_fingerprint( if owns_client: active_client.close() + if not entries: + # 空指紋不會失敗,它會永遠回報新鮮 —— 每一次 check-freshness 都在 + # 對零份來源比對雜湊。那是無聲的監控喪失,而次級佐證被排除之後 + # 一份全由摘錄組成的 run 正好會落到這裡。 + raise FreshnessInputError( + f"{run_dir}：沒有任何可重新取得的來源，" + "指紋會讓 check-freshness 永遠回報新鮮" + ) return SourceFingerprint( openapi_version=openapi_version, recorded_from=run_dir.name, diff --git a/loop_apidoc/manifest/builder.py b/loop_apidoc/manifest/builder.py index 9ae860cb..44319cb8 100644 --- a/loop_apidoc/manifest/builder.py +++ b/loop_apidoc/manifest/builder.py @@ -7,13 +7,13 @@ import httpx from loop_apidoc.manifest.models import Manifest, UrlSource -from loop_apidoc.manifest.scanner import scan_sources +from loop_apidoc.manifest.scanner import ManifestScanError, scan_sources from loop_apidoc.manifest.urls import probe_url from loop_apidoc.preparation.coverage import UrlCoverage from loop_apidoc.rendered_url import canonicalize_url, verified_rendered_url_sources -class ManifestInputError(ValueError): +class ManifestInputError(ManifestScanError): """Manifest URL evidence is malformed or internally inconsistent.""" diff --git a/loop_apidoc/manifest/models.py b/loop_apidoc/manifest/models.py index 19c4b0a5..76766b8f 100644 --- a/loop_apidoc/manifest/models.py +++ b/loop_apidoc/manifest/models.py @@ -30,6 +30,17 @@ class ProcessingStatus(str, Enum): IGNORED = "ignored" +class SourceAuthority(str, Enum): + #: 供應商正式文件。現存的每一份來源都是這個等級 —— 預設值是事實, + #: 不是相容性妥協。 + NORMATIVE = "normative" + #: 供應商補充說明的人工摘錄(信件、通訊軟體、另存成 Markdown 的 + #: 試算表)。填得了 `missing`,支撐不了 `explicit_support`;與正式 + #: 文件衝突時正式文件勝。分界線是可重新取得性:這種載體沒有 URL、 + #: 沒有版本,`check-freshness` 無法週期性重走。 + SUPPLEMENTARY = "supplementary" + + class LocalSource(BaseModel): relative_path: str mime_type: str | None @@ -40,6 +51,7 @@ class LocalSource(BaseModel): supported: bool status: ProcessingStatus duplicate_of: str | None = None + authority: SourceAuthority = SourceAuthority.NORMATIVE class UrlSource(BaseModel): diff --git a/loop_apidoc/manifest/scanner.py b/loop_apidoc/manifest/scanner.py index 71b6c499..073863a1 100644 --- a/loop_apidoc/manifest/scanner.py +++ b/loop_apidoc/manifest/scanner.py @@ -1,13 +1,29 @@ from __future__ import annotations import hashlib +import json from collections.abc import Sequence from datetime import datetime from fnmatch import fnmatch from pathlib import Path, PurePosixPath +from pydantic import ValidationError + from loop_apidoc.manifest.formats import detect_format, guess_mime_type, is_supported -from loop_apidoc.manifest.models import LocalSource, ProcessingStatus +from loop_apidoc.manifest.models import ( + LocalSource, + ProcessingStatus, + SourceAuthority, +) +from loop_apidoc.supplementary_note import SupplementaryProvenance + + +class ManifestScanError(ValueError): + """A source's authority cannot be determined, so scanning must not continue.""" + + +#: sidecar 是人或工具寫的小 JSON;超過這個大小代表它不是 sidecar。 +_MAX_SIDECAR_BYTES = 64 * 1024 _CHUNK_SIZE = 1 << 20 # 1 MiB @@ -29,6 +45,57 @@ ) +def _read_authority(path: Path, root_resolved: Path, sha256: str) -> SourceAuthority: + """Read the source's authority from its `.source.json` sidecar. + + **缺席才是 normative,讀不動不是。** 沒有 sidecar 的來源是正式文件 —— + 現存的每一份來源都是,這是事實而非為了相容編出來的預設值。但一個 + *存在卻讀不動*的 sidecar 意味著等級無從判定,而 fail open 的後果是 + 一份次級佐證靜默升級成正式文件:它重新進入 `sole_source()` 與 + `build_fingerprint`,`SUPPLEMENTARY_SUPPORT` 一條都不會報,而操作者 + 看不到任何差別。一個截斷的寫入或錯誤的權限就足以關掉整個功能。 + + 宣告必須綁在內容上:`source_file` 與 `imported_sha256` 都要對得上, + 否則一個從別處複製來的兩行 sidecar 就能把一份正式手冊降級。 + """ + sidecar = path.with_suffix(path.suffix + ".source.json") + if not sidecar.exists(): + return SourceAuthority.NORMATIVE + if not _within_root(sidecar, root_resolved) or sidecar.is_symlink(): + raise ManifestScanError(f"來源 sidecar 不在來源目錄內：{sidecar}") + try: + if sidecar.stat().st_size > _MAX_SIDECAR_BYTES: + raise ManifestScanError(f"來源 sidecar 過大：{sidecar}") + payload = json.loads(sidecar.read_text(encoding="utf-8")) + except OSError as exc: + raise ManifestScanError( + f"來源 sidecar 存在但讀不動，無法判定來源等級：{sidecar}" + ) from exc + except ValueError as exc: + raise ManifestScanError( + f"來源 sidecar 不是合法 JSON，無法判定來源等級：{sidecar}" + ) from exc + if not isinstance(payload, dict) or "authority" not in payload: + # `import-rendered-url` 寫的 provenance 沒有 authority 欄位 —— + # 那是一份已驗證出處的正式文件,不是判定失敗。 + return SourceAuthority.NORMATIVE + try: + provenance = SupplementaryProvenance.model_validate(payload) + except ValidationError as exc: + raise ManifestScanError( + f"來源 sidecar 宣告了 authority 但格式不合：{sidecar}：{exc}" + ) from exc + if provenance.source_file != path.name: + raise ManifestScanError( + f"來源 sidecar 的 source_file 與檔名不符：{sidecar}" + ) + if provenance.imported_sha256 != sha256: + raise ManifestScanError( + f"來源 sidecar 的 SHA-256 與來源內容不符：{sidecar}" + ) + return SourceAuthority.SUPPLEMENTARY + + def is_excluded(relative_path: str, patterns: Sequence[str]) -> bool: """A pattern matches either the whole POSIX relative path or the basename.""" name = PurePosixPath(relative_path).name @@ -161,6 +228,7 @@ def scan_sources( supported=supported, status=status, duplicate_of=duplicate_of, + authority=_read_authority(path, root_resolved, sha256), ) ) diff --git a/loop_apidoc/plan/classify.py b/loop_apidoc/plan/classify.py index 9b9171f4..ccb81bdf 100644 --- a/loop_apidoc/plan/classify.py +++ b/loop_apidoc/plan/classify.py @@ -5,7 +5,7 @@ from pathlib import Path from loop_apidoc.extraction.evidence import ExtractionEvidenceReference -from loop_apidoc.manifest.models import Manifest, ProcessingStatus +from loop_apidoc.manifest.models import Manifest, ProcessingStatus, SourceAuthority from loop_apidoc.plan.models import PlanItemStatus, SourceCitation _UNUSABLE = ( @@ -54,8 +54,21 @@ def sole_source(manifest: Manifest) -> str | None: """Return the lone usable *document*'s identifier if the manifest collapses to exactly one, else None. - A document is a usable local source, plus each URL whose snapshot_file does NOT - point at a usable local source. A URL saved as a local snapshot (per the + Supplementary sources DO count here, and that is the point: this function + licenses attributing an unresolvable locator to the one document present, + and that licence rests entirely on there being only one. An excerpt of + correspondence is a second document. Excluding it would let a citation + reading "供應商信件 2026-08-16" be recorded as supported by the manual — + a claim attributed to a document that never made it, which is the exact + failure this whole authority distinction exists to prevent. With an excerpt + in the corpus, an unresolvable locator is ambiguous and must stay + `UNVERIFIED` until the agent cites precisely. + + `source_guard` deliberately keys off `sole_normative_source` instead, so + adding an excerpt still does not get the whole run refused at the boundary. + + A document is a usable local source, plus each URL whose snapshot_file does + NOT point at a usable local source. A URL saved as a local snapshot (per the url-fetching SOP) is the SAME document as that snapshot, so it is not counted twice — otherwise every SOP-following URL run would have ≥2 documents and lose single-source attribution. When exactly one document remains, a citation that @@ -79,6 +92,35 @@ def sole_source(manifest: Manifest) -> str | None: return documents[0] if len(documents) == 1 else None +def sole_normative_source(manifest: Manifest) -> str | None: + """Return the lone usable *normative* document, ignoring supplementary ones. + + `source_guard.source_violations` skips its whole check when this is not + None. It must ignore supplementary carriers: otherwise adding one excerpt + to a single-manual corpus would refuse the entire run at the input + boundary, before a run directory exists — supporting material must never + cost the manual its run. + + This is deliberately NOT what `classify_item` uses. Skipping a boundary + check is a decision to report problems per-entry later; attributing an + unresolvable locator to a document is a decision to assert something about + that document. Only the first is safe once a second document exists. + """ + documents = [ + s.relative_path + for s in manifest.local_sources + if s.supported + and s.status not in _UNUSABLE + and s.authority is SourceAuthority.NORMATIVE + ] + usable_set = set(documents) + for url_source in manifest.url_sources: + if url_source.snapshot_file is not None and url_source.snapshot_file in usable_set: + continue + documents.append(url_source.url) + return documents[0] if len(documents) == 1 else None + + def classify_item( locator: str | None, *, diff --git a/loop_apidoc/score/evaluate.py b/loop_apidoc/score/evaluate.py index b1cf509f..c3390e80 100644 --- a/loop_apidoc/score/evaluate.py +++ b/loop_apidoc/score/evaluate.py @@ -26,6 +26,9 @@ # 所以歸 source grounding 而不是 completeness。 IssueCode.SOURCE_FACTS_UNSCANNED: ScoreCategory.SOURCE_GROUNDING, IssueCode.UNSUPPORTED_ASSERTION: ScoreCategory.SOURCE_GROUNDING, + # 一份契約有多少比例是靠信件撐起來的,正是 source grounding 要回答的 + # 問題。它不是成品缺了資訊 —— 資訊在,只是背書比較薄。 + IssueCode.SUPPLEMENTARY_SUPPORT: ScoreCategory.SOURCE_GROUNDING, } _WARNING_PENALTY = 12 diff --git a/loop_apidoc/shadow/bridge.py b/loop_apidoc/shadow/bridge.py index 37f1a94f..b23df34c 100644 --- a/loop_apidoc/shadow/bridge.py +++ b/loop_apidoc/shadow/bridge.py @@ -42,6 +42,7 @@ LocalSource, Manifest, ProcessingStatus, + SourceAuthority, ) from loop_apidoc.plan.claim_projection import iter_plan_claim_projections from loop_apidoc.plan.models import NormalizationPlan, PlanItemStatus, SourceCitation @@ -63,6 +64,17 @@ class BridgeInputs(FrozenModel): citation_fragments: tuple[tuple[str, tuple[str, ...]], ...] diagnostics: tuple[BridgeDiagnostic, ...] = () source_set_digest: str + #: Manifest paths of supplementary carriers. Stored as source locators + #: rather than fragment ids because `with_materialized_evidence` replaces + #: the whole bundle — fragment ids do not survive it, locators do. + supplementary_sources: tuple[str, ...] = () + + def supplementary_fragment_ids(self) -> frozenset[str]: + return frozenset( + fragment_id + for source in self.supplementary_sources + for fragment_id in self.resolve_citation(source) + ) def resolve_citation(self, manifest_source: str | None) -> tuple[str, ...]: if manifest_source is None: @@ -169,6 +181,7 @@ def build_evidence(manifest: Manifest, generated_at: datetime) -> BridgeInputs: fragments: list[EvidenceFragment] = [] citations: dict[str, set[str]] = {} local_fragments: dict[str, tuple[str, ...]] = {} + supplementary_sources: list[str] = [] diagnostics: list[BridgeDiagnostic] = [] for source in sorted(manifest.local_sources, key=lambda item: item.relative_path): @@ -191,6 +204,8 @@ def build_evidence(manifest: Manifest, generated_at: datetime) -> BridgeInputs: fragments.append(fragment) citations.setdefault(source.relative_path, set()).add(fragment.id) local_fragments[source.relative_path] = (fragment.id,) + if source.authority is SourceAuthority.SUPPLEMENTARY: + supplementary_sources.append(source.relative_path) urls_by_locator: dict[str, list] = {} for source in manifest.url_sources: @@ -255,6 +270,7 @@ def build_evidence(manifest: Manifest, generated_at: datetime) -> BridgeInputs: ), diagnostics=tuple(diagnostics), source_set_digest=source_set_digest, + supplementary_sources=tuple(supplementary_sources), ) @@ -316,6 +332,57 @@ def build_fragment_requests( return tuple(requests[key] for key in sorted(requests)) +def _downgrade_supplementary( + candidate: _ProposalCandidate, + supplementary: frozenset[str], +) -> _ProposalCandidate: + """Never let a supplementary carrier reach `explicit_support`. + + Filename-only legacy citations already degrade to `insufficient`, so the + exposure is the v1 exact reference: it owns its declared claim path and + would otherwise carry an excerpt of correspondence straight into a Core + candidate with the same standing as the manual. `strict` mode exists to + require exact evidence for every claim it writes — admitting an + unre-obtainable, human-transcribed carrier there is worse than admitting + it into legacy validation, not better (ADR 0010). + """ + if not supplementary: + return candidate + if not any( + proposal.fragment_id in supplementary + for proposal in candidate.support_proposals + ): + return candidate + # Core 只接受 runtime 提出 explicit 或 derived —— insufficient 是它自己 + # 驗證後的結論,不是 runtime 可以宣告的。所以撤掉提案與證據引用,讓 + # Core 依它自己的邏輯判定不足,而不是由這裡冒充那個判斷。 + proposals = tuple( + proposal + for proposal in candidate.support_proposals + if proposal.fragment_id not in supplementary + ) + evidence_refs = tuple( + ref for ref in candidate.evidence_refs if ref not in supplementary + ) + diagnostics = ( + *candidate.diagnostics, + BridgeDiagnostic( + code="CITATION_SUPPLEMENTARY", + message=( + "supplementary carrier cannot support a normative claim; " + "its support proposal and evidence reference were withdrawn" + ), + plan_location=candidate.plan_location, + ), + ) + return replace( + candidate, + support_proposals=proposals, + evidence_refs=evidence_refs, + diagnostics=diagnostics, + ) + + def build_runtime_result( plan: NormalizationPlan, bridge: BridgeInputs, @@ -323,7 +390,10 @@ def build_runtime_result( runtime_identity: str = SHADOW_RUNTIME_IDENTITY, runtime_version: str = SHADOW_RUNTIME_VERSION, ) -> RuntimeResult: - candidates = _proposal_candidates(plan, bridge, _BridgeLookup.build(bridge)) + candidates = [ + _downgrade_supplementary(candidate, bridge.supplementary_fragment_ids()) + for candidate in _proposal_candidates(plan, bridge, _BridgeLookup.build(bridge)) + ] diagnostics: list[BridgeDiagnostic] = [ diagnostic for candidate in candidates diff --git a/loop_apidoc/supplementary_note.py b/loop_apidoc/supplementary_note.py new file mode 100644 index 00000000..3b8f4a46 --- /dev/null +++ b/loop_apidoc/supplementary_note.py @@ -0,0 +1,140 @@ +"""Import a hand-written excerpt of supplier correspondence as a supplementary source. + +供應商信件與通訊軟體裡的補充說明常常是規範性資訊的唯一出處,但它不是 +文件:沒有 URL、沒有版本、`check-freshness` 無法週期性重走。這條路徑讓 +它進得來、留得下出處,而不冒充正式文件 —— manifest 會把它標成 +`supplementary`,填得了 `missing`,支撐不了 `explicit_support`。 + +形狀取自 `rendered_url.py`:那條路徑要解的問題完全相同 —— 一份沒有可 +驗證出處的檔案,靠強制記錄出處欄位、帶時區的時間戳、檔案雜湊、版本化 +sidecar 與 fail-closed 的讀取端驗證,換得可追溯性。 + +**接受的破口**:摘錄是人寫的,摘錄者可能寫錯或過度解讀,pipeline 無法 +分辨。`excerpted_by` 記下的是**可追責**,不是可驗證。 +""" + +from __future__ import annotations + +import hashlib +import json +from dataclasses import dataclass +from datetime import datetime +from pathlib import Path +from typing import Literal + +from pydantic import BaseModel, ConfigDict, field_validator + + +SUPPLEMENTARY_SUFFIXES = {".md", ".markdown"} + + +class SupplementaryNoteError(ValueError): + """The excerpt provenance is invalid or would overwrite existing evidence.""" + + +class SupplementaryProvenance(BaseModel): + model_config = ConfigDict(extra="forbid") + + schema_version: Literal[1] + authority: Literal["supplementary"] + received_from: str + received_at: datetime + subject: str | None = None + excerpted_by: str + imported_sha256: str + source_file: str + + @field_validator("received_at") + @classmethod + def _timezone_aware(cls, value: datetime) -> datetime: + if value.tzinfo is None or value.utcoffset() is None: + raise ValueError("received_at must include a timezone offset") + return value + + +@dataclass(frozen=True) +class SupplementaryNoteImport: + source_path: Path + provenance_path: Path + sha256: str + + +def _received_time(value: str) -> datetime: + """Return the normalized instant, not the operator's spelling. + + `+0800` 與 `+08:00`、結尾 `Z` 與 `+00:00` 是同一個時刻的不同寫法。 + 保留原樣會讓兩份 sidecar 的時間欄位無法直接比較,而 `rendered_url.py` + 寫的是 `isoformat()` —— 形狀既然取自它,這裡就不該各走各的。 + """ + try: + received_at = datetime.fromisoformat(value.replace("Z", "+00:00")) + except ValueError as exc: + raise SupplementaryNoteError( + "received_at must be an ISO-8601 timestamp" + ) from exc + if received_at.tzinfo is None or received_at.utcoffset() is None: + raise SupplementaryNoteError("received_at must include a timezone offset") + return received_at + + +def _source_name(input_file: Path, filename: str | None) -> str: + name = filename or input_file.name + candidate = Path(name) + if candidate.name != name or name in {".", ".."}: + raise SupplementaryNoteError("filename must be a single file name") + if candidate.suffix.lower() not in SUPPLEMENTARY_SUFFIXES: + raise SupplementaryNoteError("supplementary excerpt must be Markdown") + return name + + +def import_supplementary_note( + input_file: Path, + *, + received_from: str, + received_at: str, + excerpted_by: str, + sources: Path, + subject: str | None = None, + filename: str | None = None, +) -> SupplementaryNoteImport: + """Copy the excerpt into `sources/` and write its immutable provenance.""" + if not received_from.strip(): + raise SupplementaryNoteError("received_from must not be empty") + if not excerpted_by.strip(): + raise SupplementaryNoteError("excerpted_by must not be empty") + received = _received_time(received_at) + name = _source_name(input_file, filename) + source_path = sources / name + provenance_path = source_path.with_suffix(source_path.suffix + ".source.json") + for output in (source_path, provenance_path): + if output.exists() or output.is_symlink(): + raise SupplementaryNoteError(f"output already exists: {output}") + + try: + raw = input_file.read_bytes() + except OSError as exc: + raise SupplementaryNoteError(f"cannot read excerpt: {exc}") from exc + digest = hashlib.sha256(raw).hexdigest() + # 寫入端與讀取端共用同一個模型:手組 dict 會讓 schema 與實際寫出的 + # 檔案無聲漂移,而讀取端是 fail-closed 的,漂移會變成拒絕。 + provenance = SupplementaryProvenance( + schema_version=1, + authority="supplementary", + received_from=received_from, + received_at=received, + subject=subject, + excerpted_by=excerpted_by, + imported_sha256=digest, + source_file=name, + ).model_dump(mode="json", exclude_none=True) + + sources.mkdir(parents=True, exist_ok=True) + with source_path.open("xb") as handle: + handle.write(raw) + with provenance_path.open("x", encoding="utf-8") as handle: + handle.write(json.dumps(provenance, ensure_ascii=False, indent=2)) + return SupplementaryNoteImport( + source_path=source_path, + provenance_path=provenance_path, + sha256=digest, + ) diff --git a/loop_apidoc/validate/authority.py b/loop_apidoc/validate/authority.py new file mode 100644 index 00000000..09e88fc4 --- /dev/null +++ b/loop_apidoc/validate/authority.py @@ -0,0 +1,94 @@ +"""Name every claim whose only support is a supplementary carrier. + +刻意做成逐條而不是 run 層級的單一警告。`SOURCE_FACTS_UNSCANNED` 的前例 +已經證明 run 層警告會噪音化 —— 九個 benchmark case 永久帶著它,以致於 +需要一整段文件解釋該警告的三種成因如何分辨。「這條規範性主張的唯一依據 +是一封信」不該落到同一個下場。 + +Pure: 讀 plan 與 manifest,不碰檔案系統。 +""" + +from __future__ import annotations + +from collections.abc import Iterator + +from loop_apidoc.manifest.models import Manifest, SourceAuthority +from loop_apidoc.plan.models import NormalizationPlan, SourceCitation +from loop_apidoc.validate.models import Issue, IssueCode, Severity + +_PLAN_SECTIONS = ( + "environments", + "security_schemes", + "endpoints", + "schemas", + "errors", + "operational", +) +_INTEGRATION_SECTIONS = ( + "transport", + "amount_direction", + "idempotency", + "line_currency_policy", + "crypto", + "callbacks", + "field_conditions", + "test_cases", +) + + +def _cited_entries(plan: NormalizationPlan) -> Iterator[tuple[str, list[SourceCitation]]]: + for section in _PLAN_SECTIONS: + for index, entry in enumerate(getattr(plan, section) or []): + yield f"{section}[{index}]", entry.citations + # Schema 母層的 citations 已併入欄位引用,所以一個只有某欄位靠摘錄 + # 成立的 schema,母層不會是子集而被跳過 —— 欄位層必須自己走一遍。 + # 「手冊有 Order schema,信裡才說 notify_url 必填」正是這個形狀。 + for index, schema in enumerate(plan.schemas or []): + for field in schema.field_evidence or []: + yield f"schemas[{index}].field_evidence[{field.name}]", field.citations + integration = plan.integration + if integration is None: + return + for section in _INTEGRATION_SECTIONS: + for index, entry in enumerate(getattr(integration, section) or []): + yield f"integration.{section}[{index}]", entry.citations + + +def check_supplementary_support( + plan: NormalizationPlan, manifest: Manifest +) -> list[Issue]: + supplementary = { + source.relative_path + for source in manifest.local_sources + if source.authority is SourceAuthority.SUPPLEMENTARY + } + if not supplementary: + return [] + + issues: list[Issue] = [] + for location, citations in _cited_entries(plan): + named = { + citation.manifest_source + for citation in citations + if citation.manifest_source + } + if not named or not named <= supplementary: + continue + carriers = "、".join(sorted(named)) + issues.append( + Issue( + code=IssueCode.SUPPLEMENTARY_SUPPORT, + severity=Severity.WARNING, + location=location, + evidence=( + f"這條主張的唯一依據是次級佐證：{carriers}。" + "該載體無法重新取得，內容由摘錄者轉述，" + "與正式文件衝突時以正式文件為準。" + ), + suggested_fix=( + "若供應商已將此資訊寫入正式文件，改引用該文件並移除摘錄；" + "若尚未，保留現狀並在審查時確認摘錄內容。" + ), + ) + ) + return issues diff --git a/loop_apidoc/validate/models.py b/loop_apidoc/validate/models.py index 4971e556..ad803548 100644 --- a/loop_apidoc/validate/models.py +++ b/loop_apidoc/validate/models.py @@ -15,6 +15,7 @@ class IssueCode(str, Enum): FOCUS_UNMET = "FOCUS_UNMET" FOCUS_INCOMPLETE = "FOCUS_INCOMPLETE" SOURCE_FACTS_UNSCANNED = "SOURCE_FACTS_UNSCANNED" + SUPPLEMENTARY_SUPPORT = "SUPPLEMENTARY_SUPPORT" class Severity(str, Enum): diff --git a/loop_apidoc/validate/validator.py b/loop_apidoc/validate/validator.py index e4790ead..a716142c 100644 --- a/loop_apidoc/validate/validator.py +++ b/loop_apidoc/validate/validator.py @@ -8,6 +8,7 @@ from loop_apidoc.validate.consistency import check_consistency from loop_apidoc.focus.models import FocusPackage from loop_apidoc.validate.coverage import check_manifest_coverage +from loop_apidoc.validate.authority import check_supplementary_support from loop_apidoc.validate.fact_coverage import FactCoverage, check_fact_coverage from loop_apidoc.validate.focus import check_focus_outcomes from loop_apidoc.validate.integration import check_integration @@ -43,4 +44,5 @@ def validate_outputs( issues += analyze_response_contracts(result.openapi).issues issues += check_focus_outcomes(focus, error_code_floor) issues += check_fact_coverage(fact_coverage) + issues += check_supplementary_support(plan, manifest) return ValidationReport(issues=issues, root_causes=derive_root_causes(issues)) diff --git a/skills/loop-apidoc/reference/assemble-and-correction.md b/skills/loop-apidoc/reference/assemble-and-correction.md index c3e87c83..c9db2a75 100644 --- a/skills/loop-apidoc/reference/assemble-and-correction.md +++ b/skills/loop-apidoc/reference/assemble-and-correction.md @@ -97,7 +97,7 @@ the exit code. ## The `Issue` object (9 fields) ```json -{"code": "OPENAPI_INVALID|OUTPUT_MISMATCH|REQUIRED_INFO_MISSING|SOURCE_UNVERIFIED|SOURCE_CONFLICT|UNSUPPORTED_ASSERTION|FOCUS_UNMET|FOCUS_INCOMPLETE|SOURCE_FACTS_UNSCANNED", +{"code": "OPENAPI_INVALID|OUTPUT_MISMATCH|REQUIRED_INFO_MISSING|SOURCE_UNVERIFIED|SOURCE_CONFLICT|UNSUPPORTED_ASSERTION|FOCUS_UNMET|FOCUS_INCOMPLETE|SOURCE_FACTS_UNSCANNED|SUPPLEMENTARY_SUPPORT", "severity": "error|warning", "location": "str (free text)", "evidence": "str", @@ -155,6 +155,8 @@ your extraction JSON and re-running. | `FOCUS_INCOMPLETE` | error **or** warning | a `collect_error_codes` directive was answered with fewer codes than the sources tabulate. The **documented error-code floor** is the set of codes a Markdown source presents in an error-code table; reporting fewer names the ones left out, reporting more still passes (it is a floor, not an equality), and sources with no recognisable table produce no floor and no judgement. Severity comes from the directive's `kind` alone, as with `FOCUS_UNMET` | `evidence` names each omitted code with the source path and line where it is documented — open those lines, read the codes, and add them to `inventory.errors[]` with an `error_code` anchor each carrying its own evidence. Padding with invented codes fails earlier, at the gate. `target_file` is `focus-response.json`, `field_path` is `/responses/`, and `requery_scope` lists the documenting sources | | `SOURCE_FACTS_UNSCANNED` | warning (always) | the semantic completeness gate never judged that source. Either it scanned **zero endpoint facts**, or it scanned facts but **none matched** an extracted endpoint identity (`METHOD /path`). The report being clean says nothing about that source | **Do not re-read the source** — that is what `SOURCE_UNVERIFIED` asks for and it does not apply here. Open the source and decide which of three shapes it is: (a) flattened into single lines, or an unconverted PDF/Word file — re-run acquisition/preprocessing along a table-preserving path (`normalize-html-snapshot`, `preprocess`), point the manifest at the result, and re-extract; (b) structurally fine (headings, GFM tables) but its endpoints are not written as `METHOD /path` — a bare URL with the method stated elsewhere in prose, or a webhook whose path is null. The scanner will not infer the missing method (ADR 0007), so this warning is permanent for that source; say so to the user and move on, your extraction may be entirely correct; (c) prose-only with no parameter tables — legitimate, report as a known gap, never as a failure. When the issue names a line number, the cause is already known: a fence opened there and never closed (usually a close carrying an info string, e.g. ending with ```json), so everything after it went unread — fix the source and re-extract. For the zero-match shape, check whether the extraction missed the endpoints that source documents, or wrote a method/path the source does not use. `location` is the source relative path; the routing fields stay `null` on purpose, because there is no JSON field to re-fill. Never blocking (ADR 0007) | +| `SUPPLEMENTARY_SUPPORT` | warning (always) | this claim's only support is a **supplementary** carrier — an excerpt of supplier correspondence, or a spreadsheet re-saved as Markdown. The citation resolves fine; the carrier is what is weaker. It cannot be re-obtained, and its content was transcribed by a person who is named in the sidecar but whose transcription nothing can verify | **Do not re-read the source and do not re-extract** — that is what `SOURCE_UNVERIFIED` asks for, and it is exactly wrong here: an excerpt has nothing to re-read against, and the extraction is not defective. Report the claim to the requester so a human weighs it before approval. If the provider has since documented the same fact formally, cite that document instead and drop the excerpt. Never silently re-attribute the claim to a formal document that does not state it. `location` is the plan item (for example `operational[0]` or `schemas[2].field_evidence[notify_url]`), `evidence` names the carrier, and the routing fields stay `null` because there is no JSON field to re-fill (ADR 0010) | + ## Driving a correction round (default max 3 rounds; `--score` uses `--max-rounds`, default 6) 1. **Prefer the structured-routing fields.** `target_file` names where to edit diff --git a/tests/agentcli/test_assemble_supplementary_authority.py b/tests/agentcli/test_assemble_supplementary_authority.py new file mode 100644 index 00000000..4595235c --- /dev/null +++ b/tests/agentcli/test_assemble_supplementary_authority.py @@ -0,0 +1,409 @@ +"""次級佐證進入一次真實的 `assemble` 之後會發生什麼。 + +最危險的交互作用不是新功能本身,而是它對既有 run 的副作用:`sole_source()` +在 manifest 收斂成剛好一份可用文件時,讓沒有定位資訊的引用仍可歸屬,而 +`source_violations` 在同一條件下整個跳過。次級佐證若計入文件數,一份原本 +只有一份手冊的 run 加進一份信件摘錄之後,會在建立 run 目錄之前就被邊界 +拒絕 —— 補充資料反而讓正式文件失去歸屬。 +""" +from __future__ import annotations + +import hashlib +import json +from pathlib import Path + +from tests.focus.support import assemble, setup + +_NOTE = "# 補充\n\n沙箱金鑰由窗口另行以工單提供。\n" +_NOTE_SIDECAR = json.dumps( + { + "schema_version": 1, + "authority": "supplementary", + "received_from": "engineer@provider.example", + "received_at": "2026-08-16T10:00:00+08:00", + "excerpted_by": "carl", + # 讀取端把宣告綁在內容上,所以夾具也得算真的 digest。 + "imported_sha256": hashlib.sha256(_NOTE.encode("utf-8")).hexdigest(), + "source_file": "note.md", + }, + ensure_ascii=False, +) + + +def _cite_by_section(extraction: Path) -> None: + """把引用改成只指到章節,不指到檔名 —— 這正是靠單一文件歸屬的形狀。""" + inventory_path = extraction / "inventory.json" + inventory = json.loads(inventory_path.read_text(encoding="utf-8")) + for section in ("environments", "endpoints"): + for entry in inventory.get(section) or []: + entry["source"] = "第 2 節" + inventory_path.write_text( + json.dumps(inventory, ensure_ascii=False), encoding="utf-8" + ) + for endpoint_file in sorted((extraction / "endpoints").glob("*.json")): + endpoint = json.loads(endpoint_file.read_text(encoding="utf-8")) + endpoint["source"] = "第 2 節" + endpoint_file.write_text( + json.dumps(endpoint, ensure_ascii=False), encoding="utf-8" + ) + + +def test_adding_a_supplementary_note_does_not_refuse_the_run_at_the_boundary( + tmp_path: Path, +) -> None: + """加一份摘錄不得讓整個 run 在建立目錄前被拒 —— `source_guard` 問的是 + `sole_normative_source()`。但引用歸屬是另一回事:摘錄是第二份文件, + 無法解析的 locator 從此曖昧,只能是 unverified,不能被記成手冊說過。""" + sources, extraction, _ = setup( + tmp_path, + extra_sources={"note.md": _NOTE, "note.md.source.json": _NOTE_SIDECAR}, + ) + _cite_by_section(extraction) + + result = assemble(tmp_path, sources, extraction, None, "--json") + + # exit 2 是邊界拒絕(run 目錄不存在);exit 1 只是驗證 FAIL,產物仍在。 + assert result.exit_code != 2, result.stdout + payload = json.loads(result.stdout) + plan = json.loads( + (Path(payload["run_dir"]) / "plan" / "normalization-plan.json").read_text( + encoding="utf-8" + ) + ) + assert [entry["status"] for entry in plan["endpoints"]] == [ + "unverified", + "unverified", + ] + assert [ + entry["citations"][0]["manifest_source"] for entry in plan["endpoints"] + ] == [None, None] + + +def test_an_unresolvable_locator_is_never_attributed_to_the_manual( + tmp_path: Path, +) -> None: + """一條寫著「供應商信件」的引用被記成 manual.md 支持,正是這整個 + 等級區分要防的事 —— 而它會由修正本身重新引進。""" + sources, extraction, _ = setup( + tmp_path, + extra_sources={"note.md": _NOTE, "note.md.source.json": _NOTE_SIDECAR}, + ) + _cite_by_section(extraction) + + result = assemble(tmp_path, sources, extraction, None, "--json") + + payload = json.loads(result.stdout) + plan = json.loads( + (Path(payload["run_dir"]) / "plan" / "normalization-plan.json").read_text( + encoding="utf-8" + ) + ) + cited = { + citation["manifest_source"] + for entry in plan["endpoints"] + for citation in entry["citations"] + } + assert "manual.md" not in cited + + +def test_a_second_normative_document_still_ends_sole_attribution( + tmp_path: Path, +) -> None: + """對照組:排除的是次級佐證,不是「第二份文件」這件事本身。""" + sources, extraction, _ = setup( + tmp_path, extra_sources={"second.md": "# 另一份手冊\n\nGET /other\n"} + ) + _cite_by_section(extraction) + + result = assemble(tmp_path, sources, extraction, None, "--json") + + assert result.exit_code == 2, result.stdout + + +def _add_operational_citing_note(extraction: Path) -> None: + inventory_path = extraction / "inventory.json" + inventory = json.loads(inventory_path.read_text(encoding="utf-8")) + inventory["title"] = "Demo API" + inventory["operational"] = [ + { + "topic": "沙箱金鑰", + "detail": "由窗口另行以工單提供,文件未載明。", + "source": "note.md", + } + ] + inventory_path.write_text( + json.dumps(inventory, ensure_ascii=False), encoding="utf-8" + ) + + +def test_a_claim_resting_only_on_a_supplementary_source_is_named_individually( + tmp_path: Path, +) -> None: + """「這條規範性主張的唯一依據是一封信」不該變成 run 層級的背景噪音 —— + SOURCE_FACTS_UNSCANNED 的前例已經證明那會被讀者當成雜訊。""" + sources, extraction, _ = setup( + tmp_path, + extra_sources={"note.md": _NOTE, "note.md.source.json": _NOTE_SIDECAR}, + ) + _add_operational_citing_note(extraction) + + result = assemble(tmp_path, sources, extraction, None, "--json") + + assert result.exit_code != 2, result.stdout + payload = json.loads(result.stdout) + supplementary = [ + issue + for issue in payload["report"]["issues"] + if issue["code"] == "SUPPLEMENTARY_SUPPORT" + ] + assert [(issue["location"], issue["severity"]) for issue in supplementary] == [ + ("operational[0]", "warning") + ] + assert "note.md" in supplementary[0]["evidence"] + + +def test_a_claim_also_backed_by_a_normative_source_is_not_named( + tmp_path: Path, +) -> None: + """標記只出現在真正需要注意的地方。""" + sources, extraction, _ = setup( + tmp_path, + extra_sources={"note.md": _NOTE, "note.md.source.json": _NOTE_SIDECAR}, + ) + + result = assemble(tmp_path, sources, extraction, None, "--json") + + payload = json.loads(result.stdout) + assert [ + issue + for issue in payload["report"]["issues"] + if issue["code"] == "SUPPLEMENTARY_SUPPORT" + ] == [] + + +def test_freshness_fingerprint_omits_supplementary_sources(tmp_path: Path) -> None: + """`check-freshness` 的前提是來源可被重新取得並比對雜湊。一封信沒有 + URL、沒有版本,納入 watchlist 只會對它永遠給出無意義的判定。""" + from typer.testing import CliRunner + + from loop_apidoc.cli import app + + sources, extraction, _ = setup( + tmp_path, + extra_sources={"note.md": _NOTE, "note.md.source.json": _NOTE_SIDECAR}, + ) + result = assemble(tmp_path, sources, extraction, None, "--json") + assert result.exit_code != 2, result.stdout + run_dir = json.loads(result.stdout)["run_dir"] + + fingerprint_path = tmp_path / "source-fingerprint.json" + record = CliRunner().invoke( + app, + [ + "record-fingerprint", + "--run-dir", + run_dir, + "--output", + str(fingerprint_path), + ], + ) + + assert record.exit_code == 0, record.stdout + fingerprint = json.loads(fingerprint_path.read_text(encoding="utf-8")) + assert [entry["id"] for entry in fingerprint["sources"]] == ["manual.md"] + + +def _recite(extraction: Path, source: str) -> None: + inventory_path = extraction / "inventory.json" + inventory = json.loads(inventory_path.read_text(encoding="utf-8")) + for section in ("environments", "endpoints"): + for entry in inventory.get(section) or []: + entry["source"] = source + inventory_path.write_text( + json.dumps(inventory, ensure_ascii=False), encoding="utf-8" + ) + for endpoint_file in sorted((extraction / "endpoints").glob("*.json")): + endpoint = json.loads(endpoint_file.read_text(encoding="utf-8")) + endpoint["source"] = source + endpoint_file.write_text( + json.dumps(endpoint, ensure_ascii=False), encoding="utf-8" + ) + + +def test_a_run_with_nothing_re_obtainable_refuses_to_write_a_fingerprint( + tmp_path: Path, +) -> None: + """空指紋不會失敗,它會永遠回報新鮮 —— 那是無聲的監控喪失。""" + from typer.testing import CliRunner + + from loop_apidoc.cli import app + + sources, extraction, _ = setup(tmp_path) + (sources / "manual.md").unlink() + (sources / "note.md").write_text(_NOTE, encoding="utf-8") + (sources / "note.md.source.json").write_text(_NOTE_SIDECAR, encoding="utf-8") + _recite(extraction, "note.md") + result = assemble(tmp_path, sources, extraction, None, "--json") + # 條件式跳過會讓這個測試在邊界行為改變時無聲空轉,所以斷言它走到底。 + assert result.exit_code != 2, result.stdout + run_dir = json.loads(result.stdout)["run_dir"] + + record = CliRunner().invoke( + app, + [ + "record-fingerprint", + "--run-dir", + run_dir, + "--output", + str(tmp_path / "fp.json"), + ], + ) + + assert record.exit_code != 0 + assert not (tmp_path / "fp.json").exists() + + +def test_shadow_core_never_gives_a_supplementary_citation_explicit_support( + tmp_path: Path, +) -> None: + """strict/shadow 的整個意義是「每條主張都要對得上精確證據」。filename-only + 引用本來就降級成 insufficient,所以真正的暴露面是 v1 精確證據 —— 它擁有 + 自己宣告的 claim path,會把摘錄以與手冊同等的身分寫進 Core candidate。""" + from loop_apidoc.domain.evidence import fragment_digest + + sources, extraction, _ = setup( + tmp_path, + extra_sources={"note.md": _NOTE, "note.md.source.json": _NOTE_SIDECAR}, + ) + inventory_path = extraction / "inventory.json" + inventory = json.loads(inventory_path.read_text(encoding="utf-8")) + inventory["title"] = "Demo API" + inventory_path.write_text( + json.dumps(inventory, ensure_ascii=False), encoding="utf-8" + ) + endpoint_path = extraction / "endpoints" / "ep0.json" + endpoint = json.loads(endpoint_path.read_text(encoding="utf-8")) + endpoint["source"] = "note.md" + endpoint["summary"] = "沙箱金鑰由窗口另行以工單提供。" + endpoint["evidence"] = [ + { + "version": 1, + "source": "note.md", + "locator": {"kind": "line_range", "start_line": 3, "end_line": 3}, + "fragment_digest": fragment_digest("沙箱金鑰由窗口另行以工單提供。"), + "claim_path": "/summary", + } + ] + endpoint_path.write_text( + json.dumps(endpoint, ensure_ascii=False), encoding="utf-8" + ) + + result = assemble( + tmp_path, sources, extraction, None, "--json", "--architecture-mode", "shadow" + ) + + assert result.exit_code != 2, result.stdout + core = Path(json.loads(result.stdout)["run_dir"]) / "core" + assert not (core / "error.json").exists(), (core / "error.json").read_text( + encoding="utf-8" + ) + evidence = json.loads((core / "evidence.json").read_text(encoding="utf-8")) + note_artifacts = { + artifact["id"] + for artifact in evidence["artifacts"] + if any("note.md" in str(pair) for pair in artifact.get("acquisition_metadata", [])) + } + note_fragments = { + fragment["id"] + for fragment in evidence["fragments"] + if fragment.get("source_artifact_id") in note_artifacts + } + assert note_fragments, "摘錄應該仍然進入證據束 —— 它可以被引用" + + relationships = json.loads( + (core / "relationships.json").read_text(encoding="utf-8") + ) + on_note = [ + item + for item in _walk(relationships) + if isinstance(item, dict) and item.get("fragment_id") in note_fragments + ] + assert on_note == [], "摘錄不得支撐任何 Core 主張" + + # 對照組:同樣形狀的精確證據指向正式手冊時,確實會產生 explicit_support。 + # 沒有這一半,上面的斷言可能只是因為夾具根本做不出關係而空轉。 + control = _shadow_with_evidence_on(tmp_path / "control", "manual.md") + assert "explicit_support" in control + + +def _walk(value): + if isinstance(value, dict): + yield value + for item in value.values(): + yield from _walk(item) + elif isinstance(value, list): + for item in value: + yield from _walk(item) + + +def _shadow_with_evidence_on(tmp_path: Path, source_name: str) -> set[str]: + """跑一次 shadow,回傳掛在 `source_name` 片段上的關係種類。""" + from loop_apidoc.domain.evidence import fragment_digest + + summary = "沙箱金鑰由窗口另行以工單提供。" + sources, extraction, _ = setup( + tmp_path, + extra_sources={"note.md": _NOTE, "note.md.source.json": _NOTE_SIDECAR}, + ) + (sources / "manual.md").write_text( + f"# Demo API\nGET /ping\n{summary}\nPOST /notify\nSettle callback\n", + encoding="utf-8", + ) + inventory_path = extraction / "inventory.json" + inventory = json.loads(inventory_path.read_text(encoding="utf-8")) + inventory["title"] = "Demo API" + inventory_path.write_text( + json.dumps(inventory, ensure_ascii=False), encoding="utf-8" + ) + endpoint_path = extraction / "endpoints" / "ep0.json" + endpoint = json.loads(endpoint_path.read_text(encoding="utf-8")) + endpoint["source"] = source_name + endpoint["summary"] = summary + endpoint["evidence"] = [ + { + "version": 1, + "source": source_name, + "locator": {"kind": "line_range", "start_line": 3, "end_line": 3}, + "fragment_digest": fragment_digest(summary), + "claim_path": "/summary", + } + ] + endpoint_path.write_text( + json.dumps(endpoint, ensure_ascii=False), encoding="utf-8" + ) + result = assemble( + tmp_path, sources, extraction, None, "--json", "--architecture-mode", "shadow" + ) + core = Path(json.loads(result.stdout)["run_dir"]) / "core" + evidence = json.loads((core / "evidence.json").read_text(encoding="utf-8")) + artifacts = { + artifact["id"] + for artifact in evidence["artifacts"] + if any( + source_name in str(pair) + for pair in artifact.get("acquisition_metadata", []) + ) + } + fragments = { + fragment["id"] + for fragment in evidence["fragments"] + if fragment.get("source_artifact_id") in artifacts + } + relationships = json.loads( + (core / "relationships.json").read_text(encoding="utf-8") + ) + return { + item.get("relationship") + for item in _walk(relationships) + if isinstance(item, dict) and item.get("fragment_id") in fragments + } diff --git a/tests/test_cli_freshness.py b/tests/test_cli_freshness.py index c1bc4ec7..d9b9b331 100644 --- a/tests/test_cli_freshness.py +++ b/tests/test_cli_freshness.py @@ -64,7 +64,15 @@ def test_record_fingerprint_writes_and_refuses_overwrite(tmp_path: Path, monkeyp yaml.safe_dump({"openapi": "3.1.0", "info": {"version": "2.3.0"}}), encoding="utf-8") manifest = { "sources_root": "s", "generated_at": datetime.now(timezone.utc).isoformat(), - "local_sources": [], "url_sources": [], + # 至少一份可重新取得的來源:零來源的指紋會讓 check-freshness 永遠 + # 回報新鮮,現在會被拒絕,而這個測試要測的是寫檔與防覆寫。 + "local_sources": [{ + "relative_path": "manual.md", "mime_type": "text/markdown", + "source_format": "markdown", "size_bytes": 3, "sha256": "a" * 64, + "scanned_at": datetime.now(timezone.utc).isoformat(), + "supported": True, "status": "pending", + }], + "url_sources": [], } run.joinpath("manifest.json").write_text(json.dumps(manifest), encoding="utf-8") out = tmp_path / "fp.json" diff --git a/tests/test_cli_import_supplementary_note.py b/tests/test_cli_import_supplementary_note.py new file mode 100644 index 00000000..354eb339 --- /dev/null +++ b/tests/test_cli_import_supplementary_note.py @@ -0,0 +1,260 @@ +from __future__ import annotations + +import hashlib +import json +from pathlib import Path + +from typer.testing import CliRunner + +from loop_apidoc.cli import app + + +runner = CliRunner() + + +def test_import_supplementary_note_writes_source_and_provenance( + tmp_path: Path, +) -> None: + """摘錄本身無法驗證,所以出處必須與內容綁在同一個檔案雜湊上 —— + 記下摘錄者是為了可追責,這是這條路徑與其他所有來源的本質差異。""" + excerpt = tmp_path / "excerpt.md" + excerpt.write_text( + "# 測試環境金鑰\n\n沙箱金鑰由窗口另行以工單提供,不在文件中。\n", + encoding="utf-8", + ) + sources = tmp_path / "sources" + + result = runner.invoke( + app, + [ + "import-supplementary-note", + "--input", + str(excerpt), + "--from", + "engineer@provider.example", + "--received-at", + "2026-08-16T10:00:00+08:00", + "--subject", + "測試環境金鑰取得方式", + "--excerpted-by", + "carl", + "--sources", + str(sources), + ], + ) + + assert result.exit_code == 0, result.stdout + written = sources / "excerpt.md" + assert written.read_bytes() == excerpt.read_bytes() + digest = hashlib.sha256(excerpt.read_bytes()).hexdigest() + provenance = json.loads( + (sources / "excerpt.md.source.json").read_text(encoding="utf-8") + ) + assert provenance == { + "schema_version": 1, + "authority": "supplementary", + "received_from": "engineer@provider.example", + "received_at": "2026-08-16T10:00:00+08:00", + "subject": "測試環境金鑰取得方式", + "excerpted_by": "carl", + "imported_sha256": digest, + "source_file": "excerpt.md", + } + + +def _import_note(tmp_path: Path, sources: Path, *, name: str = "excerpt.md") -> None: + excerpt = tmp_path / f"in-{name}" + excerpt.write_text("# 補充\n\n沙箱金鑰由窗口另行提供。\n", encoding="utf-8") + result = runner.invoke( + app, + [ + "import-supplementary-note", + "--input", + str(excerpt), + "--from", + "engineer@provider.example", + "--received-at", + "2026-08-16T10:00:00+08:00", + "--excerpted-by", + "carl", + "--sources", + str(sources), + "--filename", + name, + ], + ) + assert result.exit_code == 0, result.stdout + + +def test_manifest_marks_supplementary_sources_and_defaults_to_normative( + tmp_path: Path, +) -> None: + """等級與出處是同一件事的兩面 —— 一份東西之所以是次級,正是因為它的 + 出處是一封信。寫在同一個檔案裡,兩者就不會各說各話。""" + sources = tmp_path / "sources" + sources.mkdir() + (sources / "manual.md").write_text("# API\n\nGET /ping\n", encoding="utf-8") + _import_note(tmp_path, sources) + + manifest_path = tmp_path / "manifest.json" + result = runner.invoke( + app, + ["manifest", "--sources", str(sources), "--output", str(manifest_path)], + ) + + assert result.exit_code == 0, result.stdout + manifest = json.loads(manifest_path.read_text(encoding="utf-8")) + authority = { + source["relative_path"]: source["authority"] + for source in manifest["local_sources"] + if source["status"] == "pending" + } + assert authority == {"excerpt.md": "supplementary", "manual.md": "normative"} + + +def _invoke(tmp_path: Path, sources: Path, **overrides) -> object: + excerpt = overrides.pop("input_file", None) + if excerpt is None: + excerpt = tmp_path / "excerpt.md" + if not excerpt.exists(): + excerpt.write_text("# 補充\n\n內容。\n", encoding="utf-8") + args = { + "--input": str(excerpt), + "--from": "engineer@provider.example", + "--received-at": "2026-08-16T10:00:00+08:00", + "--excerpted-by": "carl", + "--sources": str(sources), + } + args.update(overrides) + flat: list[str] = ["import-supplementary-note"] + for key, value in args.items(): + flat += [key, value] + return runner.invoke(app, flat) + + +def test_import_supplementary_note_requires_a_timezone_aware_timestamp( + tmp_path: Path, +) -> None: + """沒有時區的時間戳無法排序,出處紀錄就失去了它唯一的用途。""" + result = _invoke( + tmp_path, tmp_path / "sources", **{"--received-at": "2026-08-16T10:00:00"} + ) + + assert result.exit_code == 2 + assert "timezone" in result.stderr + assert not (tmp_path / "sources").exists() + + +def test_import_supplementary_note_refuses_to_overwrite_existing_evidence( + tmp_path: Path, +) -> None: + sources = tmp_path / "sources" + sources.mkdir() + (sources / "excerpt.md").write_text("先前的證據\n", encoding="utf-8") + + result = _invoke(tmp_path, sources) + + assert result.exit_code == 2 + assert "already exists" in result.stderr + assert (sources / "excerpt.md").read_text(encoding="utf-8") == "先前的證據\n" + + +def test_import_supplementary_note_rejects_a_path_as_filename(tmp_path: Path) -> None: + result = _invoke(tmp_path, tmp_path / "sources", **{"--filename": "nested/a.md"}) + + assert result.exit_code == 2 + assert "single file name" in result.stderr + + +def test_import_supplementary_note_rejects_non_markdown(tmp_path: Path) -> None: + """摘錄是人寫的散文與表格,Markdown 是唯一能被 source_facts 掃描的形狀。""" + excerpt = tmp_path / "excerpt.pdf" + excerpt.write_bytes(b"%PDF-1.4\n") + + result = _invoke(tmp_path, tmp_path / "sources", input_file=excerpt) + + assert result.exit_code == 2 + assert "Markdown" in result.stderr + + +def test_import_supplementary_note_requires_an_excerpter(tmp_path: Path) -> None: + """摘錄無法驗證,只能可追責 —— 沒有署名就連追責都做不到。""" + result = _invoke(tmp_path, tmp_path / "sources", **{"--excerpted-by": " "}) + + assert result.exit_code == 2 + assert "excerpted_by" in result.stderr + + +def test_manifest_refuses_a_sidecar_it_cannot_read(tmp_path: Path) -> None: + """缺席才是 normative,讀不動不是。一個截斷的寫入若被當成正式文件, + 整個功能就被一個壞檔案靜默關掉,而操作者看不到任何差別。""" + sources = tmp_path / "sources" + sources.mkdir() + _import_note(tmp_path, sources) + (sources / "excerpt.md.source.json").write_text("{broken", encoding="utf-8") + + result = runner.invoke( + app, + ["manifest", "--sources", str(sources), "--output", str(tmp_path / "m.json")], + ) + + assert result.exit_code == 2 + assert "sidecar" in result.stderr + assert not (tmp_path / "m.json").exists() + + +def test_manifest_refuses_a_sidecar_that_does_not_match_its_source( + tmp_path: Path, +) -> None: + """宣告要綁在內容上 —— 否則一個從別處複製來的兩行 sidecar 就能 + 把一份正式手冊降級,而降級後的手冊會整份退出新鮮度指紋。""" + sources = tmp_path / "sources" + sources.mkdir() + (sources / "manual.md").write_text("# API\n\nGET /ping\n", encoding="utf-8") + (sources / "manual.md.source.json").write_text( + json.dumps({"authority": "supplementary"}), encoding="utf-8" + ) + + result = runner.invoke( + app, + ["manifest", "--sources", str(sources), "--output", str(tmp_path / "m.json")], + ) + + assert result.exit_code == 2 + assert "sidecar" in result.stderr + + +def test_manifest_accepts_a_rendered_url_sidecar_as_normative(tmp_path: Path) -> None: + """`import-rendered-url` 寫的 provenance 沒有 authority 欄位 —— 那是 + 一份已驗證出處的正式文件,不是判定失敗。""" + sources = tmp_path / "sources" + sources.mkdir() + (sources / "page.md").write_text("# API\n\nGET /ping\n", encoding="utf-8") + (sources / "page.md.source.json").write_text( + json.dumps( + { + "schema_version": 1, + "original_url": "https://docs.example.com/a", + "canonical_url": "https://docs.example.com/a", + "captured_at": "2026-08-16T10:00:00+08:00", + "capture_method": "browser_save", + "imported_sha256": "irrelevant", + "source_file": "page.md", + } + ), + encoding="utf-8", + ) + manifest_path = tmp_path / "m.json" + + result = runner.invoke( + app, + ["manifest", "--sources", str(sources), "--output", str(manifest_path)], + ) + + assert result.exit_code == 0, result.stdout + manifest = json.loads(manifest_path.read_text(encoding="utf-8")) + assert [ + source["authority"] + for source in manifest["local_sources"] + if source["status"] == "pending" + ] == ["normative"] diff --git a/tests/validate/test_models.py b/tests/validate/test_models.py index d70c2ec2..4bc21458 100644 --- a/tests/validate/test_models.py +++ b/tests/validate/test_models.py @@ -33,6 +33,10 @@ def test_issue_code_values_match_spec(): # 與 SOURCE_UNVERIFIED 分開:那個 code 的補救是重讀來源填 JSON, # 這個的補救是換前處理路徑,混用會誘導 agent 做無效的重讀。 "SOURCE_FACTS_UNSCANNED", + # 與 SOURCE_UNVERIFIED 分開:那個 code 的意思是引用對不上 manifest + # 來源,補救是重讀來源補引用;這個的引用完全對得上,問題在載體等級, + # 而摘錄沒有東西可以重讀。 + "SUPPLEMENTARY_SUPPORT", } From 258dce4b950611a162bd7fffecbea3529a750485 Mon Sep 17 00:00:00 2001 From: carl Date: Mon, 17 Aug 2026 10:18:03 +0800 Subject: [PATCH 2/3] =?UTF-8?q?docs:=20[=20manuals=20]=20=E8=A3=9C?= =?UTF-8?q?=E4=B8=8A=20source-risk=20=E6=96=B0=E8=A6=8F=E5=89=87=E8=88=87?= =?UTF-8?q?=E6=AC=A1=E7=B4=9A=E4=BD=90=E8=AD=89=E4=BE=86=E6=BA=90?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `scripts/release.py prepare` 不碰 HTML 手冊,而 #104 改了 source-risk 的 規則集、#105 新增了一個取源指令與一個來源等級 —— 依 AGENTS.md 的非商榷 規則,這些必須在同一次發布裡改完。 operator manual（zh + en）: - source-risk 段落改寫成兩個方向(操縱 vs 洩漏),補上五條新規則、只有 自證性材料會擋的理由、Luhn 與測試卡號排除、以及 warning 的 500 筆 獨立額度與它存在的原因 - 新增 `import-supplementary-note` 一節與目錄項,含接受的破口與 sidecar 的 fail-closed 行為 architecture manual（zh + en）: - source_risk/ 模組描述補上洩漏方向與 warning 額度 - manifest/ 模組描述補上 authority 欄位、sidecar 標定與 fail-closed - 流程圖加上 import-supplementary-note 🤖 Generated with Claude Code --- docs/architecture-manual.en.html | 7 ++++--- docs/architecture-manual.html | 7 ++++--- docs/operator-manual.en.html | 13 ++++++++++++- docs/operator-manual.html | 13 ++++++++++++- 4 files changed, 32 insertions(+), 8 deletions(-) diff --git a/docs/architecture-manual.en.html b/docs/architecture-manual.en.html index 04ca9157..a2676410 100644 --- a/docs/architecture-manual.en.html +++ b/docs/architecture-manual.en.html @@ -744,6 +744,7 @@

Package Boundaries & Module Architecture

scaffold-extraction
normalize-html-snapshot
import-rendered-url
+
import-supplementary-note
related-url-pages
inspect-source-risk
assess-sources
@@ -779,7 +780,7 @@

Package Boundaries & Module Architecture

diff --git a/docs/introduction.html b/docs/introduction.html index 9ca312e2..e8286b6c 100644 --- a/docs/introduction.html +++ b/docs/introduction.html @@ -1557,7 +1557,7 @@

該有的整理好,沒有的老實說,絕不瞎掰。< 想深入技術細節?看給工程師的完整導覽 -

兩段式架構、seam JSON、CLI 28 個命令，以及風險、驗證與修正閘，一頁看懂整條 pipeline。

+

兩段式架構、seam JSON、完整的 CLI 命令表，以及風險、驗證與修正閘，一頁看懂整條 pipeline。

diff --git a/docs/onboarding.en.html b/docs/onboarding.en.html index 61e1d32b..84bc3059 100644 --- a/docs/onboarding.en.html +++ b/docs/onboarding.en.html @@ -615,6 +615,7 @@

What to take away

Supplier sources are the sole authority for what the provider documents.

Anything a source does not state is left as null and recorded in missing — never inferred, never filled in with REST/OAuth conventions.

When validation hits something missing, it fails loudly rather than using guesses to make the docs "look complete." Every design decision in this project exists to hold this line.

+

Nor does a supplementary source weaken it. A hand-written excerpt of supplier correspondence may fill missing, but it is not a formal document: the formal document wins any disagreement, the excerpt never reaches explicit_support, and every claim resting only on it is named individually (ADR 0010).

Passive Implementation Observations do not weaken this rule. They report empirical behavior only inside an exact Applicability Envelope and use a separate confirms / contradicts / inconclusive / out_of_scope axis; none of those outcomes becomes supplier-source support.

The converse also has to be evidenced: "the source does not state it" is a claim that must be earned. source_facts/ mechanically scans Markdown sources for parameter tables and example blocks, and blocks the run when a source plainly documents them but the extraction came back empty; placeholder answers such as "requires further extraction" are blocked the same way. To claim a field genuinely is not in the source, name it in missing — so this gate only ever forces a named gap, never a fabrication.

Be clear about its limits, though: the scan only recognises well-structured Markdown (headings, GFM tables, fenced blocks). If a source is flattened into long single lines it yields zero facts and the gate does nothing. A clean gate exit does not mean the extraction is complete — it catches omissions that can be mechanically proven; everything else still needs human review.

@@ -777,7 +778,7 @@

What stages a full run passes through

2inspect-source-risk - Scan manifest-bound text without rewriting it or echoing payloads; blocker or unscannable input rejects before agent access. + Scan manifest-bound text in both directions — can it manipulate the agent, can it leak material to the agent — without rewriting it or echoing payloads; blocker or unscannable input rejects before agent access. source-risk-report.{json,zh-TW.md}
@@ -873,6 +874,10 @@

CLI commands, each with its own job

import-rendered-url

Import protected browser-rendered HTML/Markdown as immutable local evidence plus versioned provenance and fetched_rendered coverage. A verified match lets manifest/assemble skip that origin probe; mismatches fail closed.

+
+ import-supplementary-note +

Import a hand-written excerpt of supplier correspondence as a supplementary source, with a provenance sidecar bound to its content (origin, receipt time, excerpter, SHA-256). It may fill missing but never supports explicit_support, and it stays out of the freshness fingerprint — the dividing line is re-obtainability. The excerpt is written by a person, so the sidecar buys accountability, not verifiability (ADR 0010).

+
related-url-pages

Output candidate page cards from body links and shared entities; the model reads candidate page bodies only when needed.

@@ -1190,8 +1195,9 @@

How the code is split into packages

- + + @@ -1205,8 +1211,8 @@

How the code is split into packages

- - + + @@ -1217,7 +1223,7 @@

How the code is split into packages

PackageResponsibility
manifest/Scan local sources + build manifest.json
manifest/Scan local sources + build manifest.json; each source's authority (normative by default, or supplementary) is declared by its same-named .source.json sidecar, and the read side fails closed
rendered_url.pyOffline rendered-source import plus read-side provenance verification; writes immutable source/sidecar/coverage and prevents a matched protected-origin probe
supplementary_note.pyImports a hand-written excerpt of supplier correspondence as a supplementary source; writes the immutable source plus a provenance sidecar bound to its content
agentcli/assemble.py (assemble JSON → plan→preparation→generate→validate), input_schema.py (pydantic validation at the boundary), evidence.py (read-side v1 exact-evidence materialization/digest and claim-path verification), gate.py/verify.py (the extraction-contract check for verify-extraction, including the source-fact and deferral gates from source_facts/, writes nothing), extraction.py (inventory → plan answers), preprocess.py (typed PDF/DOCX→markdown orchestration)
docx_*.pydocx_normalization.py is the stable facade; package-wide fail-closed OOXML validation/rendering stays pure, while docx_publish.py owns staged provenance publication with rollback on reported write failures
shadow/Opt-in legacy/Core bridge and in-memory runner; only shadow/report.py writes the observational core/ artifact set or a safe error report.
run/run-id generation, result/status models, writing the plan into the run-dir (writes files)
score/Read a completed run-dir and produce score/score.{json,md} with five weighted categories (report.py writes files)
diff/loader.py / compare.py / models.py / report.py (report.py writes files)
freshness/A cheap scheduled freshness gate: record.py writes a baseline fingerprint from a completed run-dir (sha256 for local sources, one version signal per URL source; refuses to overwrite without --force), signals.py recomputes and classifies the current signal (network reads, writes nothing), check.py aggregates one docset, batch.py fans the gate across a watchlist (a per-item error is captured in that item's result rather than aborting the batch), report.py writes the reports. The exit codes are the contract: 0 unchanged, 1 changed, 2 inconclusive
source_risk/Manifest-bound pre-agent text inspection and verified audit loading; reads exact source bytes and writes source-risk-report.{json,zh-TW.md} without rewriting or echoing source payloads
freshness/A cheap scheduled freshness gate: record.py writes a baseline fingerprint from a completed run-dir (sha256 for local sources, one version signal per URL source, supplementary carriers always excluded — they cannot be re-obtained, so including one would only report fresh forever; a run with nothing re-obtainable is refused outright, and overwriting still needs --force), signals.py recomputes and classifies the current signal (network reads, writes nothing), check.py aggregates one docset, batch.py fans the gate across a watchlist (a per-item error is captured in that item's result rather than aborting the batch), report.py writes the reports. The exit codes are the contract: 0 unchanged, 1 changed, 2 inconclusive
source_risk/Manifest-bound pre-agent text inspection and verified audit loading, covering both directions — whether a source can manipulate the agent and whether it leaks material to it. Among the leak rules only self-evidencing material (PEM private-key blocks, JWTs) blocks; a credential reference stays a warning, because a competent document documents its own Authorization header. reads exact source bytes and writes source-risk-report.{json,zh-TW.md} without rewriting or echoing source payloads
source_quality/Pre-extraction source quality assessment and source version diff report; requires and embeds a verified source-risk audit; rejected reports may expose explicit bounded required_source_refs without crawling (writes files)
focus/Requester-authored extraction focus directives: models.py (the strict contract; kind determines severity, intent determines anchor type, and there are only two outcomes, satisfied / not_found), loader.py (reads focus.json and focus-response.json — this package's only file-reading exit), gate.py/fields.py/codes.py (pure checks: one answer per directive, anchor resolution, and a not_found that must account for every readable source), report.py (writes <run-dir>/focus/, the only file-writing exit). A falsified assertion travels through the ValidationReport as FOCUS_UNMET rather than through the input gate — that is what keeps the artifacts available for a human to judge. A collect_error_codes answer reporting fewer codes than the documented floor computed from the source tables travels the same route, becoming a FOCUS_INCOMPLETE that names the omitted codes and where they are documented
source_facts/Source-fact inventory: markdown.py mechanically scans Markdown sources for endpoint declarations, parameter tables and example blocks; collect.py reads the manifest-named sources (this package's only file-reading exit); gate.py compares them against the extraction JSON (resolving schema_ref into inventory.schemas transitively, so a shared body type is deduplication rather than an omission) and deferral.py rejects placeholder answers
-

Marked rows contain file I/O exits: generate/, run/, agentcli/preprocess.py, shadow/report.py, feedback/report.py, preparation/report.py, score/report.py, diff/report.py, source_risk/report.py, source_quality/, freshness/, docx_publish.py, rendered URL import, the URL corpus cache, foundry/, and review/workflow.py writing through Foundry. feedback/loader.py, docx_normalization.py, source_risk/inspect.py, and adapters/fragments.py are read-side I/O exits; Core and Domain remain pure.

+

Marked rows contain file I/O exits: generate/, run/, agentcli/preprocess.py, shadow/report.py, feedback/report.py, preparation/report.py, score/report.py, diff/report.py, source_risk/report.py, source_quality/, freshness/, docx_publish.py, rendered URL import, supplementary_note.py, the URL corpus cache, foundry/, and review/workflow.py writing through Foundry. feedback/loader.py, docx_normalization.py, source_risk/inspect.py, and adapters/fragments.py are read-side I/O exits; Core and Domain remain pure.

diff --git a/docs/onboarding.html b/docs/onboarding.html index e2e7aa3e..bf2e12cb 100644 --- a/docs/onboarding.html +++ b/docs/onboarding.html @@ -615,6 +615,7 @@

讀完要掌握

供應商來源是「供應商文件明載內容」的唯一權威。

來源沒寫的東西，一律留 null 並記錄到 missing —— 絕不推論、絕不用 REST/OAuth 慣例補上。

驗證遇到缺漏會大聲失敗（fail loudly），而不是用猜測讓文件「看起來完整」。整個專案的每一個設計決策，都是為了守住這條線。

+

次級佐證也不會削弱它。供應商信件的人工摘錄可以填 missing,但它不是正式文件:與正式文件衝突時正式文件勝,它永遠到不了 explicit_support,而每一條只靠它成立的主張都會被逐條點名(ADR 0010)。

被動 Implementation Observation 不會削弱這條規則。它只回報精確 Applicability Envelope 內的經驗行為，使用獨立的 confirms／contradicts／inconclusive／out_of_scope 軸；任何結果都不會變成供應商來源支持。

反過來說，「來源沒寫」也必須拿得出證據：source_facts/ 會機械掃描 Markdown 來源的參數表與範例區塊，來源明明寫了、擷取卻空著就擋下；「之後再擷取」「需進一步擷取」這類佔位答案同樣擋下。要主張某欄位來源真的沒寫，就在 missing 裡具名——所以這道閘只逼出具名缺口，不會逼出捏造。

但要清楚它的界線：掃描只認得結構良好的 Markdown（標題、GFM 表格、圍籬區塊）。來源若被壓成一行行超長文字，掃出零筆事實，這道閘等於沒作用。閘門乾淨通過 ≠ 擷取完整——它擋掉的是「機械證明得了的遺漏」，其餘仍要人工核對。

@@ -777,7 +778,7 @@

一次完整執行，經過哪些階段

2inspect-source-risk - 掃描 manifest 綁定文字，不改寫來源、不回顯 payload；blocker 或 unscannable 輸入會在 agent 接觸前拒絕。 + 掃描 manifest 綁定文字的兩個方向 —— 會不會操縱 agent、會不會洩漏東西給 agent；不改寫來源、不回顯 payload；blocker 或 unscannable 輸入會在 agent 接觸前拒絕。 source-risk-report.{json,zh-TW.md}
@@ -873,6 +874,10 @@

CLI 指令，各司其職

import-rendered-url

把受保護頁面的 browser-rendered HTML/Markdown 匯入為 immutable 本機證據、版本化 provenance 與 fetched_rendered coverage。驗證匹配後 manifest/assemble 省略該 origin probe；mismatch fail closed。

+
+ import-supplementary-note +

把供應商信件／通訊軟體補充說明的人工摘錄匯入為次級佐證,寫出綁定內容的 provenance sidecar(出處、收到時間、摘錄者、SHA-256)。它填得了 missing,支撐不了 explicit_support,也不進新鮮度指紋 —— 分界線是可重新取得性。摘錄由人所寫,sidecar 買到的是可追責,不是可驗證(ADR 0010)。

+
related-url-pages

依正文連結與共享實體輸出候選頁卡片,模型只在需要時讀取候選頁正文。

@@ -1190,8 +1195,9 @@

程式碼怎麼分包

- + + @@ -1205,8 +1211,8 @@

程式碼怎麼分包

- - + + @@ -1217,7 +1223,7 @@

程式碼怎麼分包

套件職責
manifest/掃描本地來源 + 建立 manifest.json
manifest/掃描本地來源 + 建立 manifest.json；每份來源的 authority(normative 預設／supplementary)由同名 .source.json sidecar 標定,讀取端 fail closed
rendered_url.py離線 rendered-source import 與只讀 provenance 驗證；寫入 immutable source/sidecar/coverage，並防止匹配的 protected origin probe
supplementary_note.py把供應商補充說明的人工摘錄匯入為次級佐證；寫出 immutable source 與綁定內容的 provenance sidecar
agentcli/assemble.py(組裝 JSON → plan→preparation→generate→validate)、input_schema.py(交界處的 pydantic 驗證)、evidence.py(v1 exact evidence 的 read-side materialization/digest 與 claim-path 驗證)、gate.py/verify.py(verify-extraction 的擷取契約檢查,含 source_facts/ 的來源事實與延後語句閘,不寫檔)、extraction.py(inventory → plan 答案)、preprocess.py(typed PDF／DOCX→markdown 編排)
docx_*.pydocx_normalization.py 是穩定 facade；package-wide fail-closed OOXML 驗證／rendering 維持 pure，docx_publish.py 負責分段暫存且於可回報寫入失敗時回滾的 provenance publication
shadow/選用的 legacy/Core bridge 與 in-memory runner；只有 shadow/report.py 會寫出觀測性的 core/ 產物集或安全錯誤報告。
run/run-id 產生、result/status models、把計畫寫入 run-dir(寫檔)
score/讀取完成 run-dir,以五類權重產生 score/score.{json,md}(report.py 寫檔)
diff/loader.py / compare.py / models.py / report.py(report.py 寫檔)
freshness/排程用的便宜新鮮度閘:record.py 從已完成 run-dir 寫出基準 fingerprint(本地取 sha256、每個 URL 取一次版本訊號,無 --force 拒絕覆寫)、signals.py 重算當下訊號並分類(網路讀取但不寫檔)、check.py 彙總單一 docset、batch.py 依 watchlist 扇出多個 docset(單項錯誤收進該項結果不中斷整批)、report.py 輸出報告。退出碼即契約:0 未變動、1 已變動、2 無法判定
source_risk/以 manifest 綁定、在 agent 讀取前執行的文字風險掃描與 audit 驗證；讀取精確來源 bytes，寫出 source-risk-report.{json,zh-TW.md}，不改寫或回顯來源 payload
freshness/排程用的便宜新鮮度閘:record.py 從已完成 run-dir 寫出基準 fingerprint(本地取 sha256、每個 URL 取一次版本訊號,次級佐證一律排除 —— 它不可重新取得,納入只會永遠回報新鮮;零可重取得來源則直接拒絕寫出,無 --force 亦拒絕覆寫)、signals.py 重算當下訊號並分類(網路讀取但不寫檔)、check.py 彙總單一 docset、batch.py 依 watchlist 扇出多個 docset(單項錯誤收進該項結果不中斷整批)、report.py 輸出報告。退出碼即契約:0 未變動、1 已變動、2 無法判定
source_risk/以 manifest 綁定、在 agent 讀取前執行的文字風險掃描與 audit 驗證,涵蓋兩個方向:來源會不會操縱 agent,以及會不會洩漏東西給 agent。洩漏類只有自證性材料(PEM 私鑰區塊、JWT)會擋,憑證引用只給 warning —— 合格文件本來就會示範 Authorization 標頭。讀取精確來源 bytes，寫出 source-risk-report.{json,zh-TW.md}，不改寫或回顯來源 payload
source_quality/擷取前來源品質評估與來源版本差異報告；要求並嵌入已驗證的 source-risk audit；reject 可輸出明確 bounded required_source_refs 且不 crawl(寫檔)
focus/提出者的擷取重點指令:models.py(嚴格契約;kind 決定 severity、intent 決定錨點型別,只有 satisfied／not_found 兩種結局)、loader.py(讀 focus.json 與 focus-response.json,本套件唯一讀檔出口)、gate.py／fields.py／codes.py(純檢查:應答對應、錨點解析、not_found 必須交代每一份可讀來源)、report.py(寫出 <run-dir>/focus/,唯一寫檔出口)。落空的斷言走 ValidationReport 的 FOCUS_UNMET,不走輸入閘 —— 那樣產物才留得下來給人判斷。collect_error_codes 答得比來源表格算出的記載錯誤碼下界少,同樣走 ValidationReport,成為具名列出漏碼與其出處的 FOCUS_INCOMPLETE
source_facts/來源事實盤點:markdown.py 機械掃描 Markdown 來源的端點宣告、參數表與範例區塊,collect.py 讀取 manifest 指名的來源(本套件唯一讀檔出口),gate.py 比對擷取 JSON(並沿 schema_ref 遞迴解析進 inventory.schemas,讓共用型別算去重複而非遺漏)、deferral.py 攔截佔位式延後答案
-

標記列含檔案 I/O 出口:generate/、run/、agentcli/preprocess.py、shadow/report.py、feedback/report.py、preparation/report.py、score/report.py、diff/report.py、source_risk/report.py、source_quality/、freshness/、docx_publish.py、rendered URL import、URL corpus 快取、foundry/ 與經 Foundry 寫入的 review/workflow.py 會寫檔。feedback/loader.py、docx_normalization.py、source_risk/inspect.py 與 adapters/fragments.py 是 read-side I/O exit；Core 與 Domain 維持 pure。

+

標記列含檔案 I/O 出口:generate/、run/、agentcli/preprocess.py、shadow/report.py、feedback/report.py、preparation/report.py、score/report.py、diff/report.py、source_risk/report.py、source_quality/、freshness/、docx_publish.py、rendered URL import、supplementary_note.py、URL corpus 快取、foundry/ 與經 Foundry 寫入的 review/workflow.py 會寫檔。feedback/loader.py、docx_normalization.py、source_risk/inspect.py 與 adapters/fragments.py 是 read-side I/O exit；Core 與 Domain 維持 pure。