Fix #47 review findings: custom XML decoding, decision record, dashboard denominators

architecture.md

@@ -77,7 +77,11 @@ Most-touched top-level directories:
   - `harness/compat_cases.py`: openpyxl-style compatibility snippet lane
   - `harness/artifact_context.py`: chart and macro package evidence lanes
   - `harness/mutation.py`: template-mutation lane (subprocess wall time, RSS,
-    package-preservation scoring)
+    content-model preservation scoring; see DEC-027)
+  - `harness/content_model_xlsx.py`: xlsx content model used to score mutation
+    preservation
+  - `harness/adapters/json_model_adapter.py`: base for adapters driven through
+    an external JSON helper (SheetJS, ExcelJS, LibreOffice)
   - `harness/calc.py`: formula-recalculation tier engines (wolfxl, LibreOffice,
     aspose-cells-foss, zavora oracle) against the cache-free fixture
   - `modifiable.py`: engine registry for mutation-capable adapters
@@ -95,14 +99,18 @@ Most-touched top-level directories:
   - `calc/`: cache-free formula fixture + LibreOffice oracle expected values
 
 - `tools/external-oracles/`
-  - Optional subprocess helpers for non-Python oracle tools.
+  - Optional subprocess helpers for non-Python oracle tools and for the
+    JSON-model adapters (`sheetjs`, `exceljs`, `libreoffice`).
   - `excelize/`: Go helper that generates and inspects `.xlsx` fixtures with
     Excelize. Run from that directory with `go run .`.
   - `libreoffice/`: Python helper that runs LibreOffice headless open/save and
-    PDF render validation.
+    PDF render validation, plus the UNO macro behind the `libreoffice` adapter.
   - `exceljs/`: Node helper that generates workbooks with ExcelJS tables,
     formulas, data validations, rich strings, comments, hyperlinks, images,
-    merges, freeze panes, and sheet protection.
+    merges, freeze panes, and sheet protection, and serves the `exceljs`
+    adapter's model operations.
+  - `sheetjs/`: Node helper behind the `sheetjs` adapter (SheetJS Community
+    Edition).
   - `apache-poi/`: Java helper that generates workbooks with Apache POI tables,
     formulas, data validations, rich strings, comments, hyperlinks, images,
     merges, freeze panes, and sheet protection.
@@ -143,7 +151,8 @@ Common starting points by intent:
 - Add a new external oracle:
   - `src/excelbench/harness/external_oracles.py`
   - `docs/trackers/external-oracle-expansion.md`
-  - Keep helpers optional and subprocess-isolated until promoted into normal benchmark flows.
+  - Keep helpers optional and subprocess-isolated. A helper that implements the
+    JSON-model protocol is registered as a normal adapter (DEC-027).
 
 - Add a new scored feature:
   - Generator: `src/excelbench/generator/features/`
@@ -200,9 +209,12 @@ JSON fixture manifest
   -> optional WolfXL validation script checks read + in-place modify-save preservation
 ```
 
-External oracle helpers are intentionally not part of `get_all_adapters()` yet.
-Missing Go/Java/.NET/LibreOffice commands should skip cleanly, not fail the core
-suite. The first local pack is generated by
+External helpers that implement the JSON-model protocol (`sheetjs`, `exceljs`,
+`libreoffice`) are registered in `get_all_adapters()` and scored like any other
+adapter; `apache-poi`, `excelize` and `zavora-xlsx` are registered the same way
+and run in the cross-language lane (DEC-027). Registration is availability-gated:
+a missing Node/Go/Java/.NET/LibreOffice runtime or helper dependency removes the
+adapter instead of failing the core suite. The first local oracle pack is generated by
 `scripts/generate_external_oracle_fixtures.py` into `results_dev_external/` and
 validated against WolfXL with `scripts/validate_external_oracle_fixtures_with_wolfxl.py`.
 

decisions.md

@@ -40,6 +40,57 @@ Skip logging for routine bug fixes, refactors, or incremental test additions.
 
 ## Decisions
 
+### DEC-027 — External-helper adapters are scored; mutation preservation compares parsed content (2026-10-02)
+
+**Context**: DEC-021 kept external helpers out of `get_all_adapters()` until a
+later decision promoted them. The 2026-10-02 snapshot adds SheetJS, ExcelJS
+and LibreOffice Calc, which have no Python API: each is driven by a helper in
+`tools/external-oracles/` through the JSON-model protocol in
+`harness/adapters/json_model_adapter.py`. Separately, the DEC-025 mutation
+score counted retained package parts and byte-identical custom XML, so a byte
+copy scored 100 by construction and an engine that rewrote the package
+equivalently (new part names, renumbered relationship ids, other XML
+serialization) lost points without losing any content.
+
+**Decision**:
+
+- `sheetjs`, `exceljs` and `libreoffice` are registered in
+  `get_all_adapters()` and scored like any other adapter in the fidelity and
+  mutation lanes, so the comparison covers non-Python libraries. Registration
+  is availability-gated: a missing Node or `soffice` runtime removes the
+  adapter instead of failing the run. Helpers report only what the library's
+  own API exposes, and the Python side never fills in values the helper did
+  not report. The same availability-gated registration already applied to
+  `apache-poi`, `excelize` and `zavora-xlsx`, which stay in the
+  cross-language lane (`CROSS_LANGUAGE_ADAPTER_NAMES`).
+- Mutation preservation compares parsed content: the template's xlsx content
+  model (`harness/content_model_xlsx.py`) with the declared edits applied is
+  diffed against the output's model, plus a `custom_xml` feature (canonical
+  XML, datastore item id, schema references). Part names, relationship ids,
+  XML serialization and encoding, and document metadata are not content, so
+  an equivalent rewrite scores the same as an in-place edit. Cached results of
+  formula cells are recalculation state and are not compared; formula text is.
+  The result is features preserved (`kept/present`), edits applied, and an
+  unscored integrity report (dangling relationships, parts without a content
+  type). This supersedes the DEC-025 package-preservation score.
+
+**Alternatives considered**:
+
+1. **Keep JSON-model helpers in a separate context lane** — rejected. The
+   fidelity matrix would stay Python-only while the libraries users compare
+   against WolfXL and openpyxl include JavaScript and LibreOffice.
+2. **Keep the part-retention score and whitelist known renames** — rejected.
+   Every new engine would need its own whitelist, and a byte copy would still
+   score perfectly without proving the edits landed.
+
+**Consequences**:
+
+- Fidelity and mutation results depend on which helper runtimes are
+  installed; snapshot READMEs record runtime versions and lane membership.
+- Mutation scores before 2026-10-02 are not comparable with later scores.
+- Package-level differences remain visible as unscored diagnostics (lost and
+  added parts, element changes) so layout regressions are still reviewable.
+
 ### DEC-023 — Tier-4 fidelity features via openpyxl-structural fixtures (2026-09-08)
 
 **Context**: The scored matrix stopped at 19 features. Sheet protection, page
@@ -137,6 +188,8 @@ cells, save, keep everything else comparable.
 - Preservation is a structural proxy, not a semantic one. The verifier
   reopens outputs with openpyxl to catch the failure class that matters
   (corrupt relationships, as zavora 0.1.2 demonstrates).
+- The package-preservation score is superseded by content-model scoring in
+  DEC-027.
 
 ### DEC-026 — Calc tier: cache-free fixture, LibreOffice oracle, strict status semantics (2026-09-08)
 
@@ -257,7 +310,8 @@ ExcelBench install or CI job.
 - The first oracle sprint can focus on helper binaries/scripts and fixture
   truth-passing without changing benchmark scoring.
 - Promotion into public results requires a later explicit decision once the
-  generated workbooks are deterministic and manually audited.
+  generated workbooks are deterministic and manually audited. DEC-027 records
+  that decision for the JSON-model adapters.
 
 **Commit(s)**: this commit.
 

src/excelbench/harness/mutation.py

@@ -183,14 +183,13 @@ def _missing_content_types(parts: dict[str, bytes]) -> list[str]:
 
 
 def _canonical_xml(data: bytes) -> str:
-    """Return C14N 2.0 text that ignores encoding, prefixes, attribute order and padding."""
+    """Return C14N 2.0 text that ignores encoding, prefixes, attribute order and padding.
+
+    The raw bytes go to the parser so the BOM and XML declaration select the decoding.
+    """
     try:
-        return ET.canonicalize(
-            xml_data=data.decode("utf-8-sig"),
-            strip_text=True,
-            rewrite_prefixes=True,
-        )
-    except (ET.ParseError, UnicodeDecodeError):
+        return ET.canonicalize(xml_data=data, strip_text=True, rewrite_prefixes=True)
+    except ET.ParseError:
         return f"unparsable:{hashlib.sha256(data).hexdigest()}"
 
 

src/excelbench/results/html_dashboard.py

@@ -73,6 +73,20 @@
 ]
 
 
+def _scored_results(results: list[dict[str, Any]]) -> list[dict[str, Any]]:
+    """Drop rows of features no adapter scored (read and write null for every adapter).
+
+    Such a feature is not in any denominator; counting it would charge every
+    library for a feature the run could not score.
+    """
+    scored = {
+        e["feature"]
+        for e in results
+        if any(e.get("scores", {}).get(op) is not None for op in ("read", "write"))
+    }
+    return [e for e in results if e["feature"] in scored]
+
+
 def _compute_radar_data(
     fidelity: dict[str, Any],
     perf: dict[str, Any] | None,
@@ -88,7 +102,7 @@ def _compute_radar_data(
       - Feature Coverage: features_with_any_score / total_features * 100
       - Capability Breadth: R-only/W-only=33, R+W=66, R+W+Patch=100
     """
-    results = fidelity.get("results", [])
+    results = _scored_results(fidelity.get("results", []))
     libs_info = fidelity.get("libraries", {})
     focus = focus_libs or _DEFAULT_RADAR_LIBS
     available = set(libs_info.keys())
@@ -1404,8 +1418,9 @@ def _section_overview(fidelity: dict[str, Any], perf: dict[str, Any] | None) ->
     meta = fidelity.get("metadata", {})
     libs = fidelity.get("libraries", {})
     results = fidelity.get("results", [])
+    scored = _scored_results(results)
 
-    all_features = sorted({e["feature"] for e in results})
+    all_features = sorted({e["feature"] for e in scored})
     total_libs = len(libs)
     total_feats = len(all_features)
 
@@ -1424,11 +1439,11 @@ def _section_overview(fidelity: dict[str, Any], perf: dict[str, Any] | None) ->
 
     # Green count
     green = sum(
-        1 for e in results
+        1 for e in scored
         if max((s for s in [e["scores"].get("read"), e["scores"].get("write")]
                 if s is not None), default=-1) == 3
     )
-    total_scored = len(results)
+    total_scored = len(scored)
 
     cards = [
         (str(total_libs), "Libraries Tested"),
@@ -1449,11 +1464,11 @@ def _section_overview(fidelity: dict[str, Any], perf: dict[str, Any] | None) ->
 
     # WolfXL hero stats
     wolf_green = sum(
-        1 for e in results if e["library"] == "wolfxl"
+        1 for e in scored if e["library"] == "wolfxl"
         and max((s for s in [e["scores"].get("read"), e["scores"].get("write")]
                  if s is not None), default=-1) == 3
     )
-    wolf_scored = sum(1 for e in results if e["library"] == "wolfxl")
+    wolf_scored = sum(1 for e in scored if e["library"] == "wolfxl")
     wolf_hero = ""
     if wolf_scored > 0:
         # Compute throughput headline from perf data if available
@@ -1476,7 +1491,7 @@ def _section_overview(fidelity: dict[str, Any], perf: dict[str, Any] | None) ->
             f'<div class="wolf-icon">\U0001F43A</div>'
             f'<div class="wolf-text">'
             f'<h3>WolfXL at a glance (maintained by SynthGL)</h3>'
-            f'<p><b>{wolf_green}/{wolf_scored}</b> features at full fidelity (score 3)'
+            f'<p><b>{wolf_green}/{wolf_scored}</b> features scored 3 in read or write'
             f'{wolf_perf_note}. '
             f'Hybrid Rust+Python engine with an openpyxl-compatible API. '
             f'<a href="https://github.com/SynthGL/wolfxl" '

tests/test_mutation_suite.py

@@ -224,6 +224,23 @@ def test_equivalent_rewrite_scores_the_same_as_in_place_edit(tmp_path: Path) ->
     assert rewritten_score["diagnostics"]["lost_parts"]  # renamed parts are only diagnostics
 
 
+def test_utf16_custom_xml_scores_the_same_as_in_place_edit(tmp_path: Path) -> None:
+    """Re-encoding customXml as UTF-16 with a matching declaration is not a content change."""
+    in_place = _write_parts(tmp_path / "in_place.xlsx", _correct_output())
+    parts = _correct_output()
+    item = "customXml/item1.xml"
+    _replace(parts, item, b'encoding="UTF-8"', b'encoding="UTF-16"')
+    parts[item] = parts[item].decode("utf-8").encode("utf-16")
+    utf16 = _write_parts(tmp_path / "utf16.xlsx", parts)
+
+    in_place_score = score_preservation(TEMPLATE, in_place, MUTATIONS)
+    utf16_score = score_preservation(TEMPLATE, utf16, MUTATIONS)
+
+    assert utf16_score["features_changed"] == {}
+    assert utf16_score["preserved"] is in_place_score["preserved"] is True
+    assert utf16_score["feature_preservation"] == in_place_score["feature_preservation"]
+
+
 @pytest.mark.parametrize(
     ("drop", "feature"),
     [

tests/test_results_html_dashboard.py

@@ -2,7 +2,11 @@
 
 from pathlib import Path
 
-from excelbench.results.html_dashboard import _compute_radar_data, render_html_dashboard
+from excelbench.results.html_dashboard import (
+    _compute_radar_data,
+    _section_overview,
+    render_html_dashboard,
+)
 
 
 def test_render_html_dashboard_smoke(tmp_path: Path) -> None:
@@ -111,6 +115,45 @@ def test_compute_radar_data_uses_p50_when_op_count_missing() -> None:
     assert by_lib["pandas"][2] == 25.0
 
 
+def test_feature_no_adapter_scored_is_excluded_from_denominators() -> None:
+    """A feature null for every adapter leaves the denominators; one scored by any stays."""
+    fidelity = {
+        "libraries": {
+            "wolfxl": {"capabilities": ["read", "write", "modify"]},
+            "openpyxl": {"capabilities": ["read", "write"]},
+        },
+        "results": [
+            {"feature": "cell_values", "library": "wolfxl", "scores": {"read": 3, "write": 3}},
+            {"feature": "cell_values", "library": "openpyxl", "scores": {"read": 3, "write": 3}},
+            {"feature": "formulas", "library": "wolfxl", "scores": {"read": 1, "write": None}},
+            {
+                "feature": "formulas",
+                "library": "openpyxl",
+                "scores": {"read": None, "write": None},
+            },
+            {
+                "feature": "pivot_tables",
+                "library": "wolfxl",
+                "scores": {"read": None, "write": None},
+            },
+            {
+                "feature": "pivot_tables",
+                "library": "openpyxl",
+                "scores": {"read": None, "write": None},
+            },
+        ],
+    }
+
+    html = _section_overview(fidelity, None)
+
+    assert "<b>1/2</b> features scored 3 in read or write" in html
+    assert '<div class="val">2</div><div class="lbl">Features Scored</div>' in html
+    assert '<div class="val">2/4</div><div class="lbl">Score\u20033 Results</div>' in html
+    rows = _compute_radar_data(fidelity, None, focus_libs=["wolfxl", "openpyxl"])
+    coverage = {r["library"]: r["values"][3] for r in rows}
+    assert coverage == {"wolfxl": 100.0, "openpyxl": 50.0}
+
+
 def test_render_html_dashboard_shows_delta_and_unsupported(tmp_path: Path) -> None:
     fidelity = tmp_path / "results.json"
     fidelity.write_text(