Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitleaksignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# remote-source-sync receipt: "source_key" is the public git commit sha, not a credential
results-2026-10-02/perf/README.md:generic-api-key:36
27 changes: 21 additions & 6 deletions METHODOLOGY.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,12 +102,27 @@ subprocess and saves the result.

- **Wall time**: median over `--repeats` (default 3) subprocess runs.
- **Peak RSS**: median subprocess peak resident set size.
- **Preservation score**: fraction of original package parts retained after the
edit, minus damage penalties; outputs must reopen cleanly with the openpyxl
verifier or the engine is marked `integrity-failed` with score 0.

Preservation is a structural proxy (ZIP part inventory), not a semantic
equivalence claim. Cite wall time and preservation together.
- **Preservation (content model)**: the template is read into the vendored
xlsx content model (`src/excelbench/harness/content_model_xlsx.py`, copied
verbatim from `officelibs-corpus`), the two declared cell edits are applied
to that model the way a correct engine applies them, and the result is
compared with the output's model. Any difference is an unexpected change,
counted per feature (`features_changed`); `feature_preservation` is `k/n`
for the `n` features present in the template. Custom XML data items are an
extra scorer-side feature (`custom_xml`, canonical XML plus datastore id).
Cached results of formula cells are recalculation state and are not
compared; formula text is.
- **Edits applied**: each edited cell must hold its new value, checked
separately from preservation.
- **Integrity**: dangling relationships and parts without a content type are
reported but not scored.
- **Verdict**: Preserved / Changed `<features>` / Edits missing / Failed.

Part names, relationship ids, XML serialization and document metadata are not
content, so an equivalent re-serialization scores the same as a byte copy.
Part-name based checks (lost/added parts, worksheet elements in same-named
parts) survive only as an unscored `diagnostics` block. Cite wall time and
preservation together.

## Calculation Lane

Expand Down
79 changes: 49 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,24 +13,41 @@ cross-language context lane (Apache POI, Excelize, zavora-xlsx). Tier 4

## Results at a Glance

> Competitor snapshot: 2026-09-08 | WolfXL 2.1.0 wheel | aspose-cells-foss 26.7 | [Fidelity](results-2026-09-08/xlsx/README.md) | [Mutation](results-2026-09-08/mutation/README.md) | [Calc](results-2026-09-08/calc/README.md) | [Zavora cross-language](results-2026-09-08/cross-language/README.md)
> Competitor snapshot: 2026-10-02 | WolfXL 2.0.5 (latest PyPI) | 17 fidelity adapters incl. SheetJS CE 0.20.3, ExcelJS 4.4.0, LibreOffice 26.8.0.3 | [Snapshot README](results-2026-10-02/README.md) | [Fidelity](results-2026-10-02/xlsx/README.md) | [Mutation](results-2026-10-02/mutation/README.md) | [Calc](results-2026-10-02/calc/README.md) | [Cross-language](results-2026-10-02/cross-language/README.md)
>
> In that snapshot: openpyxl 22/22, WolfXL 19/22, aspose-cells-foss 11/22
> green features (WolfXL 2.1.0 gaps: conditional-formatting read,
> named-range and print-title writes).
> Green features in that snapshot (read / write, out of 21¹): openpyxl
> 21/21 / 21/21, WolfXL 19/21 / 18/21, ExcelJS 12/21 / 15/21,
> aspose-cells-foss 11/21 / 14/21, LibreOffice 12/21 / 12/21, SheetJS CE
> 9/21 / 8/21; write-only xlsxwriter 15/21.
>
> Mutation suite (two-cell edit on a corporate template): WolfXL patches in
> 1.5s with 100% preservation; openpyxl and aspose rewrite at 60%;
> zavora-xlsx 0.1.2 corrupts package relationships (0%).
> Mutation suite (two-cell edit on a corporate template, content-model
> preservation; features kept / features in the template): WolfXL
> Preserved 17/17; openpyxl 16/17 (drops custom XML); LibreOffice 15/17
> (cell styles, page setup); aspose-cells-foss 15/17 (drops 87 string
> cells, custom XML); SheetJS 8/17 and zavora-xlsx 8/17 (charts, tables,
> styles and more); ExcelJS fails to load the template. Every engine that
> ran applied both edits. Wall times in that run are not comparable (host
> under heavy unrelated load).
>
> Calc tier (133-formula financial DAG, cache-free fixture, LibreOffice
> oracle): LibreOffice 133/133, WolfXL 55/133, aspose-cells-foss 25/133,
> zavora 0/133.
> oracle): LibreOffice 133/133, aspose-cells-foss 25/133, WolfXL 0/133,
> zavora 0/133. SheetJS CE, ExcelJS, and the other non-calculating
> adapters are not applicable.
>
> Cross-language context (write lane): Apache POI 5.5.1 18/21, Excelize
> 2.10.1 18/21, zavora-xlsx 0.1.2 8/21. The POI and Excelize misses are
> the three Tier 4 features, which the ExcelBench adapters do not
> implement yet (`NotImplementedError`). They are not library gaps.
>
> ¹ `pivot_tables` is excluded from every denominator because no adapter
> produced a score for it in either competitor snapshot.
>
> Python release snapshot: 2026-04-29 UTC | wheel-backed WolfXL 2.0 rerun | [Fidelity](results-release-2026-04-28/README.md) | [Perf](results-release-2026-04-28/perf/README.md) | [Dashboard](results-release-2026-04-28/DASHBOARD.md)
>
> In that snapshot, WolfXL reaches `18/18` green features with `100%` pass rate.
>
> Performance snapshot: 2026-10-02 UTC | WolfXL 2.0.5 through its public API (`wolfxl.load_workbook`, `wolfxl.Workbook`) | [Perf](results-2026-10-02/perf/README.md). It supersedes the 2026-04-29 perf numbers for WolfXL, which measured private backend objects.
>
> Cross-language context snapshot: [Apache POI `18/18`](results-cross-language/README.md) | [Excelize `18/18`](results-cross-language/README.md)
>
> Pivot capability lane: [separate artifact](results-cross-language-pivots/README.md) because the shipped macOS pivot fixture is not scoreable, while `excelize` can still emit pivot-bearing workbooks.
Expand Down Expand Up @@ -71,7 +88,7 @@ Use this when the question is: how does WolfXL compare to serious spreadsheet to
Use this when the question is: can the cross-language helpers detect or emit pivot-bearing workbooks even when the main scored fixture is not valid on macOS?

4. **Template-mutation lane**
Use this when the question is: which engine can surgically edit a corporate template fastest while preserving the package?
Use this when the question is: which engine can surgically edit a corporate template fastest while preserving everything else in the workbook?

5. **Formula-recalculation lane**
Use this when the question is: which engine actually computes a 133-formula financial DAG from scratch (cache-free fixture, LibreOffice oracle)?
Expand All @@ -80,26 +97,26 @@ Use this when the question is: which engine actually computes a 133-formula fina

| Library | Caps | Fidelity | Read Speed | Write Speed | Modify |
|---------|:----:|:--------:|:----------:|:-----------:|:------:|
| **wolfxl** | R+W | 19/22 in 2026-09-08 competitor snapshot (18/18 in 2026-04-29 snapshot) | workload-specific | workload-specific | Patch (100% preservation in mutation suite) |
| openpyxl | R+W | 22/22 in 2026-09-08 competitor snapshot | 1x (baseline per workload) | 1x (baseline per workload) | Rewrite (60% preservation in mutation suite) |
| aspose-cells-foss | R+W | 11/22 in 2026-09-08 competitor snapshot | workload-specific | workload-specific | Rewrite (60% preservation) |
| xlsxwriter | W | 15/18 in 2026-04-29 release snapshot | -- | ~1x | No |
| xlsxwriter-constmem | W | 12/18 in 2026-04-29 release snapshot | -- | ~2x | No |
| python-calamine | R | 1/18 in 2026-04-29 release snapshot | ~1.3x | -- | No |
| pandas | R+W | 3/18 in 2026-04-29 release snapshot | <1x | <1x | Rebuild |
| polars | R | 0/18 in 2026-04-29 release snapshot | ~1x | -- | No |
| **wolfxl** | R+W | R 20/21, W 19/21 in 2026-09-08 competitor snapshot (18/18 in 2026-04-29 snapshot) | 1.69x (13.46 ms) | 1.20x (22.28 ms) | Patch (Preserved, 17/17 features in 2026-10-02 mutation suite) |
| openpyxl | R+W | R 21/21, W 21/21 in 2026-09-08 competitor snapshot | 1.00x (22.79 ms, baseline) | 1.00x (26.70 ms, baseline) | Rewrite (16/17 features in 2026-10-02 mutation suite) |
| aspose-cells-foss | R+W | R 11/21, W 14/21 in 2026-09-08 competitor snapshot | not measured | not measured | Rewrite (15/17 features) |
| xlsxwriter | W | 15/18 in 2026-04-29 release snapshot | -- | 0.86x (31.12 ms) | No |
| xlsxwriter-constmem | W | 12/18 in 2026-04-29 release snapshot | -- | 0.86x (31.17 ms) | No |
| python-calamine | R | 1/18 in 2026-04-29 release snapshot | 14.02x (1.63 ms) | -- | No |
| pandas | R+W | 3/18 in 2026-04-29 release snapshot | 0.80x (28.43 ms) | 1.00x (26.66 ms) | Rebuild |
| polars | R | 0/18 in 2026-04-29 release snapshot | 3.72x (6.13 ms) | -- | No |

> Speed numbers are snapshot-specific. Always cite the artifact date, workload, and profile. See [performance results](results/perf/README.md), [METHODOLOGY.md](METHODOLOGY.md), and [Public Reporting Status](docs/public-reporting.md).
> Speed columns come from the [2026-10-02 perf snapshot](results-2026-10-02/perf/README.md): sum of per-feature p50 wall times over the same 19 xlsx features for every library (warmup 3, 25 iterations, Apple M4 Pro, Python 3.12.3), shown as openpyxl's total divided by the library's total. WolfXL 2.0.5 is measured through its public API (`wolfxl.load_workbook`, `wolfxl.Workbook`), like every other library. aspose-cells-foss was not part of the perf run. Always cite the artifact date, workload, and profile. See [METHODOLOGY.md](METHODOLOGY.md) and [Public Reporting Status](docs/public-reporting.md).

### Key Findings
- **High-fidelity libraries are rare**: in the 2026-04-29 release snapshot, only openpyxl and WolfXL reached 18/18 green features; in the 2026-09-08 competitor snapshot (22 features), openpyxl holds 22/22 while WolfXL 2.1.0 drops to 19/22
- **High-fidelity libraries are rare**: in the 2026-04-29 release snapshot, only openpyxl and WolfXL reached 18/18 green features; in the 2026-09-08 competitor snapshot (21 scored features; `pivot_tables` unscored), openpyxl holds 21/21 read and write while WolfXL 2.1.0 reaches 20/21 read and 19/21 write
- **WolfXL 2.1.0 regression signal**: conditional-formatting read scores 0 and named-range / print-title writes score 2 in the 2026-09-08 snapshot; tracked for the WolfXL repo
- **Patch modify is structurally different**: WolfXL's `load_workbook(path, modify=True)` uses surgical ZIP patching; it is the only engine reaching 100% package preservation in the mutation suite
- **Patch modify is structurally different**: WolfXL's `load_workbook(path, modify=True)` uses surgical ZIP patching; it is the only engine whose output keeps every template feature in the 2026-10-02 mutation suite
- **The abstraction tax is real**: pandas wraps openpyxl but drops from 16 to 3 green features due to DataFrame coercion (errors become NaN)
- **Speed vs fidelity tradeoff is measurable**: use the perf snapshot together with the fidelity matrix rather than quoting one without the other
- **Optimization modes have clear costs**: openpyxl-readonly loses 13 green features for streaming speed
- **Cross-language context is now strong too**: `Apache POI` and `Excelize` land at `18/18` in the scored write lane; `zavora-xlsx` 0.1.2 writes fast but corrupts hyperlink relationships on mutate and cannot recalculate
- **Calculation is a differentiator**: only LibreOffice computes the full 133-formula DAG; WolfXL 2.1.0 covers 55/133 (power-operator family returns None), aspose-cells-foss 25/133
- **Calculation is a differentiator**: only LibreOffice computes the full 133-formula DAG (2026-10-02 snapshot); aspose-cells-foss covers 25/133; WolfXL 2.0.5 scores 0/133 because `Workbook.calculate()` runs but the saved workbook carries no cached formula values


See the [release snapshot dashboard](results-release-2026-04-28/DASHBOARD.md) for the fresh wheel-backed combined view, or the [historical dashboard](results/DASHBOARD.md) for the older public baseline.
Expand All @@ -119,9 +136,9 @@ See the [release snapshot dashboard](results-release-2026-04-28/DASHBOARD.md) fo

| Library | Version | Lang | Caps | Green Features |
|:--------|:--------|:-----|:-----|:--------------:|
| [WolfXL](https://github.com/SynthGL/wolfxl) | 2.1.0 | Python (Rust core) | R+W | 19/22 (2026-09-08 snapshot) |
| [openpyxl](https://openpyxl.readthedocs.io/) | 3.1.5 | Python | R+W | 22/22 (2026-09-08 snapshot) |
| [aspose-cells-foss](https://pypi.org/project/aspose-cells-foss/) | 26.7 | Python (JVM-core FOSS) | R+W | 11/22 (2026-09-08 snapshot) |
| [WolfXL](https://github.com/SynthGL/wolfxl) | 2.1.0 | Python (Rust core) | R+W | R 20/21, W 19/21 (2026-09-08 snapshot) |
| [openpyxl](https://openpyxl.readthedocs.io/) | 3.1.5 | Python | R+W | R 21/21, W 21/21 (2026-09-08 snapshot) |
| [aspose-cells-foss](https://pypi.org/project/aspose-cells-foss/) | 26.7 | Python (JVM-core FOSS) | R+W | R 11/21, W 14/21 (2026-09-08 snapshot) |
| [XlsxWriter](https://xlsxwriter.readthedocs.io/) | 3.2.9 | Python | W | 15/18 |
| [xlsxwriter-constmem](https://xlsxwriter.readthedocs.io/) | 3.2.9 | Python | W | 12/18 |
| [openpyxl-readonly](https://openpyxl.readthedocs.io/) | 3.1.5 | Python | R | 3/18 |
Expand All @@ -135,8 +152,9 @@ See the [release snapshot dashboard](results-release-2026-04-28/DASHBOARD.md) fo
| [xlrd](https://github.com/python-excel/xlrd) | 2.0.2 | Python | R | .xls only |

> Green-feature counts are per dated snapshot; `x/18` numbers come from the
> 2026-04-29 release snapshot and `x/22` from the 2026-09-08 competitor
> snapshot (Tier 4 added). Never mix counts across snapshots.
> 2026-04-29 release snapshot and `x/21` from the 2026-09-08 competitor
> snapshot (Tier 4 added; `pivot_tables` is unscored and excluded from the
> denominator). Never mix counts across snapshots.

### Cross-Language Context (oracle helpers)

Expand Down Expand Up @@ -289,7 +307,7 @@ uv run excelbench generate --output fixtures/excel
| **Tier 4** -- Production surfaces (2026-09-08) | Sheet protection, page setup, chart anchoring | 3 |

> Pivot tables are tested but score N/A across all adapters in the current macOS run.
> Green-feature denominators: /18 in the 2026-04-29 release snapshot, /22 in the 2026-09-08 competitor snapshot.
> Green-feature denominators: /18 in the 2026-04-29 release snapshot, /21 in the 2026-09-08 and 2026-10-02 competitor snapshots (22 features modeled; `pivot_tables` is unscored and excluded).

### Planned

Expand All @@ -305,8 +323,9 @@ None currently queued. Tier 4 (charts anchoring, print settings, protection) shi
- **[XLSX results](results/xlsx/README.md)** -- per-library, per-test-case breakdowns with tier list
- **[Release snapshot results](results-release-2026-04-28/README.md)** -- fresh wheel-backed WolfXL 2.0 rerun
- **[XLS results](results/xls/README.md)** -- legacy format results
- **[Performance results](results/perf/README.md)** -- throughput benchmarks (cells/s)
- **[Release snapshot perf](results-release-2026-04-28/perf/README.md)** -- matching wheel-backed perf snapshot
- **[Performance results](results/perf/README.md)** -- 2026-04-20 perf snapshot (historical; its WolfXL column measured private backend objects)
- **[Release snapshot perf](results-release-2026-04-28/perf/README.md)** -- 2026-04-29 perf snapshot (historical; its WolfXL column measured private backend objects)
- **[2026-10-02 perf snapshot](results-2026-10-02/perf/README.md)** -- current perf: 13 Python adapters, 19 features, WolfXL 2.0.5 through its public API
- **[Dashboard](results/DASHBOARD.md)** -- combined fidelity + performance comparison
- **[Release snapshot dashboard](results-release-2026-04-28/DASHBOARD.md)** -- combined view for the fresh rerun
- **[Heatmap (PNG)](results/xlsx/heatmap.png)** | **[SVG](results/xlsx/heatmap.svg)** -- visual score matrix
Expand Down
2 changes: 1 addition & 1 deletion docs-site/adapters/openpyxl.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -53,4 +53,4 @@ wb = load_workbook("data.xlsx", read_only=True)
openpyxl is **slow** by default. For large files:

- Use `read_only=True` for reading (no in-memory model built)
- Consider WolfXL for 4-6x read speedup with comparable fidelity
- Consider WolfXL for a 1.69x read speedup (2026-10-02 perf snapshot: WolfXL 2.0.5 `wolfxl.load_workbook` 13.46 ms vs openpyxl 22.79 ms, summed p50 over 19 features); in the 2026-10-02 fidelity snapshot WolfXL has 17/21 features green in both read and write against openpyxl's 21/21
2 changes: 1 addition & 1 deletion docs-site/adapters/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ ExcelBench tests adapters across three categories.

| Adapter | Capabilities | Notes |
|---------|-------------|-------|
| `wolfxl` | R+W+Modify | WolfXL: calamine read + rust_xlsxwriter write + XlsxPatcher |
| `wolfxl` | R+W+Modify | WolfXL public API: `load_workbook`, `Workbook`, patch-mode `load_workbook(path, modify=True)` |
| `calamine-styled` | R | calamine fork with style support |
| `rust-calamine` | R | Basic calamine (no styles) |
| `rust-xlsxwriter` | W | rust_xlsxwriter write bindings |
Expand Down
32 changes: 17 additions & 15 deletions docs-site/adapters/wolfxl.mdx
Original file line number Diff line number Diff line change
@@ -1,15 +1,17 @@
---
title: "WolfXL"
description: "The fastest Python-accessible Excel library. Read + Write + Modify in one."
description: "Rust-backed Python Excel library with an openpyxl-compatible API. Read + Write + Modify in one."
---

WolfXL is a hybrid Rust library that combines three backends:
WolfXL is a Rust-backed Python library with an openpyxl-compatible API. ExcelBench drives it through that public API:

| Operation | Backend | Speed vs openpyxl |
|-----------|---------|-------------------|
| Read | calamine (styled fork) | **4-6x faster** |
| Write | rust_xlsxwriter | **5x faster** |
| Modify | XlsxPatcher (ZIP surgery) | **10-14x faster** |
| Operation | Public API (WolfXL 2.0.5) | Speed vs openpyxl (2026-10-02 perf snapshot) |
|-----------|---------------------------|-------------------|
| Read | `wolfxl.load_workbook(path)` | **1.69x faster** (13.46 ms vs 22.79 ms) |
| Write | `wolfxl.Workbook()` then `save(path)` | **1.20x faster** (22.28 ms vs 26.70 ms) |
| Modify | `wolfxl.load_workbook(path, modify=True)` then `save(path)`: patch mode rewrites only the changed sheet XML and copies every other ZIP entry unchanged | No current measurement |

Speed figures are sums of per-feature p50 wall times over the same 19 xlsx features for both libraries (warmup 3, 25 iterations, Apple M4 Pro, Python 3.12.3), measured on WolfXL 2.0.5 through its public API (`wolfxl.load_workbook`, `wolfxl.Workbook`). Source: `results-2026-10-02/perf/README.md`.

## Install

Expand All @@ -35,7 +37,7 @@ ws = wb.active
ws["A1"] = "Hello, world"
wb.save("output.xlsx")

# Modify (surgical patch — no full rewrite)
# Modify (surgical patch, no full rewrite)
wb = wolfxl.load_workbook("template.xlsx", modify=True)
ws = wb.active
ws["B2"] = 42
Expand All @@ -46,21 +48,21 @@ wb.save("patched.xlsx")

Standard Excel libraries rewrite the entire `.xlsx` ZIP archive on save. WolfXL's modify mode instead patches only the changed XML sheets in-place, leaving all other ZIP entries untouched.

This is 10-14x faster for templates with large static content (images, charts, pivot tables) where only a few cells change.
Large static content in a template (images, charts, pivot tables) is never parsed or re-serialized when only a few cells change.

## Benchmark Scores

From the current ExcelBench run:
From the 2026-10-02 snapshot (WolfXL 2.0.5; fidelity from `results-2026-10-02/xlsx`, 21 scored features with `pivot_tables` unscored; speed from `results-2026-10-02/perf`):

| Axis | Score |
|------|-------|
| Fidelity | 87.5% (14/16 features = 3) |
| Read Speed | 100 (5.9x openpyxl, capped) |
| Write Speed | 100 (3.7x openpyxl, capped) |
| Feature Coverage | 88.2% |
| Fidelity | 17/21 features green in both read and write (read 19/21, write 18/21) |
| Read Speed | 1.69x openpyxl (WolfXL 2.0.5 public API) |
| Write Speed | 1.20x openpyxl (WolfXL 2.0.5 public API) |
| Feature Coverage | 100% (21/21 features score at least 1 in read or write) |
| Capability Breadth | 100 (R+W+Modify) |

Known gaps: `alignment` (indent level) = 1/3, `images` (read) = 0/3.
Known gaps: `named_ranges` read 2/3 and write 2/3, `tables` read 2/3, `page_setup` write 2/3, `chart_anchor` write 0/3.

## Source

Expand Down
Loading
Loading