Skip to content

Merge upstream Vortex 0.86.1 into spiceai-55 - #100

Merged
lukekim merged 500 commits into
spiceai-55from
spiceai-0.86
Sep 28, 2026
Merged

lukekim merged 500 commits into
spiceai-55from
spiceai-0.86

Conversation

@lukekim

@lukekim lukekim commented Sep 19, 2026

Copy link
Copy Markdown

Upgrades the spiceai-54 line from upstream 0.79.0 to official 0.86.1, the latest Vortex release that supports DataFusion 55.1 and Rust 1.98.1.

Workspace pins on this branch:

  • DataFusion crates: 55.1.0 (0.86.1 shipped 55.0.0; caret-compatible, locked to 55.1.0 here)
  • rust-toolchain.toml: 1.98.1 (tag was 1.98.0; MSRV remains 1.95)
  • Arrow: 59.2 (was 58.3 on 0.79.0)

Conflict resolutions

  • vortex-arrow/* Map import/export → upstream native Map (#9111), replacing the fork's List alias
  • vortex-datafusion/src/persistent/sink.rs → upstream (metrics, DF 55 APIs). target_file_size_mb remains on format options; the #33 sink patch still lives in spice2's in-repo crates/vortex
  • .github/workflows/{ci,rust-instrumented,develop-bench}.yml → union (fork runner guards + upstream sccache / ty / CMake job rename)
  • Deleted upstream workflows (bench-pr.yml, sql-benchmarks.yml) accepted
  • DuckDB convert timezone checks kept on top of vortex-spatial / UUID from 0.86.1

Fork patches verified surviving the merge

  • Tokio oneshot (cancel-path SIGSEGV) — vortex-io handle/single + vortex-file segment reads (SPICE_PATCHES rows 1–3). cancel_stress passed
  • Fixed-offset timezone resolution (timezone.rs)
  • Hashed IN-list probe + interval falsifier
  • Declared / absolute scan concurrency (SplitConcurrency, set_available_parallelism)
  • WriteStrategyBuilder as a session variable
  • Flat-reader subrange decode reuse (OnceLock)

Arrow Map alias (row 14) is dropped: 0.86.1 has a real Map type.

Tests

  • cargo check on vortex-{array,io,file,layout,datafusion,arrow,utils,session,scan} and vortex-fsst: clean
  • cargo nextest run on those consumed crates (--features tokio): 4802 passed, 1 skipped
  • Not run locally (external toolchains): vortex-duckdb, vortex-python, Java bindings

renovate Bot and others added 30 commits August 10, 2026 01:00
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [actions/stale](https://redirect.github.com/actions/stale) | action |
major | `v10` → `v11` |

---

> [!WARNING]
> Some dependencies could not be looked up. Check the [Dependency
Dashboard](..vortex-data/issues/357) for more information.

---

### Release Notes

<details>
<summary>actions/stale (actions/stale)</summary>

###
[`v11.0.0`](https://redirect.github.com/actions/stale/releases/tag/v11.0.0)

[Compare
Source](https://redirect.github.com/actions/stale/compare/v11.0.0...v11.0.0)

#### What's Changed

##### Enhancement

- Migrate to ESM and update dependencies by
[@&vortex-data#8203;chiranjib-swain](https://github-grid.enterprise.slack.com/team/U08CVLQ4JKE)
in [#&vortex-data#8203;1350](https://redirect.github.com/actions/stale/pull/1350)

##### Dependency Update

- Override brace-expansion to 5.0.8 to address 24 high-severity
dependency vulnerabilities by
[@&vortex-data#8203;dependabot](https://redirect.github.com/dependabot) in
[#&vortex-data#8203;1351](https://redirect.github.com/actions/stale/pull/1351)

**Full Changelog**:
<actions/stale@v10...v11.0.0>

###
[`v11`](https://redirect.github.com/actions/stale/compare/v10.4.0...v11.0.0)

[Compare
Source](https://redirect.github.com/actions/stale/compare/v10.4.0...v11.0.0)

</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
- Between 12:00 AM and 03:59 AM, on the second Monday and fourth Monday
of the month (`* 0-3 * * 1#2,1#4`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/vortex-data/vortex).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xMi4wIiwidXBkYXRlZEluVmVyIjoiNDQuMTIuMCIsInRhcmdldEJyYW5jaCI6ImRldmVsb3AiLCJsYWJlbHMiOlsiY2hhbmdlbG9nL2Nob3JlIl19-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
|
[taiki-e/install-action](https://redirect.github.com/taiki-e/install-action)
([changelog](https://redirect.github.com/taiki-e/install-action/compare/41049aa56687c35e0afa74eed4f09cec4f9afabf..7f4eb899022d8fe70b20c4f3de697aa85c309026))
| action | digest | `41049aa` → `7f4eb89` |

---

> [!WARNING]
> Some dependencies could not be looked up. Check the [Dependency
Dashboard](..vortex-data/issues/357) for more information.

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
- Between 12:00 AM and 03:59 AM, on the second Monday and fourth Monday
of the month (`* 0-3 * * 1#2,1#4`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/vortex-data/vortex).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xMi4wIiwidXBkYXRlZEluVmVyIjoiNDQuMTIuMCIsInRhcmdldEJyYW5jaCI6ImRldmVsb3AiLCJsYWJlbHMiOlsiY2hhbmdlbG9nL2Nob3JlIl19-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
|
[pypa/gh-action-pypi-publish](https://redirect.github.com/pypa/gh-action-pypi-publish)
([changelog](https://redirect.github.com/pypa/gh-action-pypi-publish/compare/ba38be9e461d3875417946c167d0b5f3d385a247..dc37677b2e1c63e2034f94d8a5b11f265b73ba33))
| action | digest | `ba38be9` → `dc37677` |

---

> [!WARNING]
> Some dependencies could not be looked up. Check the [Dependency
Dashboard](..vortex-data/issues/357) for more information.

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
- Between 12:00 AM and 03:59 AM, on the second Monday and fourth Monday
of the month (`* 0-3 * * 1#2,1#4`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/vortex-data/vortex).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xMi4wIiwidXBkYXRlZEluVmVyIjoiNDQuMTIuMCIsInRhcmdldEJyYW5jaCI6ImRldmVsb3AiLCJsYWJlbHMiOlsiY2hhbmdlbG9nL2Nob3JlIl19-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [gradle/actions](https://redirect.github.com/gradle/actions)
([changelog](https://redirect.github.com/gradle/actions/compare/3f131e8634966bd73d06cc69884922b02e6faf92..9c971963bec38e04b3d30dcc455b5382be2fdbfb))
| action | digest | `3f131e8` → `9c97196` |

---

> [!WARNING]
> Some dependencies could not be looked up. Check the [Dependency
Dashboard](..vortex-data/issues/357) for more information.

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
- Between 12:00 AM and 03:59 AM, on the second Monday and fourth Monday
of the month (`* 0-3 * * 1#2,1#4`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/vortex-data/vortex).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xMi4wIiwidXBkYXRlZEluVmVyIjoiNDQuMTIuMCIsInRhcmdldEJyYW5jaCI6ImRldmVsb3AiLCJsYWJlbHMiOlsiY2hhbmdlbG9nL2Nob3JlIl19-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
|
[dtolnay/rust-toolchain](https://redirect.github.com/dtolnay/rust-toolchain)
([changelog](https://redirect.github.com/dtolnay/rust-toolchain/compare/4cda84d5c5c54efe2404f9d843567869ab1699d4..4360b52568e2003a75bf9bc1d59f33a8e3fc893c))
| action | digest | `4cda84d` → `4360b52` |

---

> [!WARNING]
> Some dependencies could not be looked up. Check the [Dependency
Dashboard](..vortex-data/issues/357) for more information.

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
- Between 12:00 AM and 03:59 AM, on the second Monday and fourth Monday
of the month (`* 0-3 * * 1#2,1#4`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/vortex-data/vortex).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xMi4wIiwidXBkYXRlZEluVmVyIjoiNDQuMTIuMCIsInRhcmdldEJyYW5jaCI6ImRldmVsb3AiLCJsYWJlbHMiOlsiY2hhbmdlbG9nL2Nob3JlIl19-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [dorny/paths-filter](https://redirect.github.com/dorny/paths-filter)
([changelog](https://redirect.github.com/dorny/paths-filter/compare/7b450fff21473bca461d4b92ce414b9d0420d706..ceb8a2b8f2d89434be7ff52d3de7ec3738c5cc9d))
| action | digest | `7b450ff` → `ceb8a2b` |

---

> [!WARNING]
> Some dependencies could not be looked up. Check the [Dependency
Dashboard](..vortex-data/issues/357) for more information.

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
- Between 12:00 AM and 03:59 AM, on the second Monday and fourth Monday
of the month (`* 0-3 * * 1#2,1#4`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/vortex-data/vortex).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xMi4wIiwidXBkYXRlZEluVmVyIjoiNDQuMTIuMCIsInRhcmdldEJyYW5jaCI6ImRldmVsb3AiLCJsYWJlbHMiOlsiY2hhbmdlbG9nL2Nob3JlIl19-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [CodSpeedHQ/action](https://redirect.github.com/CodSpeedHQ/action)
([changelog](https://redirect.github.com/CodSpeedHQ/action/compare/0ca9cbbf4623b599a6c3ed4fc8a922942705d9f1..4296e51e7041e24dadb86d1d6e8b9320d223dbe8))
| action | digest | `0ca9cbb` → `4296e51` |

---

> [!WARNING]
> Some dependencies could not be looked up. Check the [Dependency
Dashboard](..vortex-data/issues/357) for more information.

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
- Between 12:00 AM and 03:59 AM, on the second Monday and fourth Monday
of the month (`* 0-3 * * 1#2,1#4`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/vortex-data/vortex).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xMi4wIiwidXBkYXRlZEluVmVyIjoiNDQuMTIuMCIsInRhcmdldEJyYW5jaCI6ImRldmVsb3AiLCJsYWJlbHMiOlsiY2hhbmdlbG9nL2Nob3JlIl19-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
|
[anthropics/claude-code-action](https://redirect.github.com/anthropics/claude-code-action)
([changelog](https://redirect.github.com/anthropics/claude-code-action/compare/be7b93b1907a4abad570368f3c74b6fe3807510b..6b082c41935b4c8a3b8b0ef85ba4ba4d9eeb8975))
| action | digest | `be7b93b` → `6b082c4` |

---

> [!WARNING]
> Some dependencies could not be looked up. Check the [Dependency
Dashboard](..vortex-data/issues/357) for more information.

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
- Between 12:00 AM and 03:59 AM, on the second Monday and fourth Monday
of the month (`* 0-3 * * 1#2,1#4`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/vortex-data/vortex).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xMi4wIiwidXBkYXRlZEluVmVyIjoiNDQuMTIuMCIsInRhcmdldEJyYW5jaCI6ImRldmVsb3AiLCJsYWJlbHMiOlsiY2hhbmdlbG9nL2Nob3JlIl19-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [actions/setup-java](https://redirect.github.com/actions/setup-java)
([changelog](https://redirect.github.com/actions/setup-java/compare/03ad4de0992f5dab5e18fcb136590ce7c4a0ac95..b6effb05e454b25005698d916606bdc6ffcbf961))
| action | digest | `03ad4de` → `b6effb0` |

---

> [!WARNING]
> Some dependencies could not be looked up. Check the [Dependency
Dashboard](..vortex-data/issues/357) for more information.

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
- Between 12:00 AM and 03:59 AM, on the second Monday and fourth Monday
of the month (`* 0-3 * * 1#2,1#4`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/vortex-data/vortex).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xMi4wIiwidXBkYXRlZEluVmVyIjoiNDQuMTIuMCIsInRhcmdldEJyYW5jaCI6ImRldmVsb3AiLCJsYWJlbHMiOlsiY2hhbmdlbG9nL2Nob3JlIl19-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [actions/checkout](https://redirect.github.com/actions/checkout)
([changelog](https://redirect.github.com/actions/checkout/compare/9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0..3d3c42e5aac5ba805825da76410c181273ba90b1))
| action | digest | `9c091bb` → `3d3c42e` |

---

> [!WARNING]
> Some dependencies could not be looked up. Check the [Dependency
Dashboard](..vortex-data/issues/357) for more information.

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
- Between 12:00 AM and 03:59 AM, on the second Monday and fourth Monday
of the month (`* 0-3 * * 1#2,1#4`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/vortex-data/vortex).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xMi4wIiwidXBkYXRlZEluVmVyIjoiNDQuMTIuMCIsInRhcmdldEJyYW5jaCI6ImRldmVsb3AiLCJsYWJlbHMiOlsiY2hhbmdlbG9nL2Nob3JlIl19-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
> ℹ️ **Note**
> 
> This PR body was truncated due to platform limits.

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [lance](https://redirect.github.com/lance-format/lance) | dependencies
| major | `9` → `10` |
| [lance-encoding](https://redirect.github.com/lance-format/lance) |
dependencies | major | `9` → `10` |

---

> [!WARNING]
> Some dependencies could not be looked up. Check the [Dependency
Dashboard](../issues/357) for more information.

---

### Release Notes

<details>
<summary>apache/arrow-rs (arrow-cast)</summary>

###
[`v59.2.0`](https://redirect.github.com/apache/arrow-rs/blob/HEAD/CHANGELOG.md#5920---2026-08-02)

[Compare
Source](https://redirect.github.com/apache/arrow-rs/compare/59.1.0...59.2.0)

[Full
Changelog](https://redirect.github.com/apache/arrow-rs/compare/59.1.0...59.2.0)

##### Enhancements

- arrow-avro: Deprecate object\_store integration by
[@&#8203;brancz](https://redirect.github.com/brancz) in
[#&#8203;10484](https://redirect.github.com/apache/arrow-rs/pull/10484)
- chore(arrow): add unchecked array builder methods by
[@&#8203;Rich-T-kid](https://redirect.github.com/Rich-T-kid) in
[#&#8203;10440](https://redirect.github.com/apache/arrow-rs/pull/10440)
- feat(arrow-ipc): add sans-IO stream encoder by
[@&#8203;Phoenix500526](https://redirect.github.com/Phoenix500526) in
[#&#8203;10277](https://redirect.github.com/apache/arrow-rs/pull/10277)
- chore(arrow-cast): support suffix white space in arrow-cast parse by
[@&#8203;Rich-T-kid](https://redirect.github.com/Rich-T-kid) in
[#&#8203;10396](https://redirect.github.com/apache/arrow-rs/pull/10396)
- feat(coalesce): add size function by
[@&#8203;rluvaton](https://redirect.github.com/rluvaton) in
[#&#8203;10331](https://redirect.github.com/apache/arrow-rs/pull/10331)
- fix(parquet): support mask filtering across skipped pages by
[@&#8203;hhhizzz](https://redirect.github.com/hhhizzz) in
[#&#8203;10288](https://redirect.github.com/apache/arrow-rs/pull/10288)
- chore(arrow-cast): support white space prefixed parse for ints and
floats by [@&#8203;Rich-T-kid](https://redirect.github.com/Rich-T-kid)
in
[#&#8203;10374](https://redirect.github.com/apache/arrow-rs/pull/10374)
- chore(arrow-buffer): Make more of i256 available in const code by
[@&#8203;AdamGS](https://redirect.github.com/AdamGS) in
[#&#8203;10363](https://redirect.github.com/apache/arrow-rs/pull/10363)
- feat(arrow-array): Expose builder buffer capacity accessors by
[@&#8203;Weijun-H](https://redirect.github.com/Weijun-H) in
[#&#8203;10342](https://redirect.github.com/apache/arrow-rs/pull/10342)
- feat(arrow-arith): Add interval multiplication by i64 by
[@&#8203;peterxcli](https://redirect.github.com/peterxcli) in
[#&#8203;10336](https://redirect.github.com/apache/arrow-rs/pull/10336)
- fix: Make `parquet-index` work with column paths by
[@&#8203;korowa](https://redirect.github.com/korowa) in
[#&#8203;10330](https://redirect.github.com/apache/arrow-rs/pull/10330)
- feat(arrow-csv): add support for parsing `Float16` by
[@&#8203;Glatzel](https://redirect.github.com/Glatzel) in
[#&#8203;10343](https://redirect.github.com/apache/arrow-rs/pull/10343)
- Remove the unmaintained paste dependency from arrow by
[@&#8203;Phoenix500526](https://redirect.github.com/Phoenix500526) in
[#&#8203;10303](https://redirect.github.com/apache/arrow-rs/pull/10303)
- chore: formalize the default map field names to match default arrow
spec by [@&#8203;rluvaton](https://redirect.github.com/rluvaton) in
[#&#8203;10297](https://redirect.github.com/apache/arrow-rs/pull/10297)
- feat(variant): Add `variant_to_arrow` `Map` type support by
[@&#8203;sdf-jkl](https://redirect.github.com/sdf-jkl) in
[#&#8203;10307](https://redirect.github.com/apache/arrow-rs/pull/10307)

##### Bug fixes

- fix(arrow-schema): Persist dictionary ordered flag on FFI schema
import by [@&#8203;borchero](https://redirect.github.com/borchero) in
[#&#8203;10514](https://redirect.github.com/apache/arrow-rs/pull/10514)
- fix(arrow-json): validate ListView child nullability by
[@&#8203;dk3yyyy](https://redirect.github.com/dk3yyyy) in
[#&#8203;10486](https://redirect.github.com/apache/arrow-rs/pull/10486)
- fix(arrow-json): validate map value nullability by
[@&#8203;subotac](https://redirect.github.com/subotac) in
[#&#8203;10475](https://redirect.github.com/apache/arrow-rs/pull/10475)
- avro: bound VLQDecoder::long against overlong varints by
[@&#8203;STiFLeR7](https://redirect.github.com/STiFLeR7) in
[#&#8203;10407](https://redirect.github.com/apache/arrow-rs/pull/10407)
- arrow-row: Fix decode\_fixed\_size\_list to apply the corrected\_type
step for dictionary children by
[@&#8203;zhuqi-lucas](https://redirect.github.com/zhuqi-lucas) in
[#&#8203;10414](https://redirect.github.com/apache/arrow-rs/pull/10414)
- \[Variant] make `value` mandatory field for
`VariantArray`/`ShreddingState` by
[@&#8203;sdf-jkl](https://redirect.github.com/sdf-jkl) in
[#&#8203;10318](https://redirect.github.com/apache/arrow-rs/pull/10318)
- fix off by one error for slice accounting by
[@&#8203;Rich-T-kid](https://redirect.github.com/Rich-T-kid) in
[#&#8203;10406](https://redirect.github.com/apache/arrow-rs/pull/10406)
- fix: `GenericByteViewArray::gc()` drops inline views on the
multi-buffer slow path by
[@&#8203;adriangb](https://redirect.github.com/adriangb) in
[#&#8203;10287](https://redirect.github.com/apache/arrow-rs/pull/10287)
- fix(arrow-json): render coerced f32 as its value in the string decoder
by [@&#8203;hareshkh](https://redirect.github.com/hareshkh) in
[#&#8203;10386](https://redirect.github.com/apache/arrow-rs/pull/10386)
- fix(arrow-cast): make `b64_encode` reject invalid UTF-8 from
misbehaving `Engine` impls by
[@&#8203;bit2swaz](https://redirect.github.com/bit2swaz) in
[#&#8203;10324](https://redirect.github.com/apache/arrow-rs/pull/10324)
- fix: take FFI\_ArrowArrayStream errno values from libc by
[@&#8203;fornwall](https://redirect.github.com/fornwall) in
[#&#8203;10299](https://redirect.github.com/apache/arrow-rs/pull/10299)
- fix(arrow-data): allow full dictionary key range when concatenating by
[@&#8203;raphaelroshan](https://redirect.github.com/raphaelroshan) in
[#&#8203;10323](https://redirect.github.com/apache/arrow-rs/pull/10323)
- Don't panic on invalid c ffi schema name by
[@&#8203;robert3005](https://redirect.github.com/robert3005) in
[#&#8203;10328](https://redirect.github.com/apache/arrow-rs/pull/10328)
- fix(REE): check upfront if sorting empty array or 0 limit by
[@&#8203;Jefffrey](https://redirect.github.com/Jefffrey) in
[#&#8203;10293](https://redirect.github.com/apache/arrow-rs/pull/10293)
- fix(arrow-avro): bound untrusted OCF block size and item counts by
[@&#8203;miniex](https://redirect.github.com/miniex) in
[#&#8203;10237](https://redirect.github.com/apache/arrow-rs/pull/10237)
- fix(arrow-array): disallow creating `MapArray` with nullable key field
by [@&#8203;rluvaton](https://redirect.github.com/rluvaton) in
[#&#8203;10272](https://redirect.github.com/apache/arrow-rs/pull/10272)
- fix: don't panic on `ArrayData::try_new` on bad input even when
`force_validate` feature is on by
[@&#8203;rluvaton](https://redirect.github.com/rluvaton) in
[#&#8203;10282](https://redirect.github.com/apache/arrow-rs/pull/10282)

##### Performance improvements

- perf(parquet): slice up contiguous buffer for decimals and fsb by
[@&#8203;MassivePizza](https://redirect.github.com/MassivePizza) in
[#&#8203;10364](https://redirect.github.com/apache/arrow-rs/pull/10364)
- feat(parquet): `RowSelection` can be backed by a `BooleanBuffer` by
[@&#8203;haohuaijin](https://redirect.github.com/haohuaijin) in
[#&#8203;10141](https://redirect.github.com/apache/arrow-rs/pull/10141)
- perf(parquet): use Cursor in ZSTDCodec to avoid Vec alloc and copy by
[@&#8203;MassivePizza](https://redirect.github.com/MassivePizza) in
[#&#8203;10345](https://redirect.github.com/apache/arrow-rs/pull/10345)
- optimize(parquet): Nested list batching child.write calls by
[@&#8203;mapleFU](https://redirect.github.com/mapleFU) in
[#&#8203;10085](https://redirect.github.com/apache/arrow-rs/pull/10085)
- perf(parquet): splice buffered pages with `write_all` instead of
`io::copy` (adapts
[#&#8203;10052](https://redirect.github.com/apache/arrow-rs/issues/10052))
by [@&#8203;adriangb](https://redirect.github.com/adriangb) in
[#&#8203;10353](https://redirect.github.com/apache/arrow-rs/pull/10353)
- hoist calls for null\_sentinel by
[@&#8203;Rich-T-kid](https://redirect.github.com/Rich-T-kid) in
[#&#8203;10356](https://redirect.github.com/apache/arrow-rs/pull/10356)
- perf: speed up substring\_by\_char with an ASCII fast path and
single-pass bounds by
[@&#8203;andygrove](https://redirect.github.com/andygrove) in
[#&#8203;10334](https://redirect.github.com/apache/arrow-rs/pull/10334)
- Cache encoded field name in FieldEncoder by
[@&#8203;MassivePizza](https://redirect.github.com/MassivePizza) in
[#&#8203;10296](https://redirect.github.com/apache/arrow-rs/pull/10296)
- perf: allow users to skip utf8 validation in arrow-row by
[@&#8203;Rich-T-kid](https://redirect.github.com/Rich-T-kid) in
[#&#8203;10319](https://redirect.github.com/apache/arrow-rs/pull/10319)
- perf: Improve decimal addition and subtraction when scale is equal by
[@&#8203;AdamGS](https://redirect.github.com/AdamGS) in
[#&#8203;10333](https://redirect.github.com/apache/arrow-rs/pull/10333)
- optimize(interleave): implement interleave for FixedSizeList/Map type
by [@&#8203;mapleFU](https://redirect.github.com/mapleFU) in
[#&#8203;10046](https://redirect.github.com/apache/arrow-rs/pull/10046)
- Perf: Pre-size buffer allocations to avoid intermediate allocations by
[@&#8203;Rich-T-kid](https://redirect.github.com/Rich-T-kid) in
[#&#8203;10262](https://redirect.github.com/apache/arrow-rs/pull/10262)
- Perf: create dictionary reader config and default unsafeflag to false
by [@&#8203;Rich-T-kid](https://redirect.github.com/Rich-T-kid) in
[#&#8203;10260](https://redirect.github.com/apache/arrow-rs/pull/10260)
- Perf: Introduce zero copy path when tonic returns an aligned buffer by
[@&#8203;Rich-T-kid](https://redirect.github.com/Rich-T-kid) in
[#&#8203;10273](https://redirect.github.com/apache/arrow-rs/pull/10273)
- Validate short view strings in separate buffer in arrow-row by
[@&#8203;Jefffrey](https://redirect.github.com/Jefffrey) in
[#&#8203;10250](https://redirect.github.com/apache/arrow-rs/pull/10250)

##### Documentation updates

- chore(parquet): add link to ticket in object\_store deprecation
message by [@&#8203;alamb](https://redirect.github.com/alamb) in
[#&#8203;10502](https://redirect.github.com/apache/arrow-rs/pull/10502)
- chore(avro): add link to ticket in object\_store deprecation message
by [@&#8203;alamb](https://redirect.github.com/alamb) in
[#&#8203;10503](https://redirect.github.com/apache/arrow-rs/pull/10503)
- docs: clarify decimal negative scale behavior by
[@&#8203;ByteBaker](https://redirect.github.com/ByteBaker) in
[#&#8203;10304](https://redirect.github.com/apache/arrow-rs/pull/10304)
- Docs: fix mutableArrayData comments by
[@&#8203;Rich-T-kid](https://redirect.github.com/Rich-T-kid) in
[#&#8203;10326](https://redirect.github.com/apache/arrow-rs/pull/10326)
- docs: trim release schedule for released versions by
[@&#8203;alamb](https://redirect.github.com/alamb) in
[#&#8203;10280](https://redirect.github.com/apache/arrow-rs/pull/10280)
- Align parquet-geospatial crate docs with README by
[@&#8203;paleolimbot](https://redirect.github.com/paleolimbot) in
[#&#8203;10302](https://redirect.github.com/apache/arrow-rs/pull/10302)

##### Miscellaneous

- Revert "chore: formalize the default map field names to match default
arrow spec
([#&#8203;10297](https://redirect.github.com/apache/arrow-rs/issues/10297))"
by [@&#8203;alamb](https://redirect.github.com/alamb) in
[#&#8203;10506](https://redirect.github.com/apache/arrow-rs/pull/10506)
- parquet: deprecate explicit `object_store` integration by
[@&#8203;brancz](https://redirect.github.com/brancz) in
[#&#8203;10354](https://redirect.github.com/apache/arrow-rs/pull/10354)
- fix(parquet): restore opaque return type for `RowSelection::iter` by
[@&#8203;haohuaijin](https://redirect.github.com/haohuaijin) in
[#&#8203;10450](https://redirect.github.com/apache/arrow-rs/pull/10450)
- refactor(parquet): split `arrow_reader/selection` into smaller modules
by [@&#8203;haohuaijin](https://redirect.github.com/haohuaijin) in
[#&#8203;10434](https://redirect.github.com/apache/arrow-rs/pull/10434)
- chore: deduplicate filter nulls code in coalesce/filter kernel by
[@&#8203;Jefffrey](https://redirect.github.com/Jefffrey) in
[#&#8203;10348](https://redirect.github.com/apache/arrow-rs/pull/10348)
- chore: remove parquet dependency from parquet\_derive by
[@&#8203;ByteBaker](https://redirect.github.com/ByteBaker) in
[#&#8203;10327](https://redirect.github.com/apache/arrow-rs/pull/10327)

###
[`v59.1.0`](https://redirect.github.com/apache/arrow-rs/blob/HEAD/CHANGELOG.md#5910-2026-07-03)

[Compare
Source](https://redirect.github.com/apache/arrow-rs/compare/59.0.0...59.1.0)

[Full
Changelog](https://redirect.github.com/apache/arrow-rs/compare/59.0.0...59.1.0)

**Implemented enhancements:**

- Fast path for nested `DictionaryArray` casting
[#&#8203;10247](https://redirect.github.com/apache/arrow-rs/issues/10247)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- parquet/arrow: reading multiple nested columns fails with "Not all
children array length are the same!" when a list continues across
DataPageV2 page boundary
[#&#8203;10243](https://redirect.github.com/apache/arrow-rs/issues/10243)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
- Add product aggregate kernel to arrow-rs
[#&#8203;10150](https://redirect.github.com/apache/arrow-rs/issues/10150)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- Stricter `DataType` parsing
[#&#8203;10146](https://redirect.github.com/apache/arrow-rs/issues/10146)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- Support validating CSV headers against Schema
[#&#8203;10143](https://redirect.github.com/apache/arrow-rs/issues/10143)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- arrow-ipc: Supports compression level configuration for arrow-ipc
writer
[#&#8203;10132](https://redirect.github.com/apache/arrow-rs/issues/10132)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- \[Variant] `VariantArray` field API naming
[#&#8203;10093](https://redirect.github.com/apache/arrow-rs/issues/10093)
- Add `StructArray::field_` APIs symmetric to `StructArray::column_`
ones
[#&#8203;10092](https://redirect.github.com/apache/arrow-rs/issues/10092)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- arrow-buffer: implement Saturating, CheckedShl, Not num-traits for
i256
[#&#8203;10087](https://redirect.github.com/apache/arrow-rs/issues/10087)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- feat: native concat for `MapArray`
[#&#8203;10047](https://redirect.github.com/apache/arrow-rs/issues/10047)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- \[Variant] Add `variant_to_arrow` `Dictionary/REE` type support
[#&#8203;10013](https://redirect.github.com/apache/arrow-rs/issues/10013)

**Fixed bugs:**

- arrow-row on fixed size binary/list with size 0 and no nulls return
wrong length
[#&#8203;10270](https://redirect.github.com/apache/arrow-rs/issues/10270)
- casting list to 0-size fixedsizelist can cause incorrect output length
[#&#8203;10227](https://redirect.github.com/apache/arrow-rs/issues/10227)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- Buffer count mismatched with metadata when encoding records with
dictionary of dictionaries
[#&#8203;10213](https://redirect.github.com/apache/arrow-rs/issues/10213)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- `Buffer::into_mutable` is not consistent regarding sliced data and can
lead to panics
[#&#8203;10117](https://redirect.github.com/apache/arrow-rs/issues/10117)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- parquet\_derive: cannot read or write columns whose name is a Rust
keyword (raw identifiers like r#type become column "r#type")
[#&#8203;10112](https://redirect.github.com/apache/arrow-rs/issues/10112)
- parquet: fix OffsetBuffer panic on corrupt input
[#&#8203;10107](https://redirect.github.com/apache/arrow-rs/issues/10107)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
- Parquet geospatial conversion uses metadata key "algorithm" instead of
"edges" in geoarrow metadata
[#&#8203;9929](https://redirect.github.com/apache/arrow-rs/issues/9929)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]

**Documentation updates:**

- doc: More comments to `concat_batches`
[#&#8203;10178](https://redirect.github.com/apache/arrow-rs/pull/10178)
([2010YOUY01](https://redirect.github.com/2010YOUY01))
- Minor: improve PageStore docs with a temp-file spilling example
[#&#8203;10074](https://redirect.github.com/apache/arrow-rs/pull/10074)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([alamb](https://redirect.github.com/alamb))

**Performance improvements:**

- perf: interleave\_list for List\<Primitive> could be optimized?
[#&#8203;10022](https://redirect.github.com/apache/arrow-rs/issues/10022)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- perf(arrow-ord): Avoid full index materialization for small-limit
lexsorts
[#&#8203;9990](https://redirect.github.com/apache/arrow-rs/issues/9990)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- Replace conversion of binary->string in arrow-row from arraydata to
direct construction
[#&#8203;10261](https://redirect.github.com/apache/arrow-rs/pull/10261)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([Jefffrey](https://redirect.github.com/Jefffrey))
- \[arrow-flight encode path]re-use flatbufferbuilder
[#&#8203;10220](https://redirect.github.com/apache/arrow-rs/pull/10220)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([Rich-T-kid](https://redirect.github.com/Rich-T-kid))
- \[10125] arrow-flight decode path optimizations (add `skip_validation`
to arrow-flight)
[#&#8203;10206](https://redirect.github.com/apache/arrow-rs/pull/10206)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
\[[arrow-flight](https://redirect.github.com/apache/arrow-rs/labels/arrow-flight)]
([Rich-T-kid](https://redirect.github.com/Rich-T-kid))
- Improve performance of `concat_elements` ByteViewArray concatenation
[#&#8203;10161](https://redirect.github.com/apache/arrow-rs/pull/10161)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([pepijnve](https://redirect.github.com/pepijnve))
- \[arrow-flight] Optimize flight, remove some allocations, add
dictionary focused benchmarks
[#&#8203;10126](https://redirect.github.com/apache/arrow-rs/pull/10126)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
\[[arrow-flight](https://redirect.github.com/apache/arrow-rs/labels/arrow-flight)]
([Rich-T-kid](https://redirect.github.com/Rich-T-kid))
- optimize(concat): concat map implementation
[#&#8203;10048](https://redirect.github.com/apache/arrow-rs/pull/10048)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([mapleFU](https://redirect.github.com/mapleFU))
- Reduce copies in Arrow IPC writer
[#&#8203;10044](https://redirect.github.com/apache/arrow-rs/pull/10044)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([Rich-T-kid](https://redirect.github.com/Rich-T-kid))
- perf(interleave): Optimize list interleave\_list when child is
primitive
[#&#8203;10025](https://redirect.github.com/apache/arrow-rs/pull/10025)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([mapleFU](https://redirect.github.com/mapleFU))

**Closed issues:**

- Soundness: Unsound alignment contract in public `FromBytes` trait and
`BitReader::get_batch`
[#&#8203;10164](https://redirect.github.com/apache/arrow-rs/issues/10164)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
- ParquetPushDecoder: expose the next row-group index that
try\_next\_reader will yield
[#&#8203;10148](https://redirect.github.com/apache/arrow-rs/issues/10148)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
- arrow-ipc: Extend writer benchmarks to include dictionaries
[#&#8203;10119](https://redirect.github.com/apache/arrow-rs/issues/10119)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- bench(parquet): benchmark for nested list write
[#&#8203;10083](https://redirect.github.com/apache/arrow-rs/issues/10083)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
- Support i256 implement From\<i128>
[#&#8203;10080](https://redirect.github.com/apache/arrow-rs/issues/10080)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]

**Merged pull requests:**

- fix(arrow-row): allow to convert non empty fixed size binary/list
array with size length 0 and no nulls
[#&#8203;10271](https://redirect.github.com/apache/arrow-rs/pull/10271)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([rluvaton](https://redirect.github.com/rluvaton))
- chore: Fix audit CI run by ignore quick-xml audit advisories
[#&#8203;10267](https://redirect.github.com/apache/arrow-rs/pull/10267)
([alamb](https://redirect.github.com/alamb))
- fix main: parquet test compilation failure
[#&#8203;10266](https://redirect.github.com/apache/arrow-rs/pull/10266)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([Jefffrey](https://redirect.github.com/Jefffrey))
- minor: drive-by refactors for dicts in substring & filter
[#&#8203;10264](https://redirect.github.com/apache/arrow-rs/pull/10264)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([Jefffrey](https://redirect.github.com/Jefffrey))
- Add validated row decode benchmark
[#&#8203;10259](https://redirect.github.com/apache/arrow-rs/pull/10259)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([alamb](https://redirect.github.com/alamb))
- arrow-cast: Add optimized path for unnesting a dict
[#&#8203;10248](https://redirect.github.com/apache/arrow-rs/pull/10248)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([brancz](https://redirect.github.com/brancz))
- feat: support uuid from fixed type of length 16
[#&#8203;10241](https://redirect.github.com/apache/arrow-rs/pull/10241)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([ariel-miculas](https://redirect.github.com/ariel-miculas))
- chore(deps): bump actions/cache from 6.0.0 to 6.1.0
[#&#8203;10240](https://redirect.github.com/apache/arrow-rs/pull/10240)
([dependabot\[bot\]](https://redirect.github.com/apps/dependabot))
- fix: Rename parquet feature flag 'flate2-rust\_backened' to
'flate2-rust\_backend'
[#&#8203;10239](https://redirect.github.com/apache/arrow-rs/pull/10239)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([dannycjones](https://redirect.github.com/dannycjones))
- chore: Make clippy::question\_mark happy
[#&#8203;10231](https://redirect.github.com/apache/arrow-rs/pull/10231)
([Tpt](https://redirect.github.com/Tpt))
- fix(ipc): reject dictionary-encoded dictionary values
[#&#8203;10230](https://redirect.github.com/apache/arrow-rs/pull/10230)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([goutamadwant](https://redirect.github.com/goutamadwant))
- Replace `ArrayData` with direct `Array` construction in `arrow-row`
[#&#8203;10229](https://redirect.github.com/apache/arrow-rs/pull/10229)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([Jefffrey](https://redirect.github.com/Jefffrey))
- fix: casting list to fixedsizelist didn't respect input length
[#&#8203;10228](https://redirect.github.com/apache/arrow-rs/pull/10228)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([Jefffrey](https://redirect.github.com/Jefffrey))
- chore: Fix clippy::byte\_char\_slices (use byte strings instead of
explicit arrays)
[#&#8203;10225](https://redirect.github.com/apache/arrow-rs/pull/10225)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([Tpt](https://redirect.github.com/Tpt))
- nit: arrow-pyarrow: Use string interning
[#&#8203;10224](https://redirect.github.com/apache/arrow-rs/pull/10224)
([Tpt](https://redirect.github.com/Tpt))
- Support concatenation of mixed FixedSizeBinary via
`concat_elements_dyn`
[#&#8203;10222](https://redirect.github.com/apache/arrow-rs/pull/10222)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([pepijnve](https://redirect.github.com/pepijnve))
- rename Compression struct
[#&#8203;10221](https://redirect.github.com/apache/arrow-rs/pull/10221)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
\[[arrow-flight](https://redirect.github.com/apache/arrow-rs/labels/arrow-flight)]
([Rich-T-kid](https://redirect.github.com/Rich-T-kid))
- chore(deps): bump the all-other-cargo-deps group across 1 directory
with 16 updates
[#&#8203;10218](https://redirect.github.com/apache/arrow-rs/pull/10218)
([dependabot\[bot\]](https://redirect.github.com/apps/dependabot))
- chore(deps): bump actions/setup-python from 6.2.0 to 6.3.0
[#&#8203;10210](https://redirect.github.com/apache/arrow-rs/pull/10210)
([dependabot\[bot\]](https://redirect.github.com/apps/dependabot))
- \[10125] Introduce mult-batch decode benchmarks
[#&#8203;10207](https://redirect.github.com/apache/arrow-rs/pull/10207)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
\[[arrow-flight](https://redirect.github.com/apache/arrow-rs/labels/arrow-flight)]
([Rich-T-kid](https://redirect.github.com/Rich-T-kid))
- chore(deps): bump actions/cache from 5.0.5 to 6.0.0
[#&#8203;10203](https://redirect.github.com/apache/arrow-rs/pull/10203)
([dependabot\[bot\]](https://redirect.github.com/apps/dependabot))
- introduce decode benchmarks
[#&#8203;10202](https://redirect.github.com/apache/arrow-rs/pull/10202)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
\[[arrow-flight](https://redirect.github.com/apache/arrow-rs/labels/arrow-flight)]
([Rich-T-kid](https://redirect.github.com/Rich-T-kid))
- Fix `merge_kernels` benchmark panic due to not wrapping with `Scalar`
[#&#8203;10199](https://redirect.github.com/apache/arrow-rs/pull/10199)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([Jefffrey](https://redirect.github.com/Jefffrey))
- Benchmarks and performance improvement for parquet boolean reader
[#&#8203;10196](https://redirect.github.com/apache/arrow-rs/pull/10196)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([jhorstmann](https://redirect.github.com/jhorstmann))
- add stale PR workflow
[#&#8203;10194](https://redirect.github.com/apache/arrow-rs/pull/10194)
([Jefffrey](https://redirect.github.com/Jefffrey))
- chore: group minor/patch dependabot updates
[#&#8203;10193](https://redirect.github.com/apache/arrow-rs/pull/10193)
([Jefffrey](https://redirect.github.com/Jefffrey))
- chore(deps): bump http from 1.4.0 to 1.4.2
[#&#8203;10191](https://redirect.github.com/apache/arrow-rs/pull/10191)
([dependabot\[bot\]](https://redirect.github.com/apps/dependabot))
- chore(deps): bump syn from 2.0.117 to 2.0.118
[#&#8203;10190](https://redirect.github.com/apache/arrow-rs/pull/10190)
([dependabot\[bot\]](https://redirect.github.com/apps/dependabot))
- chore(deps): bump chrono from 0.4.44 to 0.4.45
[#&#8203;10188](https://redirect.github.com/apache/arrow-rs/pull/10188)
([dependabot\[bot\]](https://redirect.github.com/apps/dependabot))
- chore(deps): bump uuid from 1.23.1 to 1.23.3
[#&#8203;10186](https://redirect.github.com/apache/arrow-rs/pull/10186)
([dependabot\[bot\]](https://redirect.github.com/apps/dependabot))
- chore: run `cargo update` to bump quinn
[#&#8203;10181](https://redirect.github.com/apache/arrow-rs/pull/10181)
([Jefffrey](https://redirect.github.com/Jefffrey))
- test: cover signed integers and bool in BitReader::get\_batch test
[#&#8203;10180](https://redirect.github.com/apache/arrow-rs/pull/10180)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([alamb](https://redirect.github.com/alamb))
- \[arrow-select] perf: Replace `ArrayData` with direct `Array`
construction in take kernels
[#&#8203;10176](https://redirect.github.com/apache/arrow-rs/pull/10176)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([liamzwbao](https://redirect.github.com/liamzwbao))
- Return PyValueError for nullable PyArrow struct imports
[#&#8203;10174](https://redirect.github.com/apache/arrow-rs/pull/10174)
([fallintoplace](https://redirect.github.com/fallintoplace))
- Fix Variant time microsecond JSON formatting
[#&#8203;10173](https://redirect.github.com/apache/arrow-rs/pull/10173)
([fallintoplace](https://redirect.github.com/fallintoplace))
- Split traits for plain and bitpacked decoding and fix soundness issue
in BitReader::get\_batch
[#&#8203;10172](https://redirect.github.com/apache/arrow-rs/pull/10172)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([jhorstmann](https://redirect.github.com/jhorstmann))
- fix: switch generic usages of `i128` to `IntervalMonthDayNano` for
MonthDayNano type
[#&#8203;10171](https://redirect.github.com/apache/arrow-rs/pull/10171)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([Jefffrey](https://redirect.github.com/Jefffrey))
- chore: specify `--locked` when cargo installing `cargo-audit`
[#&#8203;10170](https://redirect.github.com/apache/arrow-rs/pull/10170)
([Jefffrey](https://redirect.github.com/Jefffrey))
- chore: Fix clippy::useless\_borrows\_in\_formatting
[#&#8203;10163](https://redirect.github.com/apache/arrow-rs/pull/10163)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([Tpt](https://redirect.github.com/Tpt))
- fix(arrow-cast): respect cast safety for overflowing temporal casts
[#&#8203;10162](https://redirect.github.com/apache/arrow-rs/pull/10162)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([SAY-5](https://redirect.github.com/SAY-5))
- chore(deps): bump actions/checkout from 6 to 7
[#&#8203;10159](https://redirect.github.com/apache/arrow-rs/pull/10159)
([dependabot\[bot\]](https://redirect.github.com/apps/dependabot))
- feat(parquet): add ParquetPushDecoder::peek\_next\_row\_group()
[#&#8203;10158](https://redirect.github.com/apache/arrow-rs/pull/10158)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([zhuqi-lucas](https://redirect.github.com/zhuqi-lucas))
- feat(pyarrow) `FromPyArrow` on `Vec<T>`: allow any iterable for input
[#&#8203;10155](https://redirect.github.com/apache/arrow-rs/pull/10155)
([Tpt](https://redirect.github.com/Tpt))
- nit: pyarrow: simplify class validation error creation
[#&#8203;10154](https://redirect.github.com/apache/arrow-rs/pull/10154)
([Tpt](https://redirect.github.com/Tpt))
- \[Variant] add doc reference to `VariantArrayBuilder`
[#&#8203;10152](https://redirect.github.com/apache/arrow-rs/pull/10152)
([sdf-jkl](https://redirect.github.com/sdf-jkl))
- feat: Adds product aggregate compute kernel
[#&#8203;10151](https://redirect.github.com/apache/arrow-rs/pull/10151)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([devanbenz](https://redirect.github.com/devanbenz))
- Stricter datatype parsing for decimals, fixedsizelists and time32/64
[#&#8203;10147](https://redirect.github.com/apache/arrow-rs/pull/10147)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([Jefffrey](https://redirect.github.com/Jefffrey))
- feat(arrow\_csv): add header validation option
[#&#8203;10144](https://redirect.github.com/apache/arrow-rs/pull/10144)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([XiNiHa](https://redirect.github.com/XiNiHa))
- \[Parquet] route dictionary page through the PageStore
[#&#8203;10142](https://redirect.github.com/apache/arrow-rs/pull/10142)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([liamzwbao](https://redirect.github.com/liamzwbao))
- chore: update pyo3 dependency to 0.29
[#&#8203;10134](https://redirect.github.com/apache/arrow-rs/pull/10134)
([timsaucer](https://redirect.github.com/timsaucer))
- feat(ipc): Supports compression level configuration
[#&#8203;10133](https://redirect.github.com/apache/arrow-rs/pull/10133)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([wForget](https://redirect.github.com/wForget))
- fix: write error for dbg output of out of range timestamps
[#&#8203;10130](https://redirect.github.com/apache/arrow-rs/pull/10130)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([Jefffrey](https://redirect.github.com/Jefffrey))
- \[Variant] `VariantArray` field API naming
[#&#8203;10124](https://redirect.github.com/apache/arrow-rs/pull/10124)
([sdf-jkl](https://redirect.github.com/sdf-jkl))
- feat(arrow\_array): add helper function to create MapArray from
`Vec<Option<Vec<(Key, Option<Value>)>>>` for tests
[#&#8203;10123](https://redirect.github.com/apache/arrow-rs/pull/10123)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
\[[arrow-flight](https://redirect.github.com/apache/arrow-rs/labels/arrow-flight)]
([rluvaton](https://redirect.github.com/rluvaton))
- perf(arrow-ipc): Add writer benchmarks for dictionaries
[#&#8203;10122](https://redirect.github.com/apache/arrow-rs/pull/10122)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([JakeDern](https://redirect.github.com/JakeDern))
- feat: support `MapArray` in lengths kernel
[#&#8203;10121](https://redirect.github.com/apache/arrow-rs/pull/10121)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([rluvaton](https://redirect.github.com/rluvaton))
- feat: add `OffsetBuffer::subtract` to allow to shift offsets by value
[#&#8203;10120](https://redirect.github.com/apache/arrow-rs/pull/10120)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([rluvaton](https://redirect.github.com/rluvaton))
- fix: `Buffer::into_mutable` return error instead of panic for
converting owned sliced when not start at 0 and fix returned Mutable
length
[#&#8203;10118](https://redirect.github.com/apache/arrow-rs/pull/10118)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([rluvaton](https://redirect.github.com/rluvaton))
- chore: update `Bytes` visibility to correctly reflect the actual
visibility
[#&#8203;10115](https://redirect.github.com/apache/arrow-rs/pull/10115)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([rluvaton](https://redirect.github.com/rluvaton))
- fix(parquet\_derive): support raw identifiers as column names
[#&#8203;10113](https://redirect.github.com/apache/arrow-rs/pull/10113)
([cbmixx](https://redirect.github.com/cbmixx))
- removed clippy ignore statment
[#&#8203;10111](https://redirect.github.com/apache/arrow-rs/pull/10111)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([Rich-T-kid](https://redirect.github.com/Rich-T-kid))
- Add `StructArray::field_` APIs symmetric to `StructArray::column_`
ones
[#&#8203;10110](https://redirect.github.com/apache/arrow-rs/pull/10110)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([sdf-jkl](https://redirect.github.com/sdf-jkl))
- fix(parquet): return error instead of panicking in pad\_nulls on
corrupt input
[#&#8203;10108](https://redirect.github.com/apache/arrow-rs/pull/10108)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([thepenguinco](https://redirect.github.com/thepenguinco))
- Minor: Add interleave tests for List\<Decimal128> and
List\<Timestamp(tz)>
[#&#8203;10099](https://redirect.github.com/apache/arrow-rs/pull/10099)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([alamb](https://redirect.github.com/alamb))
- Add arrow-flight test coverage for IPC compression
[#&#8203;10097](https://redirect.github.com/apache/arrow-rs/pull/10097)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
\[[arrow-flight](https://redirect.github.com/apache/arrow-rs/labels/arrow-flight)]
([alamb](https://redirect.github.com/alamb))
- chore(deps): bump pyspark from 3.3.2 to 3.4.4 in /parquet/pytest
[#&#8203;10091](https://redirect.github.com/apache/arrow-rs/pull/10091)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([dependabot\[bot\]](https://redirect.github.com/apps/dependabot))
- refactor(parquet): bundle array reader recursion args into
`ReaderArgs`
[#&#8203;10089](https://redirect.github.com/apache/arrow-rs/pull/10089)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([HippoBaro](https://redirect.github.com/HippoBaro))
- arrow-buffer: implement Saturating, Checked num-traits for i256
[#&#8203;10088](https://redirect.github.com/apache/arrow-rs/pull/10088)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([theirix](https://redirect.github.com/theirix))
- bench(parquet): add nested list writer benchmarks
[#&#8203;10084](https://redirect.github.com/apache/arrow-rs/pull/10084)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([mapleFU](https://redirect.github.com/mapleFU))
- Implement From\<i128> for i256
[#&#8203;10081](https://redirect.github.com/apache/arrow-rs/pull/10081)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([AdamGS](https://redirect.github.com/AdamGS))
- test(parquet): drop confusing `main` reference in page-roundtrip test
comment
[#&#8203;10072](https://redirect.github.com/apache/arrow-rs/pull/10072)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([adriangb](https://redirect.github.com/adriangb))
- ci: Split miri tests into 4 parallel shards
[#&#8203;10067](https://redirect.github.com/apache/arrow-rs/pull/10067)
([AdamGS](https://redirect.github.com/AdamGS))
- Add tests and fix corner cases for Parquet/GeoArrow extension type
conversion
[#&#8203;10065](https://redirect.github.com/apache/arrow-rs/pull/10065)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([paleolimbot](https://redirect.github.com/paleolimbot))
- Support writing REE arrays directly to Parquet
[#&#8203;10064](https://redirect.github.com/apache/arrow-rs/pull/10064)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([Rich-T-kid](https://redirect.github.com/Rich-T-kid))
- test(arrow-select): additional tests for inline-view filter fast path
(tests for
[#&#8203;9755](https://redirect.github.com/apache/arrow-rs/issues/9755))
[#&#8203;10054](https://redirect.github.com/apache/arrow-rs/pull/10054)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([alamb](https://redirect.github.com/alamb))
- test(arrow-select): add take\_bytes coverage for sliced values and
nullable offset overflow
[#&#8203;10053](https://redirect.github.com/apache/arrow-rs/pull/10053)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([alamb](https://redirect.github.com/alamb))
- Consolidate `filter_null_mask` into `FilterPredicate::filter_nulls`
[#&#8203;10049](https://redirect.github.com/apache/arrow-rs/pull/10049)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([alamb](https://redirect.github.com/alamb))
- \[Variant] Add `VariantBuilder` values check
[#&#8203;10016](https://redirect.github.com/apache/arrow-rs/pull/10016)
([sdf-jkl](https://redirect.github.com/sdf-jkl))
- \[Variant] Preserve `UUID` extension type metadata for Parquet writer
[#&#8203;10015](https://redirect.github.com/apache/arrow-rs/pull/10015)
([sdf-jkl](https://redirect.github.com/sdf-jkl))
- feat(parquet-variant): add Dictionary and REE variant\_to\_arrow
support
[#&#8203;10014](https://redirect.github.com/apache/arrow-rs/pull/10014)
([mneetika](https://redirect.github.com/mneetika))
- perf(arrow-ord): Avoid full index materialization for small-limit
lexsorts
[#&#8203;9991](https://redirect.github.com/apache/arrow-rs/pull/9991)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([pchintar](https://redirect.github.com/pchintar))

\* *This Changelog was automatically generated by
[github\_changelog\_generator](https://redirect.github.com/github-changelog-generator/github-changelog-generator)*

###
[`v59.0.0`](https://redirect.github.com/apache/arrow-rs/releases/tag/59.0.0):
arrow 59.0.0

[Compare
Source](https://redirect.github.com/apache/arrow-rs/compare/58.4.0...59.0.0)

<!---
  Licensed to the Apache Software Foundation (ASF) under one
  or more contributor license agreements.  See the NOTICE file
  distributed with this work for additional information
  regarding copyright ownership.  The ASF licenses this file
  to you under the Apache License, Version 2.0 (the
  "License"); you may not use this file except in compliance
  with the License.  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing,
  software distributed under the License is distributed on an
  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
  KIND, either express or implied.  See the License for the
  specific language governing permissions and limitations
  under the License.
-->

##### Changelog

##### [59.0.0](https://redirect.github.com/apache/arrow-rs/tree/59.0.0)
(2026-06-04)

[Full
Changelog](https://redirect.github.com/apache/arrow-rs/compare/57.3.1...59.0.0)

**Breaking changes:**

- chore: Remove some deprecated Arrow functions from the public API
[#&#8203;10040](https://redirect.github.com/apache/arrow-rs/pull/10040)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([etseidl](https://redirect.github.com/etseidl))
- chore: Remove some deprecated functions from parquet crate
[#&#8203;10035](https://redirect.github.com/apache/arrow-rs/pull/10035)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([etseidl](https://redirect.github.com/etseidl))
- Replace `From<Vec<_>>` impls with `TryFrom`s for
`FixedSizeBinaryArray`
[#&#8203;10019](https://redirect.github.com/apache/arrow-rs/pull/10019)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([quantumish](https://redirect.github.com/quantumish))
- Use Thrift macro to generate Parquet `LogicalType` serialization code
[#&#8203;9997](https://redirect.github.com/apache/arrow-rs/pull/9997)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([etseidl](https://redirect.github.com/etseidl))
- refactor: make `BloomFilterProperties` fpp/ndv private with accessors
[#&#8203;9969](https://redirect.github.com/apache/arrow-rs/pull/9969)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([CuteChuanChuan](https://redirect.github.com/CuteChuanChuan))
- Remove deprecated parquet::format module and thrift dependency
[#&#8203;9962](https://redirect.github.com/apache/arrow-rs/pull/9962)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([jhorstmann](https://redirect.github.com/jhorstmann))
- generic channel support for FlightClient
[#&#8203;9933](https://redirect.github.com/apache/arrow-rs/pull/9933)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
\[[arrow-flight](https://redirect.github.com/apache/arrow-rs/labels/arrow-flight)]
([rumenov](https://redirect.github.com/rumenov))
- Add `CompressionCodec` Thrift enum for Parquet metadata
[#&#8203;9864](https://redirect.github.com/apache/arrow-rs/pull/9864)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([etseidl](https://redirect.github.com/etseidl))
- \[Variant] remove `BorrowedShreddingState`
[#&#8203;9791](https://redirect.github.com/apache/arrow-rs/pull/9791)
([sdf-jkl](https://redirect.github.com/sdf-jkl))
- Remove deprecated legacy `like` kernels in `arrow-string`
[#&#8203;9674](https://redirect.github.com/apache/arrow-rs/pull/9674)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([AdamGS](https://redirect.github.com/AdamGS))

**Implemented enhancements:**

- Allow casting plain struct to dictionary encoded struct
[#&#8203;10038](https://redirect.github.com/apache/arrow-rs/issues/10038)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- Optimize arrow-flight
[#&#8203;10029](https://redirect.github.com/apache/arrow-rs/issues/10029)
- Align buffers when importing via `from_ffi` / `ArrowArrayStreamReader`
[#&#8203;10028](https://redirect.github.com/apache/arrow-rs/issues/10028)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- Switch Parquet `LogicalType` enum to macro generated version
[#&#8203;9995](https://redirect.github.com/apache/arrow-rs/issues/9995)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
- Future proof Parquet Thrift parser
[#&#8203;9973](https://redirect.github.com/apache/arrow-rs/issues/9973)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
- Add `DatePart` 1-indexed variants
[#&#8203;9964](https://redirect.github.com/apache/arrow-rs/issues/9964)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- perf: Rework Parquet Thrift handling of boolean fields
[#&#8203;9946](https://redirect.github.com/apache/arrow-rs/issues/9946)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
- Add benchmarks for REE to parquet
[#&#8203;9935](https://redirect.github.com/apache/arrow-rs/issues/9935)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- (re) Allow Large `FixedSizeBinaryArray`s
[#&#8203;9906](https://redirect.github.com/apache/arrow-rs/issues/9906)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- Add a is\_normalized flag to DictionaryArray
[#&#8203;9841](https://redirect.github.com/apache/arrow-rs/issues/9841)
- \[Variant] Remove `BorrowedShreddingState`
[#&#8203;9790](https://redirect.github.com/apache/arrow-rs/issues/9790)
- \[parquet] Expose whether FileDecryptionProperties uses a KeyRetriever
[#&#8203;9721](https://redirect.github.com/apache/arrow-rs/issues/9721)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
- Align cast logic for from/to\_decimal for variant to cast kernel
[#&#8203;9688](https://redirect.github.com/apache/arrow-rs/issues/9688)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]

**Fixed bugs:**

- parquet-variant build might fail on s390x
[#&#8203;10026](https://redirect.github.com/apache/arrow-rs/issues/10026)
- `FixedSizeBinaryArray` implements `From<Vec<&[u8]>>` etc despite
conversion being fallible
[#&#8203;10018](https://redirect.github.com/apache/arrow-rs/issues/10018)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- string -> decimal cast should not treat empty string as 0
[#&#8203;10009](https://redirect.github.com/apache/arrow-rs/issues/10009)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- Cast FixedSizeList to List will lost datatype metadata in list
[#&#8203;10004](https://redirect.github.com/apache/arrow-rs/issues/10004)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- Record reader panics with "index out of bounds" when row group
num\_rows exceeds actual column data
[#&#8203;9992](https://redirect.github.com/apache/arrow-rs/issues/9992)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
- parquet predicate-cache: panic / silent row drop on single-leaf
nullable struct
[#&#8203;9982](https://redirect.github.com/apache/arrow-rs/issues/9982)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
- parquet-variant doesn't build on 32-bit targets
[#&#8203;9977](https://redirect.github.com/apache/arrow-rs/issues/9977)
- Date32 doesn't parse date with large year
[#&#8203;9960](https://redirect.github.com/apache/arrow-rs/issues/9960)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- msrv check failing on main due to `tonic@0.14.6`
[#&#8203;9938](https://redirect.github.com/apache/arrow-rs/issues/9938)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
\[[arrow-flight](https://redirect.github.com/apache/arrow-rs/labels/arrow-flight)]

**Documentation updates:**

- Release arrow-rs / parquet Minor/Patch version `58.3.0` or `58.2.1`
(May 2026)
[#&#8203;9859](https://redirect.github.com/apache/arrow-rs/issues/9859)
- Add docs for `BitWriter`
[#&#8203;9949](https://redirect.github.com/apache/arrow-rs/pull/9949)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([alamb](https://redirect.github.com/alamb))
- Add docs for `BitReader`
[#&#8203;9948](https://redirect.github.com/apache/arrow-rs/pull/9948)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([alamb](https://redirect.github.com/alamb))

**Performance improvements:**

- perf: parquet LevelInfoBuilder::write\_list can be optimized?
[#&#8203;10023](https://redirect.github.com/apache/arrow-rs/issues/10023)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
- perf(parquet): LevelInfoBuilder batch write when no repetition childs
[#&#8203;10037](https://redirect.github.com/apache/arrow-rs/pull/10037)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([mapleFU](https://redirect.github.com/mapleFU))
- \[arrow-select] Replace `ArrayData` with direct `Array` construction
in filter kernels
[#&#8203;9986](https://redirect.github.com/apache/arrow-rs/pull/9986)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([liamzwbao](https://redirect.github.com/liamzwbao))
- Bulk-fill definition levels for majority-null leaf columns
[#&#8203;9967](https://redirect.github.com/apache/arrow-rs/pull/9967)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([RyanJamesStewart](https://redirect.github.com/RyanJamesStewart))
- perf: Remove `bool_val` from Parquet Thrift `FieldIdentifier`
[#&#8203;9945](https://redirect.github.com/apache/arrow-rs/pull/9945)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([etseidl](https://redirect.github.com/etseidl))
- feat(parquet): compact level representation with generic writer
dispatch
[#&#8203;9831](https://redirect.github.com/apache/arrow-rs/pull/9831)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([HippoBaro](https://redirect.github.com/HippoBaro))

**Closed issues:**

- Bound ArrowWriter peak memory
[#&#8203;10071](https://redirect.github.com/apache/arrow-rs/issues/10071)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
- Parquet writer can produce massively oversized data pages for large
variable-width values
[#&#8203;10061](https://redirect.github.com/apache/arrow-rs/issues/10061)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
- Remove the `fused_inline_view_columns` field from `BatchCoalescer` if
possible
[#&#8203;10055](https://redirect.github.com/apache/arrow-rs/issues/10055)
- DataType parser permits negative FixedSizeBinary size
[#&#8203;10033](https://redirect.github.com/apache/arrow-rs/issues/10033)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- Parquet: return error for overlong INT96 column metadata statistics
[#&#8203;10002](https://redirect.github.com/apache/arrow-rs/issues/10002)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
- `Uuid` extension type fails to deserialize when
`ARROW:extension:metadata` is an empty string
[#&#8203;10000](https://redirect.github.com/apache/arrow-rs/issues/10000)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- parquet: timeline for removing thrift crate dependency
(CVE-2026-43868)
[#&#8203;9999](https://redirect.github.com/apache/arrow-rs/issues/9999)
- Failure in CI: `Archery test With other arrows` - `binary_view Rust
producing, .NET consuming`
[#&#8203;9989](https://redirect.github.com/apache/arrow-rs/issues/9989)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- Validate FIXED\_LEN\_BYTE\_ARRAY type\_length for DECIMAL and INTERVAL
in Parquet → Arrow schema conversion
[#&#8203;9984](https://redirect.github.com/apache/arrow-rs/issues/9984)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
- IPC reader projection does not handle duplicate projection indices
correctly
[#&#8203;9950](https://redirect.github.com/apache/arrow-rs/issues/9950)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- `AnyRunArray` trait
[#&#8203;9909](https://redirect.github.com/apache/arrow-rs/issues/9909)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- Release arrow-rs / parquet Patch version `57.3.1` (May 2026)
[#&#8203;9858](https://redirect.github.com/apache/arrow-rs/issues/9858)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- Release arrow-rs / parquet Patch version `56.2.1` (May 2026)
[#&#8203;9857](https://redirect.github.com/apache/arrow-rs/issues/9857)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- parquet/arrow: should sync/async readers converge on a shared physical
read planner
[#&#8203;9764](https://redirect.github.com/apache/arrow-rs/issues/9764)
- `arrow-string` has a lot of macro-generated deprecated kernels in
`like.rs`
[#&#8203;9675](https://redirect.github.com/apache/arrow-rs/issues/9675)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
- \[parquet] Add BloomFilterProperties builder API to make bloom filter
configuration explicit
[#&#8203;9667](https://redirect.github.com/apache/arrow-rs/issues/9667)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]

**Merged pull requests:**

- Bump max throughput in `flight` benchmark before blocking
[#&#8203;10070](https://redirect.github.com/apache/arrow-rs/pull/10070)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
\[[arrow-flight](https://redirect.github.com/apache/arrow-rs/labels/arrow-flight)]
([Rich-T-kid](https://redirect.github.com/Rich-T-kid))
- Add coalesce inline-view filter benchmarks
[#&#8203;10050](https://redirect.github.com/apache/arrow-rs/pull/10050)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([ClSlaid](https://redirect.github.com/ClSlaid))
- fix: better error handling for negative size of FixedSizeBinary
[#&#8203;10042](https://redirect.github.com/apache/arrow-rs/pull/10042)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([theirix](https://redirect.github.com/theirix))
- bench(parquet): add Sbbf check/insert benchmarks
[#&#8203;10041](https://redirect.github.com/apache/arrow-rs/pull/10041)
\[[parquet](https://redirect.github.com/apache/arrow-rs/labels/parquet)]
([dmatth1](https://redirect.github.com/dmatth1))
- arrow-cast: Add ability to cast plain struct to dictionary
[#&#8203;10039](https://redirect.github.com/apache/arrow-rs/pull/10039)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([brancz](https://redirect.github.com/brancz))
-
\[[#&#8203;10029](https://redirect.github.com/apache/arrow-rs/issues/10029)]\[benchmarks]
arrow-flight roundtrip as well as encode/decode
[#&#8203;10031](https://redirect.github.com/apache/arrow-rs/pull/10031)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
\[[arrow-flight](https://redirect.github.com/apache/arrow-rs/labels/arrow-flight)]
([Rich-T-kid](https://redirect.github.com/Rich-T-kid))
- Call `align_buffers()` in `from_ffi`, remove redundant call from
`arrow-pyarrow`
[#&#8203;10030](https://redirect.github.com/apache/arrow-rs/pull/10030)
\[[arrow](https://redirect.github.com/apache/arrow-rs/labels/arrow)]
([mbutrovich](https://redirect.github.com/mbutrovich))
- Adjust Variant size expectation for s390x architecture
[#&#8203;10027](https://redirect.github.com/apache/arrow-rs/pull/10027)
([frantisekz](https://redirect.github.com/frantisekz

> ✂ **Note**
> 
> PR body was truncated to here.


</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
- Between 12:00 AM and 03:59 AM, on the second Monday and fourth Monday
of the month (`* 0-3 * * 1#2,1#4`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get
[config
help](https://redirect.github.com/renovatebot/renovate/discussions) if
that's undesired.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/vortex-data/vortex).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xMi4wIiwidXBkYXRlZEluVmVyIjoiNDQuMTIuMCIsInRhcmdldEJyYW5jaCI6ImRldmVsb3AiLCJsYWJlbHMiOlsiY2hhbmdlbG9nL2Nob3JlIl19-->

---------

Signed-off-by: Robert Kruszewski <github@robertk.io>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Robert Kruszewski <github@robertk.io>
## Summary

- add shared tree traversal and indented/branch renderers in
`vortex-utils`
- migrate array and expression tree displays to the shared
implementation
- preserve existing display output while removing duplicated traversal
code

---------

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
…rks (vortex-data#9287)

## Rationale for this change

These steps are independent, so we can run them in parallel.

---------

Signed-off-by: Adam Gutglick <adam@spiraldb.com>
## Rationale for this change

`list_view_from_list` executed the lazy offsets that `reset_offsets`
produces inside `build_sizes_from_offsets`, dropped that result, and
then sliced the still-lazy handle into the view, so the subtraction
kernel ran a second time when a consumer canonicalized the offsets
child. Extracted from investigating vortex-data#9255, where the CodSpeed report
showed ~35% improvements on the `take_filter_list` benchmarks. This fix
alone recovers about half of that locally (12.6µs to 10.1µs median on
`take_filter_list_random_mask_sequential_indices[768, 50]`); the
remainder is the cost of building and executing the lazy subtraction
node at all.

## What changes are included in this PR?

`list_view_from_list` executes the offsets to a `PrimitiveArray` once at
the top, computes `sizes` from that array, and slices it zero-copy for
the view's offsets. `build_sizes_from_offsets` now takes the
materialized offsets instead of executing its own copy. Also adds a
regression test that list-to-listview conversion resets nonzero leading
offsets.

Signed-off-by: "Connor Tsui" <connor@spiraldb.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…ex-data#8586)

## Rationale for this change

Part of vortex-data#9329.

This is a step towards making the benchmark binaries smaller, shifting
as much of the functionality into the python tool

## What changes are included in this PR?

Removes data generation from the engine specific binaries, replacing it
with an informative error.

## What APIs are changed? Are there any user-facing changes?

None

---------

Signed-off-by: Adam Gutglick <adam@spiraldb.com>
Found by fuzzer in vortex-data#9297. Map type comparison can delegate to the
underlying list type

Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
Signed-off-by: Claude <noreply@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
…#8918)

We have a new interface that handles dynamically registering extensions,
the legacy methods shouldn't be expose as they don't properly delegate
…9291)

## Summary

- execute Interleave arrays whose values use primitive dtypes
- gather from canonical primitive buffers while preserving constant
value sources
- share selector length and bounds validation with the Boolean kernel
- cover nullable primitive columns, constants, null constants, and
invalid selectors

## Motivation

Interleave previously only had an execution kernel for Boolean values.
This adds the corresponding primitive kernel as a general vortex-array
capability. Spatial MakeLine uses this support in the separate stacked
PR vortex-data#9201.

---------

Signed-off-by: Nemo Yu <zyu379@wisc.edu>
## Rationale

Vortex native geometry arrays should support constructing two-point
LineStrings directly from paired Point columns without an Arrow or WKB
round trip.

## What changes are included?

- Add vortex.st.make_line for paired native Point arrays.
- Promote mixed XY, XYZ, XYM, and XYZM inputs, filling absent ordinates
with zero.
- Propagate CRS metadata, constants, and endpoint nulls.
- Build native LineString storage directly and add focused tests and
CodSpeed benchmarks.

## Stack

vortex-data#9291 (primitive Interleave execution) has merged into develop, so this
PR is now standalone: rebased onto develop, it contains only the spatial
function.

ST_Length was split into vortex-data#9290 so each scalar function can also be
reviewed independently.

Signed-off-by: Nemo Yu <zyu379@wisc.edu>
## Summary

- add a native `GeoArea` scalar function backed by
`geo::Area::unsigned_area`
- support every native geometry dtype, including zero-area point and
line families and rectangular boxes
- preserve input nullability and propagate null rows through unary
columnar execution
- give every geo scalar function a local operand validator that checks
exact arity before dtype validation
- parameterize repeated Area, Envelope, and MakeLine cases with `rstest`
for clearer per-dtype failures and broader operand-order coverage

## Semantics

This follows DuckDB `ST_Area` semantics over the native Vortex geometry
types: point and line families return zero, polygon holes are
subtracted, multipolygon component areas are summed, rectangles use
width times height, and higher coordinate dimensions are ignored. WKB
remains excluded because it is a storage representation rather than a
native decoded geometry.

---------

Signed-off-by: Nemo Yu <zyu379@wisc.edu>
## Rationale

Vortex native LineString arrays should support planar length without an
Arrow, WKB, or row-oriented geo_types conversion.

## What changes are included?

- Add vortex.st.length for native LineString inputs.
- Compute every row directly from flattened coordinate buffers and row
offsets.
- Sum all adjacent segments, supporting empty, one-point, and
arbitrary-length LineStrings.
- Preserve constant results and strict null propagation.
- Add focused correctness tests and CodSpeed benchmarks.

## Split

This was split from vortex-data#9201. It is standalone on develop; vortex-data#9201 now
contains only ST_MakeLine and primitive Interleave support.

Signed-off-by: Nemo Yu <zyu379@wisc.edu>
…ing hf:// (vortex-data#9273)

Remove special handling of object stores from vortex-jni and use
upstream logic from vortex-cloud

---------

Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
## Rationale for this change

Make sure we run the benchmarks with the exact dependencies we have
checked into the repo.

Signed-off-by: Adam Gutglick <adam@spiraldb.com>
This PR contains the following updates:

| Update | Change |
|---|---|
| lockFileMaintenance | All locks refreshed |

---

> [!WARNING]
> Some dependencies could not be looked up. Check the [Dependency
Dashboard](..vortex-data/issues/357) for more information.

🔧 This Pull Request updates lock files to use the latest dependency
versions.

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get
[config
help](https://redirect.github.com/renovatebot/renovate/discussions) if
that's undesired.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/vortex-data/vortex).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xMi4wIiwidXBkYXRlZEluVmVyIjoiNDQuMTIuMCIsInRhcmdldEJyYW5jaCI6ImRldmVsb3AiLCJsYWJlbHMiOlsiY2hhbmdlbG9nL2Nob3JlIl19-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
## Rationale for this change

~The upcoming Apache DataFusion release includes this requirement. This
version is 3 releases behind the latest (as of this writing 1.97.1, with
1.98 being released in about 10 days).~

According to [libs.rs](https://lib.rs/stats), 67% of crates.io requests
for the top 4000 crates use this version or newer.~

Started as an upgrade to 1.94, but realized that we'll need 1.95 for the
full DF 55 upgrade because of geoarrow.

---------

Signed-off-by: Adam Gutglick <adam@spiraldb.com>
…#9323)

## Rationale for this change

`resolveTransforms` turns Spark partition transforms into the directory
keys and column indices a writer uses, and had no tests. It is a pure
static function, so the behaviour is cheap to pin.

## What changes are included in this PR?

Test only. Covers the directory keys derived for identity,
`years`/`months`/`days`/`hours` and `bucket`; the type guards (temporal
transforms reject a non-temporal column, `hours` also rejects a date);
the bucket count and multi-column index resolution; and the rejections
for an unknown transform name, a missing `numBuckets` argument, a
zero-reference transform and a column outside the schema.

Also covers the Java serialization round trip of `ResolvedTransform` —
the reason resolution happens eagerly on the driver, since Spark's
`Transform` objects cannot cross to executors.

Verified non-vacuous: disabling the temporal type guard fails the guard
cases.

- `./gradlew :vortex-spark_2.13:test --tests
'…PartitionedVortexDataWriterTest'` — 11 pass
- same on `:vortex-spark_2.12:test` — 11 pass
- `spotlessCheck` (both variants) and `javadoc` — clean

## What APIs are changed? Are there any user-facing changes?

None.

## AI assistance

Prepared with agentic AI assistance. I read the resolution code and its
callers, confirmed each assertion against the current implementation,
and verified the guard tests fail when the guard is removed.

Signed-off-by: jackylee <qcsd2011@gmail.com>
…x-data#9325)

## Rationale for this change

`bytesWritten` accumulated `FieldVector.getBufferSize()` per batch — the
uncompressed in-memory Arrow size — and handed that to
`VortexWriterCommitMessage`, whose javadoc documents it as "the number
of bytes written". For a compressed file the two differ by the
compression ratio.

The real value was already available: `VortexWriter.finish()` returns a
summary carrying the file's physical size, and `close()` was only
calling `finish()` and discarding the result.

## What changes are included in this PR?

Take the size from `finish().fileSize()` and drop the per-batch
accumulation, which also removes work from the write path.

- `./gradlew :vortex-spark_2.13:test --tests
'…VortexDataSourceWriteTest' --tests '…VortexDataSourceBasicTest'` — 15
pass
- same on `:vortex-spark_2.12:test` — 15 pass
- `spotlessCheck` (both variants) and `javadoc` — clean

No new test: the value only becomes observable through a commit message
a driver collects, which the suite has no fixture for.

## What APIs are changed? Are there any user-facing changes?

None. `VortexWriterCommitMessage.bytesWritten()` now reports the file
size rather than the in-memory size, which is what its javadoc already
promised.

## AI assistance

Prepared with agentic AI assistance. I read the writer's finalization
path and `VortexWriter.finish()` to confirm the summary is produced
there and that `close()` discarded it.

Signed-off-by: jackylee <qcsd2011@gmail.com>
vortex-data#9332)

## Rationale for this change

The javadoc on `createTable` said Spark writes a `CREATE TABLE ... AS
SELECT` result into the table this overload returns. It does not.
`DelegatingCatalogExtension` sends the `Column[]` overload straight to
the delegate, and that is the overload both supported Spark versions
call — 3.5's `CreateTableAsSelectExec` directly, 4.1's via the
`TableInfo` default. This override never runs; CTAS gets its Vortex
table from the `loadTable` override instead.

Had it run it would have thrown: `V2SessionCatalog.createTable` returns
`null` on its normal path, and `asVortexTableIfVortex` dereferenced the
argument immediately.

## What changes are included in this PR?

Tolerate `null` in `asVortexTableIfVortex`, and replace the javadoc
claim with what actually happens.

- `./gradlew :vortex-spark_2.13:test --tests '…VortexSessionCatalogTest'
--tests '…VortexSqlTest'` — 10 pass
- same on `:vortex-spark_2.12:test`
- `spotlessCheck` (both variants) and `javadoc` — clean

`VortexSessionCatalogTest` already covers the managed-table lifecycle on
both versions, which is the path CTAS actually takes.

## What APIs are changed? Are there any user-facing changes?

None.

## AI assistance

Prepared with agentic AI assistance. I verified the dispatch against the
compiled `DelegatingCatalogExtension`, `CreateTableAsSelectExec` and
`V2SessionCatalog` in both Spark versions rather than reasoning from the
source I had.

Signed-off-by: jackylee <qcsd2011@gmail.com>
robert3005 and others added 2 commits September 11, 2026 12:58
This fixes compatibility with newer rust versions

---------

Signed-off-by: Robert Kruszewski <github@robertk.io>
Upgrade spiceai-54 from Vortex 0.79.0 to official 0.86.1, the latest
upstream release that supports DataFusion 55.x and Rust 1.98.x.

Pin workspace DataFusion crates to 55.1.0 and rust-toolchain.toml to
1.98.1 so the fork matches Spice's DF 55.1 / Rust 1.98.1 line.

Carry Spice patches that did not land upstream (tokio oneshot, timezone
resolution, hashed IN-list probe, declared/absolute scan concurrency,
WriteStrategyBuilder session var). Take 0.86.1 native Map in place of
the fork's List alias.

Signed-off-by: Luke Kim <80174+lukekim@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 19, 2026 21:07
Comment thread vortex-web/package-lock.json Fixed
Comment thread vortex-web/package-lock.json Fixed
Comment thread vortex-web/package-lock.json Fixed
Comment thread vortex-web/package-lock.json Fixed
Comment thread uv.lock Fixed
Comment thread vortex-web/package-lock.json Fixed
Comment thread vortex-web/package-lock.json Fixed
Comment thread vortex-web/package-lock.json Fixed
Comment thread vortex-web/package-lock.json Fixed
Comment thread vortex-web/package-lock.json Fixed
@lukekim

lukekim commented Sep 19, 2026

Copy link
Copy Markdown
Author

Retargeting: spiceai-0.86 is the new version-line base, not the PR head. Replacing this with a PR into spiceai-0.86.

@lukekim lukekim closed this Sep 19, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

A critical panic path and two moderate integration and benchmark issues remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 High severity · 2 Medium severity · 1 Low severity

Open (4)
What changed in this PR

Upgrades the spiceai-54 branch from Vortex 0.79.0 to 0.86.1 while retaining fork-specific runtime, scan, and DataFusion behavior.

Changes:

  • Updates Rust 1.98.1, Arrow 59.2, DataFusion 55.1, encodings, and array APIs.
  • Adds native Map/Union support and refreshes Python, Java, C/C++, CUDA, DuckDB, and FFI integrations.
  • Updates benchmarks, SQL logic tests, snapshots, documentation, and CI workflows.

Reported verification includes 4,802 passing tests; DuckDB, Python, Java, and CUDA were not tested locally.

Unresolved comments:

  • Critical (1 vote): Return an error instead of panicking for unsupported Union min_max.
  • Moderate (3 votes): Reconcile the 100× benchmark dataset increase with documentation and CI resources.
  • Moderate (1 vote): Add GoosefsStore to public Python store annotations.
  • Nit (3 votes): Restore the cloned FFI session ownership/lifecycle documentation.
File Result
vortex/​editions/​zstd/​family.toml Reviewed
vortex/​editions/​tensor/​family.toml Reviewed
vortex/​editions/​spatial/​family.toml Reviewed
vortex/​editions/​preview/​family.toml Reviewed
vortex/​editions/​json/​family.toml Reviewed
vortex-utils/​src/​lib.rs Reviewed
vortex-tui/​src/​wasm.rs Reviewed
vortex-tui/​src/​datafusion_helper.rs Reviewed
vortex-tui/​src/​browse/​ui/​segments.rs Reviewed
vortex-tui/​src/​browse/​ui/​layouts.rs Reviewed
vortex-tui/​README.md Reviewed
vortex-test/​e2e-cuda/​Cargo.toml Reviewed
vortex-test/​compat-gen/​src/​fixtures/​arrays/​synthetic/​encodings/​alprd.rs Reviewed
vortex-test/​compat-gen/​src/​fixtures/​arrays/​synthetic/​arrays/​mod.rs Reviewed
vortex-test/​compat-gen/​Cargo.toml Reviewed
vortex-tensor/​src/​vector_search.rs Reviewed
vortex-tensor/​src/​encodings/​mod.rs Reviewed
vortex-sqllogictest/​slt/​tpch/​duckdb/​tpch.slt Reviewed
vortex-sqllogictest/​slt/​tpch/​duckdb/​parquet.slt Reviewed
vortex-sqllogictest/​slt/​tpch/​datafusion/​tpch.slt Reviewed
vortex-sqllogictest/​slt/​tpch/​datafusion/​parquet.slt Reviewed
vortex-sqllogictest/​slt/​tpch/​datafusion/​drop.slt.no Reviewed
vortex-sqllogictest/​slt/​duckdb/​version.slt Reviewed
vortex-sqllogictest/​slt/​duckdb/​copy_errors.slt Reviewed
vortex-sqllogictest/​slt/​clickbench/​duckdb/​results/​q7.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​duckdb/​results/​q6.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​duckdb/​results/​q5.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​duckdb/​results/​q4.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​duckdb/​results/​q3.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​duckdb/​results/​q20.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​duckdb/​results/​q2.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​duckdb/​results/​q19.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​duckdb/​results/​q1.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​duckdb/​results/​q0.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​duckdb/​parquet.slt Reviewed
vortex-sqllogictest/​slt/​clickbench/​duckdb/​drop.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​duckdb/​create.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​datafusion/​results/​q7.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​datafusion/​results/​q6.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​datafusion/​results/​q5.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​datafusion/​results/​q4.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​datafusion/​results/​q20.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​datafusion/​results/​q2.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​datafusion/​results/​q19.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​datafusion/​results/​q1.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​datafusion/​results/​q0.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​datafusion/​parquet.slt Reviewed
vortex-sqllogictest/​slt/​clickbench/​datafusion/​drop.slt.no Reviewed
vortex-sqllogictest/​slt/​clickbench/​datafusion/​clickbench.slt Reviewed
vortex-sqllogictest/​slt/​clickbench/​.gitignore Reviewed
vortex-sqllogictest/​Cargo.toml Reviewed
vortex-spatial/​src/​tests/​point.rs Reviewed
vortex-spatial/​src/​tests/​multipolygon.rs Reviewed
vortex-spatial/​src/​tests/​multipoint.rs Reviewed
vortex-spatial/​src/​tests/​multilinestring.rs Reviewed
vortex-spatial/​src/​tests/​linestring.rs Reviewed
vortex-spatial/​src/​scalar_fn/​mod.rs Reviewed
vortex-spatial/​src/​aggregate_fn/​mod.rs Reviewed
vortex-session/​src/​lib.rs Reviewed
vortex-session/​Cargo.toml Reviewed
vortex-scan/​src/​lib.rs Reviewed
vortex-scan/​README.md Reviewed
vortex-row/​src/​size.rs Reviewed
vortex-row/​src/​encode.rs Reviewed
vortex-row/​benches/​row_encode.rs Reviewed
vortex-python/​test/​test_version.py Reviewed
vortex-python/​test/​test_scalar.py Reviewed
vortex-python/​test/​test_encodings.py Reviewed
vortex-python/​test/​test_duckdb.py Reviewed
vortex-python/​test/​test_dtype.py Reviewed
vortex-python/​test/​test_cuda.py Reviewed
vortex-python/​src/​scalar/​mod.rs Reviewed
vortex-python/​src/​scalar/​into_py.rs Reviewed
vortex-python/​src/​scalar/​factory.rs Reviewed
vortex-python/​src/​python_repr.rs Reviewed
vortex-python/​src/​object_store/​mod.rs Reviewed
vortex-python/​src/​iter/​python.rs Reviewed
vortex-python/​python/​vortex/​type_aliases.py Reviewed
vortex-python/​python/​vortex/​store/​_memory.py Reviewed
vortex-python/​python/​vortex/​store/​_local.py Reviewed
vortex-python/​python/​vortex/​store/​_goosefs.py Moderate (1 vote): Public store annotations omit GoosefsStore
vortex-python/​python/​vortex/​store/​_cos.py Reviewed
vortex-python/​python/​vortex/​serde.py Reviewed
vortex-python/​python/​vortex/​registry.py Reviewed
vortex-python/​python/​vortex/​io.py Reviewed
vortex-python/​python/​vortex/​cli.py Reviewed
vortex-python/​python/​vortex/​_lib/​store/​_http.pyi Reviewed
vortex-python/​python/​vortex/​_lib/​store/​_gcs.pyi Reviewed
vortex-python/​python/​vortex/​_lib/​store/​_azure.pyi Reviewed
vortex-python/​python/​vortex/​_lib/​store/​_aws.pyi Reviewed
vortex-python/​python/​vortex/​_lib/​arrays.pyi Reviewed
vortex-python/​check.sh Reviewed
vortex-python-cuda/​pyproject.toml Reviewed
vortex-proto/​proto/​scalar.proto Reviewed
vortex-layout/​tests/​list_contains_pruning.rs Reviewed
vortex-layout/​src/​test.rs Reviewed
vortex-layout/​src/​scan/​arrow.rs Reviewed
vortex-layout/​src/​layouts/​zoned/​aggregates/​mod.rs Reviewed
vortex-json/​Cargo.toml Reviewed
vortex-jni/​src/​lib.rs Reviewed
vortex-jni/​src/​io/​mod.rs Reviewed
vortex-ipc/​src/​messages/​decoder.rs Reviewed
vortex-io/​src/​std_file/​mod.rs Reviewed
vortex-io/​src/​session.rs Reviewed
vortex-io/​src/​runtime/​tests.rs Reviewed
vortex-io/​src/​runtime/​platform/​native.rs Reviewed
vortex-io/​src/​object_store/​mod.rs Reviewed
vortex-io/​src/​io_buf.rs Reviewed
vortex-geo/​README.md Reviewed
vortex-flatbuffers/​src/​generated/​message.rs Reviewed
vortex-file/​tests/​test_write_table.rs Reviewed
vortex-file/​src/​segments/​writer.rs Reviewed
vortex-file/​src/​read/​mod.rs Reviewed
vortex-file/​src/​multi/​session.rs Reviewed
vortex-file/​src/​counting.rs Reviewed
vortex-ffi/​src/​session.rs Nit (3 votes): Restore clone ownership/lifecycle documentation
vortex-ffi/​src/​ptype.rs Reviewed
vortex-ffi/​src/​log.rs Reviewed
vortex-ffi/​src/​error.rs Reviewed
vortex-ffi/​examples/​scan_to_arrow.c Reviewed
vortex-ffi/​cmake/​tests/​fixtures/​native/​vortex-ffi/​lib.rs Reviewed
vortex-ffi/​cmake/​tests/​fixtures/​native/​vortex-ffi/​header-version Reviewed
vortex-ffi/​cmake/​tests/​fixtures/​native/​vortex-ffi/​Cargo.toml Reviewed
vortex-ffi/​cmake/​tests/​fixtures/​native/​native-helper/​native.cpp Reviewed
vortex-ffi/​cmake/​tests/​fixtures/​native/​native-helper/​native.c Reviewed
vortex-ffi/​cmake/​tests/​fixtures/​native/​native-helper/​lib.rs Reviewed
vortex-ffi/​cmake/​tests/​fixtures/​native/​native-helper/​Cargo.toml.in Reviewed
vortex-ffi/​cmake/​tests/​fixtures/​native/​header_consumer.c Reviewed
vortex-ffi/​cmake/​tests/​fixtures/​native/​Cargo.toml Reviewed
vortex-error/​Cargo.toml Reviewed
vortex-duckdb/​src/​exporter/​rle.rs Reviewed
vortex-duckdb/​src/​exporter/​cache.rs Reviewed
vortex-duckdb/​src/​exporter/​all_invalid.rs Reviewed
vortex-duckdb/​src/​e2e_test/​mod.rs Reviewed
vortex-duckdb/​src/​duckdb/​vector.rs Reviewed
vortex-duckdb/​src/​duckdb/​logical_type.rs Reviewed
vortex-duckdb/​src/​duckdb/​database.rs Reviewed
vortex-duckdb/​src/​duckdb/​aggregate_pushdown.rs Reviewed
vortex-duckdb/​cpp/​vector.cpp Reviewed
vortex-duckdb/​cpp/​scalar_fn_pushdown.cpp Reviewed
vortex-duckdb/​cpp/​include/​vector.h Reviewed
vortex-duckdb/​cpp/​cast_pushdown.cpp Reviewed
vortex-datafusion/​src/​tests/​schema_evolution.rs Reviewed
vortex-datafusion/​src/​convert/​mod.rs Reviewed
vortex-datafusion/​examples/​vortex_table.rs Reviewed
vortex-cuda/​src/​session.rs Reviewed
vortex-cuda/​src/​kernel/​slice/​mod.rs Reviewed
vortex-cuda/​src/​kernel/​encodings/​zstd.rs Reviewed
vortex-cuda/​src/​kernel/​encodings/​decimal_byte_parts.rs Reviewed
vortex-cuda/​src/​kernel/​arrays/​mod.rs Reviewed
vortex-cuda/​src/​hybrid_dispatch/​mod.rs Reviewed
vortex-cuda/​gpu-scan-cli/​Cargo.toml Reviewed
vortex-cuda/​ffi/​README.md Reviewed
vortex-cuda/​benches/​list_view_cuda.rs Reviewed
vortex-cuda/​benches/​bench_config/​mod.rs Reviewed
vortex-cuda/​benches/​arrow_binary_cuda.rs Reviewed
vortex-compute/​Cargo.toml Reviewed
vortex-compressor/​src/​compressor/​mod.rs Reviewed
vortex-buffer/​src/​serde.rs Reviewed
vortex-buffer/​src/​const.rs Reviewed
vortex-btrblocks/​tests/​snapshots/​golden__unstable__int_wide_random.snap Reviewed
vortex-btrblocks/​tests/​snapshots/​golden__unstable__int_negatives.snap Reviewed
vortex-btrblocks/​tests/​snapshots/​golden__unstable__int_arithmetic_sequence.snap Reviewed
vortex-btrblocks/​tests/​snapshots/​golden__unstable__float_alp_prices.snap Reviewed
vortex-btrblocks/​tests/​snapshots/​golden__unstable__decimal_prices.snap Reviewed
vortex-btrblocks/​tests/​snapshots/​golden__unstable__bool_random.snap Reviewed
vortex-btrblocks/​tests/​snapshots/​golden__unstable__binary_low_cardinality.snap Reviewed
vortex-btrblocks/​tests/​snapshots/​golden__regular__int_wide_random.snap Reviewed
vortex-btrblocks/​tests/​snapshots/​golden__regular__int_negatives.snap Reviewed
vortex-btrblocks/​tests/​snapshots/​golden__regular__int_monotone_jitter.snap Reviewed
vortex-btrblocks/​tests/​snapshots/​golden__regular__int_arithmetic_sequence.snap Reviewed
vortex-btrblocks/​tests/​snapshots/​golden__regular__float_alp_prices.snap Reviewed
vortex-btrblocks/​tests/​snapshots/​golden__regular__decimal_prices.snap Reviewed
vortex-btrblocks/​tests/​snapshots/​golden__regular__bool_random.snap Reviewed
vortex-btrblocks/​src/​schemes/​string/​tests.rs Reviewed
vortex-btrblocks/​src/​schemes/​float/​scheme_selection_tests.rs Reviewed
vortex-btrblocks/​src/​lib.rs Reviewed
vortex-bench/​src/​vortex_queries.rs Reviewed
vortex-bench/​src/​vector_dataset/​download.rs Reviewed
vortex-bench/​src/​utils/​file.rs Reviewed
vortex-bench/​src/​statpopgen/​statpopgen_benchmark.rs Reviewed
vortex-bench/​src/​spatialbench/​mod.rs Reviewed
vortex-bench/​src/​spatialbench/​datagen/​mod.rs Reviewed
vortex-bench/​src/​snapshots/​vortex_bench__v3__tests__snapshot_random_access_time.snap Reviewed
vortex-bench/​src/​snapshots/​vortex_bench__v3__tests__snapshot_compression_size.snap Reviewed
vortex-bench/​src/​realnest/​gharchive.rs Reviewed
vortex-bench/​src/​public_bi.rs Reviewed
vortex-bench/​src/​polarsignals/​benchmark.rs Reviewed
vortex-bench/​src/​fineweb/​mod.rs Reviewed
vortex-bench/​src/​appian/​mod.rs Reviewed
vortex-bench/​sql/​vortex/​init.sql Moderate (3 votes): Reconcile 100× dataset increase with docs and CI resources
vortex-bench/​sql/​vortex/​2_sum-range.sql Reviewed
vortex-bench/​sql/​vortex/​1_sum.sql Reviewed
vortex-bench/​sql/​statpopgen.sql Reviewed
vortex-bench/​sql/​public-bi.md Reviewed
vortex-bench/​REUSE.toml Reviewed
vortex-array/​src/​test_harness/​mod.rs Reviewed
vortex-array/​src/​stats/​mod.rs Reviewed
vortex-array/​src/​scalar/​typed_view/​mod.rs Reviewed
vortex-array/​src/​scalar/​typed_view/​decimal/​mod.rs Reviewed
vortex-array/​src/​scalar/​mod.rs Reviewed
vortex-array/​src/​scalar_fn/​unstable/​mod.rs Reviewed
vortex-array/​src/​scalar_fn/​fns/​mod.rs Reviewed
vortex-array/​src/​scalar_fn/​fns/​ext_storage.rs Reviewed
vortex-array/​src/​scalar_fn/​fns/​byte_length.rs Reviewed
vortex-array/​src/​scalar_fn/​fns/​binary/​boolean.rs Reviewed
vortex-array/​src/​matcher.rs Reviewed
vortex-array/​src/​expr/​transform/​mod.rs Reviewed
vortex-array/​src/​expr/​analysis/​mod.rs Reviewed
vortex-array/​src/​dtype/​decimal/​precision.rs Reviewed
vortex-array/​src/​display/​mod.rs Reviewed
vortex-array/​src/​display/​extractors/​stats.rs Reviewed
vortex-array/​src/​display/​extractors/​nbytes.rs Reviewed
vortex-array/​src/​display/​extractors/​metadata.rs Reviewed
vortex-array/​src/​display/​extractors/​encoding_summary.rs Reviewed
vortex-array/​src/​display/​extractors/​buffer.rs Reviewed
vortex-array/​src/​builders/​null.rs Reviewed
vortex-array/​src/​arrays/​variant/​vtable/​validity.rs Reviewed
vortex-array/​src/​arrays/​variant/​vtable/​operations.rs Reviewed
vortex-array/​src/​arrays/​variant/​compute/​take.rs Reviewed
vortex-array/​src/​arrays/​variant/​compute/​slice.rs Reviewed
vortex-array/​src/​arrays/​variant/​compute/​filter.rs Reviewed
vortex-array/​src/​arrays/​varbinview/​mod.rs Reviewed
vortex-array/​src/​arrays/​varbinview/​compute/​zip.rs Reviewed
vortex-array/​src/​arrays/​varbin/​mod.rs Reviewed
vortex-array/​src/​arrays/​varbin/​compute/​slice.rs Reviewed
vortex-array/​src/​arrays/​varbin/​compute/​mod.rs Reviewed
vortex-array/​src/​arrays/​varbin/​compute/​mask.rs Reviewed
vortex-array/​src/​arrays/​varbin/​compute/​cast.rs Reviewed
vortex-array/​src/​arrays/​union/​compute/​mod.rs Reviewed
vortex-array/​src/​arrays/​struct_/​mod.rs Reviewed
vortex-array/​src/​arrays/​struct_/​compute/​take.rs Reviewed
vortex-array/​src/​arrays/​struct_/​compute/​mask.rs Reviewed
vortex-array/​src/​arrays/​slice/​slice_.rs Reviewed
vortex-array/​src/​arrays/​shared/​mod.rs Reviewed
vortex-array/​src/​arrays/​scalar_fn/​mod.rs Reviewed
vortex-array/​src/​arrays/​primitive/​mod.rs Reviewed
vortex-array/​src/​arrays/​primitive/​compute/​mod.rs Reviewed
vortex-array/​src/​arrays/​masked/​vtable/​mod.rs Reviewed
vortex-array/​src/​arrays/​masked/​array.rs Reviewed
vortex-array/​src/​arrays/​map/​vtable/​validity.rs Reviewed
vortex-array/​src/​arrays/​listview/​tests/​take.rs Reviewed
vortex-array/​src/​arrays/​listview/​tests/​operations.rs Reviewed
vortex-array/​src/​arrays/​listview/​tests/​nullability.rs Reviewed
vortex-array/​src/​arrays/​listview/​tests/​filter.rs Reviewed
vortex-array/​src/​arrays/​listview/​tests/​density.rs Reviewed
vortex-array/​src/​arrays/​listview/​mod.rs Reviewed
vortex-array/​src/​arrays/​listview/​compute/​take.rs Reviewed
vortex-array/​src/​arrays/​listview/​compute/​slice.rs Reviewed
vortex-array/​src/​arrays/​listview/​compute/​rules.rs Reviewed
vortex-array/​src/​arrays/​listview/​compute/​mask.rs Reviewed
vortex-array/​src/​arrays/​listview/​compute/​cast.rs Reviewed
vortex-array/​src/​arrays/​list/​tests.rs Reviewed
vortex-array/​src/​arrays/​list/​mod.rs Reviewed
vortex-array/​src/​arrays/​list/​compute/​slice.rs Reviewed
vortex-array/​src/​arrays/​list/​compute/​mask.rs Reviewed
vortex-array/​src/​arrays/​list/​compute/​cast.rs Reviewed
vortex-array/​src/​arrays/​fixed_size_list/​tests/​degenerate.rs Reviewed
vortex-array/​src/​arrays/​fixed_size_list/​tests/​basic.rs Reviewed
vortex-array/​src/​arrays/​fixed_size_list/​mod.rs Reviewed
vortex-array/​src/​arrays/​fixed_size_list/​compute/​slice.rs Reviewed
vortex-array/​src/​arrays/​fixed_size_list/​compute/​mask.rs Reviewed
vortex-array/​src/​arrays/​fixed_size_list/​compute/​cast.rs Reviewed
vortex-array/​src/​arrays/​filter/​mod.rs Reviewed
vortex-array/​src/​arrays/​filter/​execute/​take.rs Reviewed
vortex-array/​src/​arrays/​extension/​mod.rs Reviewed
vortex-array/​src/​arrays/​decimal/​mod.rs Reviewed
vortex-array/​src/​arrays/​chunked/​mod.rs Reviewed
vortex-array/​src/​arrays/​bool/​mod.rs Reviewed
vortex-array/​src/​aggregate_fn/​fns/​uncompressed_size_in_bytes/​list_view.rs Reviewed
vortex-array/​src/​aggregate_fn/​fns/​uncompressed_size_in_bytes/​fixed_size_list.rs Reviewed
vortex-array/​src/​aggregate_fn/​fns/​sum/​decimal.rs Reviewed
vortex-array/​src/​aggregate_fn/​fns/​sum/​constant.rs Reviewed
vortex-array/​src/​aggregate_fn/​fns/​sum/​bool.rs Reviewed
vortex-array/​src/​aggregate_fn/​fns/​mod.rs Reviewed
vortex-array/​src/​aggregate_fn/​fns/​min_max/​mod.rs Critical (1 vote): Unsupported Union arrays panic instead of returning an error
vortex-array/​src/​aggregate_fn/​fns/​is_constant/​struct_.rs Reviewed
vortex-array/​src/​aggregate_fn/​fns/​is_constant/​list.rs Reviewed
vortex-array/​src/​aggregate_fn/​fns/​all_non_distinct/​list.rs Reviewed
vortex-array/​src/​aggregate_fn/​fns/​all_non_distinct/​fixed_size_list.rs Reviewed
vortex-array/​src/​aggregate_fn/​erased.rs Reviewed
vortex-array/​benches/​scalar_at_struct.rs Reviewed
vortex-array/​benches/​patches_lookup.rs Reviewed
vortex-array/​benches/​list_sum.rs Reviewed
vortex-array/​benches/​list_length.rs Reviewed
vortex-array/​benches/​expr/​optimize_bench.rs Reviewed
vortex-array/​benches/​dict_unreferenced_mask.rs Reviewed
vortex-array/​benches/​dict_mask.rs Reviewed
vortex-array/​benches/​chunked_fsl_canonicalize.rs Reviewed
vortex-array/​benches/​aggregate_max.rs Reviewed
scripts/​s3-upload.py Reviewed
scripts/​s3-download.py Reviewed
scripts/​compare-file-sizes.py Reviewed
scripts/​capture-file-sizes.py Reviewed
rust-toolchain.toml Reviewed
lang/​cpp/​tests/​float16_t_compat.cpp Reviewed
lang/​cpp/​tests/​expression.cpp Reviewed
lang/​cpp/​src/​scalar.cpp Reviewed
lang/​cpp/​include/​vortex/​common.hpp Reviewed
java/​vortex-spark/​src/​main/​java/​dev/​vortex/​spark/​config/​HadoopUtils.java Reviewed
java/​vortex-jni/​src/​main/​java/​dev/​vortex/​api/​Scan.java Reviewed
java/​vortex-jni/​src/​main/​java/​dev/​vortex/​api/​DataSource.java Reviewed
java/​vortex-jni/​.gitignore Reviewed
java/​gradle/​libs.versions.toml Reviewed
fuzz/​src/​gpu/​mod.rs Reviewed
fuzz/​src/​fsst_like.rs Reviewed
fuzz/​src/​array/​sort.rs Reviewed
fuzz/​src/​array/​search_sorted.rs Reviewed
fuzz/​src/​array/​mod.rs Reviewed
fuzz/​src/​array/​fill_null.rs Reviewed
fuzz/​src/​array/​compare.rs Reviewed
encodings/​zstd/​src/​slice.rs Reviewed
encodings/​zstd/​Cargo.toml Reviewed
encodings/​zigzag/​src/​slice.rs Reviewed
encodings/​zigzag/​src/​compute/​cast.rs Reviewed
encodings/​sequence/​src/​lib.rs Reviewed
encodings/​sequence/​src/​compute/​slice.rs Reviewed
encodings/​runend/​src/​rules.rs Reviewed
encodings/​runend/​src/​ops.rs Reviewed
encodings/​runend/​src/​lib.rs Reviewed
encodings/​runend/​src/​kernel.rs Reviewed
encodings/​runend/​src/​decompress_bool.rs Reviewed
encodings/​runend/​src/​compute/​take.rs Reviewed
encodings/​runend/​src/​compute/​take_from.rs Reviewed
encodings/​runend/​src/​compute/​min_max.rs Reviewed
encodings/​runend/​src/​compute/​is_sorted.rs Reviewed
encodings/​runend/​src/​compute/​is_constant.rs Reviewed
encodings/​runend/​src/​compute/​fill_null.rs Reviewed
encodings/​runend/​src/​compute/​compare.rs Reviewed
encodings/​runend/​src/​compute/​cast.rs Reviewed
encodings/​runend/​Cargo.toml Reviewed
encodings/​parquet-variant/​src/​validity.rs Reviewed
encodings/​parquet-variant/​src/​lib.rs Reviewed
encodings/​parquet-variant/​Cargo.toml Reviewed
encodings/​onpair/​src/​rules.rs Reviewed
encodings/​onpair/​src/​compute/​slice.rs Reviewed
encodings/​onpair/​src/​compute/​mod.rs Reviewed
encodings/​onpair/​goldenfiles/​onpair.metadata Reviewed
encodings/​fsst/​src/​test_utils.rs Reviewed
encodings/​fsst/​src/​slice.rs Reviewed
encodings/​fsst/​src/​compute/​filter.rs Reviewed
encodings/​fsst/​src/​compute/​compare.rs Reviewed
encodings/​fsst/​src/​compute/​byte_length.rs Reviewed
encodings/​fastlanes/​src/​rle/​vtable/​validity.rs Reviewed
encodings/​fastlanes/​src/​rle/​vtable/​operations.rs Reviewed
encodings/​fastlanes/​src/​rle/​mod.rs Reviewed
encodings/​fastlanes/​src/​rle/​kernel.rs Reviewed
encodings/​fastlanes/​src/​rle/​compute/​cast.rs Reviewed
encodings/​fastlanes/​src/​for/​vtable/​validity.rs Reviewed
encodings/​fastlanes/​src/​for/​vtable/​slice.rs Reviewed
encodings/​fastlanes/​src/​for/​vtable/​rules.rs Reviewed
encodings/​fastlanes/​src/​for/​vtable/​operations.rs Reviewed
encodings/​fastlanes/​src/​for/​mod.rs Reviewed
encodings/​fastlanes/​src/​for/​compute/​mod.rs Reviewed
encodings/​fastlanes/​src/​for/​compute/​is_constant.rs Reviewed
encodings/​fastlanes/​src/​for/​compute/​compare.rs Reviewed
encodings/​fastlanes/​src/​for/​compute/​cast.rs Reviewed
encodings/​fastlanes/​src/​for/​array/​for_compress.rs Reviewed
encodings/​fastlanes/​src/​delta/​vtable/​slice.rs Reviewed
encodings/​fastlanes/​src/​delta/​mod.rs Reviewed
encodings/​fastlanes/​src/​delta/​compute/​cast.rs Reviewed
encodings/​fastlanes/​src/​bitpacking/​compute/​compare.rs Reviewed
encodings/​fastlanes/​Cargo.toml Reviewed
encodings/​experimental/​onpair/​goldenfiles/​onpair.metadata Reviewed
encodings/​decimal-byte-parts/​src/​decimal_byte_parts/​slice.rs Reviewed
encodings/​decimal-byte-parts/​src/​decimal_byte_parts/​rules.rs Reviewed
encodings/​decimal-byte-parts/​src/​decimal_byte_parts/​compute/​take.rs Reviewed
encodings/​decimal-byte-parts/​src/​decimal_byte_parts/​compute/​is_constant.rs Reviewed
encodings/​decimal-byte-parts/​src/​decimal_byte_parts/​compute/​filter.rs Reviewed
encodings/​decimal-byte-parts/​src/​decimal_byte_parts/​compute/​compare.rs Reviewed
encodings/​decimal-byte-parts/​src/​decimal_byte_parts/​compute/​cast.rs Reviewed
encodings/​datetime-parts/​src/​array.rs Reviewed
encodings/​datetime-parts/​Cargo.toml Reviewed
encodings/​alp/​src/​lib.rs Reviewed
encodings/​alp/​src/​alp_rd/​slice.rs Reviewed
encodings/​alp/​src/​alp_rd/​compute/​mod.rs Reviewed
encodings/​alp/​src/​alp_rd/​compute/​cast.rs Reviewed
encodings/​alp/​Cargo.toml Reviewed
docs/​user-guide/​pyarrow.md Reviewed
docs/​user-guide/​pandas.md Reviewed
docs/​specs/​index.md Reviewed
docs/​project/​roadmap.md Reviewed
docs/​project/​index.md Reviewed
docs/​project/​contributing.md Reviewed
docs/​getting-started/​query.md Reviewed
docs/​getting-started/​index.md Reviewed
docs/​developer-guide/​index.md Reviewed
docs/​developer-guide/​embedding/​scan-api.md Reviewed
docs/​developer-guide/​embedding/​gpu.md Reviewed
docs/​developer-guide/​embedding/​ffi.md Reviewed
docs/​developer-guide/​embedding/​cxx.md Reviewed
docs/​api/​python/​store.rst Reviewed
docs/​api/​index.md Reviewed
docs/​AGENTS.md Reviewed
docs/​_theme/​vortex/​theme.toml Reviewed
benchmarks/​random-access-bench/​Cargo.toml Reviewed
benchmarks/​lance-bench/​src/​random_access.rs Reviewed
benchmarks/​datafusion-bench/​src/​tracer.rs Reviewed
benchmarks/​compress-bench/​src/​lib.rs Reviewed
.github/​workflows/​typos.yml Reviewed
.github/​workflows/​stale.yml Reviewed
.github/​workflows/​reuse.yml Reviewed
.github/​workflows/​release-binaries.yml Reviewed
.github/​workflows/​pr-bench-string.yml Reviewed
.github/​workflows/​pr-bench-compress.yml Reviewed
.github/​workflows/​compat-validation.yml Reviewed
.github/​workflows/​claude-write.yml Reviewed
.github/​workflows/​claude-review.yml Reviewed
.github/​scripts/​pyproject.toml Reviewed
.github/​scripts/​fuzz_report/​template.py Reviewed
.github/​scripts/​fuzz_report/​cli.py Reviewed
.github/​scripts/​close_fixed_fuzzer_issues.py Reviewed

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread vortex-array/src/aggregate_fn/fns/min_max/mod.rs Outdated
Comment thread vortex-bench/sql/vortex/init.sql
Comment thread vortex-python/python/vortex/store/_goosefs.py
Comment thread vortex-ffi/src/session.rs Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Public C ownership documentation was removed and the newly linked benchmark documentation no longer matches its 2.5B-row schema and query set.

Review effort: Balanced
Findings: 1 High severity · 2 Medium severity · 1 Low severity

Open (4)

…I docs

- Return VortexResult unsupported-type error for Union min_max (no todo!/panic)
- Document 2.5B-row three-column vortex SQL bench + range query
- Expose GoosefsStore on vortex.file.open / _lib/file.pyi like CosStore
- Restore vx_session_clone ownership/lifecycle note for generated C API docs
…0924

fix: Copilot review for #100 (min_max, bench docs, GooseFS, FFI)
@lukekim
lukekim changed the base branch from spiceai-54 to spiceai-55 September 28, 2026 18:35
@lukekim lukekim changed the title Merge upstream Vortex 0.86.1 into spiceai-54 Merge upstream Vortex 0.86.1 into spiceai-55 Sep 28, 2026
@lukekim
lukekim merged commit 806e48d into spiceai-55 Sep 28, 2026
56 of 67 checks passed

This branch had an error being deployed

1 failed deployment
github-pages — 17fd97dd Deployed Sep 28, 2026 by lukekim via Deploy to Cloudflare Pages #118
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.