Skip to content

Commit 44dfa07

Browse files
lancedb: Add versions 0.38.0, 0.39.0 (#1987)
1 parent 72ebaf3 commit 44dfa07

6 files changed

Lines changed: 2767 additions & 11 deletions

‎.github/workflows/build-lancedb.yml‎

Lines changed: 53 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -77,16 +77,23 @@ jobs:
7777
- name: Apply lancedb patches
7878
run: git apply -v python-wheels/patches/lancedb/${{ env.LANCEDB_VERSION }}/0001*.patch
7979

80+
# lancedb pins lance-core/lance-linalg to a different exact version per
81+
# release (10.0.0 for v0.37.1, 11.0.0 for v0.38.0, 12.0.0 for v0.39.0),
82+
# so the sibling checkout below must track whatever this lancedb tag's
83+
# own Cargo.toml pins rather than a fixed tag.
84+
- name: Determine the lance-core/lance-linalg version lancedb pins
85+
run: echo "LANCE_VERSION=$(grep -m1 '^lance-core = ' Cargo.toml | grep -oE '[0-9]+\.[0-9]+\.[0-9]+')" >> "$GITHUB_ENV"
86+
8087
# lance-core's SIMD_SUPPORT static and lance-linalg's f32x8/f32x16/
8188
# i32x8/f64x4/f64x8 SIMD types compile only for x86_64/aarch64/
8289
# loongarch64 upstream; the 0002 patch adds a portable riscv64 fallback
8390
# to this checkout, and Cargo.toml's `[patch.crates-io]` (0001) redirects
8491
# both crates here.
85-
- name: Checkout lance v10.0.0 and add a riscv64 SIMD fallback
92+
- name: Checkout lance v${{ env.LANCE_VERSION }} and add a riscv64 SIMD fallback
8693
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
8794
with:
8895
repository: lancedb/lance
89-
ref: v10.0.0
96+
ref: v${{ env.LANCE_VERSION }}
9097
path: lance
9198
persist-credentials: false
9299

@@ -144,18 +151,38 @@ jobs:
144151
CARGO_BUILD_JOBS=2
145152
PIP_EXTRA_INDEX_URL=https://pypi.riseproject.dev/simple/
146153
# Without this pip prefers PyPI's releases, which have no riscv64
147-
# wheel for these, and source-builds them in the container.
148-
CIBW_TEST_ENVIRONMENT: PIP_ONLY_BINARY=numpy,pandas,pyarrow,duckdb,pydantic-core
149-
CIBW_TEST_REQUIRES: pytest pytest-asyncio numpy pandas pyarrow duckdb
154+
# wheel for these, and source-builds them in the container. pylance
155+
# is pinned to our registry's exact published version (not just
156+
# named) per gotcha 30: with PIP_EXTRA_INDEX_URL alone pip picks the
157+
# highest version across both indexes, and PyPI's latest pylance is
158+
# newer than what pypi.riseproject.dev hosts, so an unpinned
159+
# `pylance` would resolve to a PyPI release with no riscv64 wheel
160+
# and source-build the whole lance Rust workspace here.
161+
CIBW_TEST_ENVIRONMENT: PIP_ONLY_BINARY=numpy,pandas,pyarrow,duckdb,pydantic-core,pylance
162+
CIBW_TEST_REQUIRES: pytest pytest-asyncio numpy pandas pyarrow duckdb pylance==11.0.0
150163
CIBW_TEST_SOURCES: python/pyproject.toml python/python/tests
151164
# test_table.py, test_util.py, test_embeddings.py and
152165
# test_namespace_integration.py import `polars` and/or `lance`
153-
# (pylance) unconditionally at module scope, as do
154-
# docs/test_guide_tables.py and docs/test_python.py; neither package
155-
# has a riscv64 wheel and both are large enough Rust projects in
156-
# their own right that building them from source here is out of
157-
# scope, so those six files fail collection and are excluded rather
158-
# than left to error. Otherwise mirrors run_tests/action.yml's
166+
# (pylance) and/or `lance_namespace` unconditionally at module
167+
# scope, as do docs/test_guide_tables.py and docs/test_python.py;
168+
# polars has no riscv64 wheel and is a large enough Rust project in
169+
# its own right that building it from source here is out of scope,
170+
# so those six files fail collection and stay excluded. This was
171+
# not re-checked file-by-file against pylance's newer availability
172+
# below -- only test_blob.py, the one file CI actually reported
173+
# failing here, was fixed rather than excluded.
174+
#
175+
# test_blob.py imports `lance` (the pylance package) unconditionally
176+
# at module scope too, but is new in 0.38.0/0.39.0 (not present in
177+
# 0.37.1, when the six-file exclusion above was established) so it
178+
# was never checked against pylance's availability until now. It
179+
# isn't excluded because pypi.riseproject.dev already hosts a
180+
# pylance riscv64 wheel (see docs/packages/pylance.yaml, added to
181+
# CIBW_TEST_REQUIRES above) and lance's python/python/lance/blob.py
182+
# is unchanged between the v9.0.0 pylance/lance release lancedb's
183+
# own pyproject.toml `tests` extra pins and the v11.0.0 this repo
184+
# publishes, aside from an added BlobFile.read_ranges method
185+
# test_blob.py doesn't use. Otherwise mirrors run_tests/action.yml's
159186
# non-integration path (`-m "not slow and not s3_test"`, no docker
160187
# compose/localstack).
161188
#
@@ -179,6 +206,19 @@ jobs:
179206
# identically either way) and the tests run and fail anyway, as seen
180207
# in CI run 33843581031. `cd python` first puts pyproject.toml back
181208
# in the search path so --deselect (and the -m marker filter) work.
209+
#
210+
# test_first_class_function_slice1.py/slice2.py are new (0.38.0/
211+
# 0.39.0) and hit the exact same class of trap as
212+
# test_pyo3_abi_matches_minimum_supported_python above: they read
213+
# golden fixtures from outside the CIBW_TEST_SOURCES tree --
214+
# docs/src/python/python.md and rust/lancedb/tests/fixtures/
215+
# first_class_functions/v1/*.json, both siblings of python/ in the
216+
# full checkout but not copied into the isolated test dir (only
217+
# pyproject.toml and python/tests are) -- and fail with the same
218+
# FileNotFoundError regardless of platform. They assert Python/Rust
219+
# golden-contract and docs consistency, not installed-wheel runtime
220+
# behavior, so ignore both files rather than widen
221+
# CIBW_TEST_SOURCES to mirror docs/ and rust/'s layout.
182222
CIBW_TEST_COMMAND: >-
183223
cd python &&
184224
python -m pytest python/tests -vv --durations=30
@@ -189,6 +229,8 @@ jobs:
189229
--ignore=python/tests/test_namespace_integration.py
190230
--ignore=python/tests/docs/test_guide_tables.py
191231
--ignore=python/tests/docs/test_python.py
232+
--ignore=python/tests/test_first_class_function_slice1.py
233+
--ignore=python/tests/test_first_class_function_slice2.py
192234
--deselect=python/tests/test_db.py::test_create_table_stable_row_ids_via_storage_options
193235
--deselect=python/tests/test_db.py::test_create_table_stable_row_ids_via_storage_options_sync
194236
--deselect=python/tests/test_db.py::test_create_table_stable_row_ids_table_level_override

‎docs/packages/lancedb.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,5 @@ versions:
99
- filename: lancedb-0.37.1-cp310-abi3-manylinux_2_39_riscv64.whl
1010
sha256: 405656a126ed0c70028834c5370463e3e2b5f9b53dc7ef36a6d79b000ee276ac
1111
requires-python: '>=3.10'
12+
- version: 0.38.0
13+
- version: 0.39.0
Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
2+
From: Ludovic Henry <git@ludovic.dev>
3+
Date: Thu, 03 Sep 2026 00:00:00 +0000
4+
Subject: [PATCH] cargo: redirect lance-core/lance-linalg to a riscv64-patched
5+
checkout
6+
7+
lance-core's SIMD_SUPPORT static (rust/lance-core/src/utils/cpu.rs) and
8+
lance-linalg's f32x8/f32x16/i32x8/f64x4/f64x8 SIMD types
9+
(rust/lance-linalg/src/simd/{f32,i32,f64}.rs) are compiled for
10+
x86_64/aarch64/loongarch64 only, with no fallback arm for any other
11+
architecture: the SIMD_SUPPORT closure has no trailing expression on
12+
riscv64 ("expected SimdSupport, found ()"), and the SIMD type structs are
13+
defined only under those three `#[cfg(target_arch = ...)]` attributes
14+
while every method implementing them is unconditional, so riscv64 hits
15+
"cannot find type `f32x8` in this scope" as soon as lance-linalg is
16+
built.
17+
18+
Both crates are pulled in from crates.io at a pinned `=10.0.0`, so the
19+
fix cannot be applied in place. Redirect them to a checkout of the same
20+
v10.0.0 tag carrying a portable scalar fallback for both, see
21+
0002-lance-linalg-add-a-portable-riscv64-simd-fallback.patch. `exclude`
22+
keeps that checkout's own [workspace] from being silently absorbed into
23+
this one, which would drop its `[workspace.package]` inheritance
24+
(version, edition, ...) for lance-core and lance-linalg.
25+
26+
Only the source line of the lance-core/lance-linalg entries changes in
27+
Cargo.lock, so every other dependency stays pinned exactly as upstream
28+
released it.
29+
30+
Upstream-Status: Inappropriate [redirects two pinned dependencies to a local checkout; the fix itself belongs in lancedb/lance]
31+
32+
Signed-off-by: Ludovic Henry <git@ludovic.dev>
33+
---
34+
Cargo.toml | 12 ++++++++++++
35+
1 file changed, 12 insertions(+)
36+
37+
diff --git a/Cargo.toml b/Cargo.toml
38+
index a47e59e..ab6cf5c 100644
39+
--- a/Cargo.toml
40+
+++ b/Cargo.toml
41+
@@ -1,5 +1,10 @@
42+
[workspace]
43+
members = ["rust/lancedb", "nodejs", "python"]
44+
+# ./lance (see [patch.crates-io] below) carries its own [workspace]; excluding
45+
+# it keeps that workspace independent instead of being silently absorbed into
46+
+# this one, which would drop its `[workspace.package]` inheritance (version,
47+
+# edition, ...) for lance-core and lance-linalg.
48+
+exclude = ["lance"]
49+
resolver = "2"
50+
51+
[workspace.package]
52+
@@ -80,3 +85,10 @@ debug = false
53+
debug-assertions = false
54+
strip = "debuginfo"
55+
incremental = false
56+
+
57+
+# lance-core's SIMD_SUPPORT and lance-linalg's f32x8/f32x16/i32x8/f64x4/f64x8
58+
+# SIMD types compile only for x86_64/aarch64/loongarch64 upstream; ./lance is
59+
+# the same v10.0.0 tag with a riscv64 scalar fallback added to both.
60+
+[patch.crates-io]
61+
+lance-core = { path = "lance/rust/lance-core" }
62+
+lance-linalg = { path = "lance/rust/lance-linalg" }
63+
--
64+
2.43.0

0 commit comments

Comments
 (0)