Skip to content

Commit 9db6600

Browse files
luhenryluhenry
committed
openviking: install perl for openssl-src, restore rustc caps, relax tree-sitter pins
From 0.4.18 crates/ragfs enables fred's vendored-openssl feature through the cache feature that ragfs-python always builds, so openssl-src compiles OpenSSL from source. Its Configure script needs FindBin.pm, which the riscv64 manylinux image's minimal perl-interpreter lacks: Can't locate FindBin.pm in @inc ... at ./Configure line 15. That failed 0.4.22 in run 37959647170, and it hits every version from 0.4.18 through 0.4.23. Upstream builds its Linux wheels in ubuntu:20.04, which has full perl, and ships the OpenSSL statically linked, so install the whole perl distribution in CIBW_BEFORE_ALL_LINUX (gotcha 46, as build-rfc3161-client.yml, build-nassl.yml and build-solders.yml do). Switching to the system OpenSSL with OPENSSL_NO_VENDOR would diverge from upstream's wheel and has no precedent in this repo. The nightly-upgrade bot's force-push dropped this fix together with the others from ac592b3. That commit's run 36397583332 built and tested 0.4.17 through 0.4.20 green with them, so restore them as well: - When Cargo.lock pins openssl-src, fetch that OpenSSL release's LICENSE.txt as LICENSE.openssl and require it in the licence check, since the wheel now redistributes OpenSSL. - CARGO_PROFILE_RELEASE_LTO=thin and CARGO_BUILD_JOBS=2 for the rustc heap corruption and hangs compiling ragfs-python (gotcha 228). In that run 0.4.21 and 0.4.22 still failed to install the built wheel: "tree-sitter-rust==0.24.0 (from versions: 0.24.2)". 0.4.21 replaced the parsing stack's lower bounds with exact pins, and the old patch relaxed only tree-sitter. A binary-only riscv64 resolution of 0.4.23's dependencies shows four exact pins without riscv64 wheels: tree-sitter 0.25.2, tree-sitter-rust 0.24.0, tree-sitter-c-sharp 0.23.1 and tree-sitter-language-pack 1.13.3. The new 0002 patch for 0.4.21, 0.4.22 and 0.4.23 turns those four into lower bounds. With it, the full dependency set plus test requirements resolves for cp312, cp313 and cp314. Every version's patch set was checked with git apply --check against its own tag.
1 parent 928a800 commit 9db6600

4 files changed

Lines changed: 174 additions & 1 deletion

‎.github/workflows/build-openviking.yml‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,12 @@ jobs:
116116
cp third_party/spdlog-1.14.1/LICENSE LICENSE.spdlog
117117
cp third_party/croaring/LICENSE LICENSE.croaring
118118
cp third_party/rapidjson/LICENSE LICENSE.rapidjson
119+
# From 0.4.18 fred's vendored-openssl links OpenSSL statically into the ragfs extension.
120+
openssl=$(grep -A1 '^name = "openssl-src"$' Cargo.lock | sed -nE 's/^version = "[0-9.]+\+([0-9.]+)"$/\1/p' || true)
121+
if [ -n "${openssl}" ]; then
122+
curl -fsSL -o LICENSE.openssl "https://raw.githubusercontent.com/openssl/openssl/openssl-${openssl}/LICENSE.txt"
123+
echo "OV_EXTRA_LICENCES=LICENSE.openssl" >> "${GITHUB_ENV}"
124+
fi
119125
120126
- name: Write smoke test
121127
run: |
@@ -186,10 +192,15 @@ jobs:
186192
CIBW_ARCHS: riscv64
187193
CIBW_BUILD: "cp310-* cp311-* cp312-* cp313-* cp314-*"
188194
CIBW_SKIP: "*-musllinux_*"
195+
# openssl-src (0.4.18+) runs OpenSSL's Configure, which needs FindBin (gotcha 46).
189196
CIBW_BEFORE_ALL_LINUX: >-
197+
dnf -y --disablerepo=extras install perl &&
190198
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal
199+
# Fat LTO and parallel rustc of ragfs-python corrupt rustc's heap or hang on these runners (gotcha 228).
191200
CIBW_ENVIRONMENT: >-
192201
PATH="$PATH:$HOME/.cargo/bin"
202+
CARGO_PROFILE_RELEASE_LTO=thin
203+
CARGO_BUILD_JOBS=2
193204
PIP_EXTRA_INDEX_URL=https://pypi.riseproject.dev/simple/
194205
OV_REQUIRE_RAGFS_BUILD=1
195206
SETUPTOOLS_SCM_PRETEND_VERSION_FOR_OPENVIKING=${{ env.OPENVIKING_VERSION }}
@@ -201,7 +212,7 @@ jobs:
201212
CIBW_TEST_SOURCES: tests pyproject.toml smoke_test.py
202213
CIBW_TEST_COMMAND: >-
203214
python smoke_test.py &&
204-
python -c "import importlib.metadata as m; l = set(p.name for p in m.files('openviking') if '.dist-info/licenses/' in str(p)); assert set(['LICENSE', 'LICENSE.leveldb', 'LICENSE.spdlog', 'LICENSE.croaring', 'LICENSE.rapidjson']) <= l, l" &&
215+
python -c "import importlib.metadata as m; l = set(p.name for p in m.files('openviking') if '.dist-info/licenses/' in str(p)); assert set(['LICENSE', 'LICENSE.leveldb', 'LICENSE.spdlog', 'LICENSE.croaring', 'LICENSE.rapidjson'] + '${{ env.OV_EXTRA_LICENCES }}'.split()) <= l, l" &&
205216
python -m pytest -q -o addopts=''
206217
tests/vectordb/test_cuvs_config.py
207218
tests/vectordb/test_cuvs_index.py

‎patches/openviking/0.4.21/0002-pyproject-relax-tree-sitter-pins-that-have-no-riscv64-.patch‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
2+
From: Ludovic Henry <git@ludovic.dev>
3+
Date: Fri, 9 Oct 2026 00:00:00 +0000
4+
Subject: [PATCH] pyproject: relax tree-sitter pins that have no riscv64 wheel
5+
6+
0.4.21 replaced the parsing stack's lower bounds with exact pins. Four of
7+
those exact versions were never published as riscv64 wheels, so
8+
installing the built wheel with binary-only resolution fails:
9+
10+
tree-sitter==0.25.2 (riscv64 wheels start at 0.26.0)
11+
tree-sitter-rust==0.24.0 (riscv64 wheels start at 0.24.2)
12+
tree-sitter-c-sharp==0.23.1 (riscv64 wheels start at 0.23.5)
13+
tree-sitter-language-pack==1.13.3 (riscv64 wheels start at 1.15.8)
14+
15+
Turn those four back into lower bounds (language-pack keeps the <2.0 cap
16+
it had before 0.4.21) so the resolver can pick the oldest riscv64 wheel.
17+
The other exact pins already have riscv64 wheels and are left alone.
18+
19+
Upstream-Status: Inappropriate [riscv64 wheel availability is specific to this port]
20+
21+
Signed-off-by: Ludovic Henry <git@ludovic.dev>
22+
---
23+
diff --git a/pyproject.toml b/pyproject.toml
24+
index 0000000..0000000 100644
25+
--- a/pyproject.toml
26+
+++ b/pyproject.toml
27+
@@ -61,15 +61,15 @@ dependencies = [
28+
"litellm>=1.83.7,<1.91.2",
29+
"python-multipart>=0.0.31",
30+
# Pin the parsing stack to avoid unvalidated upstream API and grammar changes.
31+
- "tree-sitter==0.25.2",
32+
+ "tree-sitter>=0.25.2",
33+
"tree-sitter-python==0.25.0",
34+
"tree-sitter-javascript==0.25.0",
35+
"tree-sitter-typescript==0.23.2",
36+
"tree-sitter-java==0.23.5",
37+
"tree-sitter-cpp==0.23.4",
38+
- "tree-sitter-rust==0.24.0",
39+
+ "tree-sitter-rust>=0.24.0",
40+
"tree-sitter-go==0.25.0",
41+
- "tree-sitter-c-sharp==0.23.1",
42+
+ "tree-sitter-c-sharp>=0.23.1",
43+
"tree-sitter-php==0.24.1",
44+
"tree-sitter-lua==0.5.0",
45+
# OpenTelemetry
46+
@@ -85,7 +85,7 @@ dependencies = [
47+
"mcp>=1.27,<2",
48+
"pathspec>=1.1.1",
49+
"grep-ast==0.9.0",
50+
- "tree-sitter-language-pack==1.13.3",
51+
+ "tree-sitter-language-pack>=1.13.3,<2.0",
52+
]
53+
54+

‎patches/openviking/0.4.22/0002-pyproject-relax-tree-sitter-pins-that-have-no-riscv64-.patch‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
2+
From: Ludovic Henry <git@ludovic.dev>
3+
Date: Fri, 9 Oct 2026 00:00:00 +0000
4+
Subject: [PATCH] pyproject: relax tree-sitter pins that have no riscv64 wheel
5+
6+
0.4.21 replaced the parsing stack's lower bounds with exact pins. Four of
7+
those exact versions were never published as riscv64 wheels, so
8+
installing the built wheel with binary-only resolution fails:
9+
10+
tree-sitter==0.25.2 (riscv64 wheels start at 0.26.0)
11+
tree-sitter-rust==0.24.0 (riscv64 wheels start at 0.24.2)
12+
tree-sitter-c-sharp==0.23.1 (riscv64 wheels start at 0.23.5)
13+
tree-sitter-language-pack==1.13.3 (riscv64 wheels start at 1.15.8)
14+
15+
Turn those four back into lower bounds (language-pack keeps the <2.0 cap
16+
it had before 0.4.21) so the resolver can pick the oldest riscv64 wheel.
17+
The other exact pins already have riscv64 wheels and are left alone.
18+
19+
Upstream-Status: Inappropriate [riscv64 wheel availability is specific to this port]
20+
21+
Signed-off-by: Ludovic Henry <git@ludovic.dev>
22+
---
23+
diff --git a/pyproject.toml b/pyproject.toml
24+
index 0000000..0000000 100644
25+
--- a/pyproject.toml
26+
+++ b/pyproject.toml
27+
@@ -61,15 +61,15 @@ dependencies = [
28+
"litellm>=1.83.7,<1.91.2",
29+
"python-multipart>=0.0.31",
30+
# Pin the parsing stack to avoid unvalidated upstream API and grammar changes.
31+
- "tree-sitter==0.25.2",
32+
+ "tree-sitter>=0.25.2",
33+
"tree-sitter-python==0.25.0",
34+
"tree-sitter-javascript==0.25.0",
35+
"tree-sitter-typescript==0.23.2",
36+
"tree-sitter-java==0.23.5",
37+
"tree-sitter-cpp==0.23.4",
38+
- "tree-sitter-rust==0.24.0",
39+
+ "tree-sitter-rust>=0.24.0",
40+
"tree-sitter-go==0.25.0",
41+
- "tree-sitter-c-sharp==0.23.1",
42+
+ "tree-sitter-c-sharp>=0.23.1",
43+
"tree-sitter-php==0.24.1",
44+
"tree-sitter-lua==0.5.0",
45+
# OpenTelemetry
46+
@@ -85,7 +85,7 @@ dependencies = [
47+
"mcp>=1.27,<2",
48+
"pathspec>=1.1.1",
49+
"grep-ast==0.9.0",
50+
- "tree-sitter-language-pack==1.13.3",
51+
+ "tree-sitter-language-pack>=1.13.3,<2.0",
52+
]
53+
54+

‎patches/openviking/0.4.23/0002-pyproject-relax-tree-sitter-pins-that-have-no-riscv64-.patch‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
2+
From: Ludovic Henry <git@ludovic.dev>
3+
Date: Fri, 9 Oct 2026 00:00:00 +0000
4+
Subject: [PATCH] pyproject: relax tree-sitter pins that have no riscv64 wheel
5+
6+
0.4.21 replaced the parsing stack's lower bounds with exact pins. Four of
7+
those exact versions were never published as riscv64 wheels, so
8+
installing the built wheel with binary-only resolution fails:
9+
10+
tree-sitter==0.25.2 (riscv64 wheels start at 0.26.0)
11+
tree-sitter-rust==0.24.0 (riscv64 wheels start at 0.24.2)
12+
tree-sitter-c-sharp==0.23.1 (riscv64 wheels start at 0.23.5)
13+
tree-sitter-language-pack==1.13.3 (riscv64 wheels start at 1.15.8)
14+
15+
Turn those four back into lower bounds (language-pack keeps the <2.0 cap
16+
it had before 0.4.21) so the resolver can pick the oldest riscv64 wheel.
17+
The other exact pins already have riscv64 wheels and are left alone.
18+
19+
Upstream-Status: Inappropriate [riscv64 wheel availability is specific to this port]
20+
21+
Signed-off-by: Ludovic Henry <git@ludovic.dev>
22+
---
23+
diff --git a/pyproject.toml b/pyproject.toml
24+
index 0000000..0000000 100644
25+
--- a/pyproject.toml
26+
+++ b/pyproject.toml
27+
@@ -61,15 +61,15 @@ dependencies = [
28+
"litellm>=1.83.7,<1.91.2",
29+
"python-multipart>=0.0.31",
30+
# Pin the parsing stack to avoid unvalidated upstream API and grammar changes.
31+
- "tree-sitter==0.25.2",
32+
+ "tree-sitter>=0.25.2",
33+
"tree-sitter-python==0.25.0",
34+
"tree-sitter-javascript==0.25.0",
35+
"tree-sitter-typescript==0.23.2",
36+
"tree-sitter-java==0.23.5",
37+
"tree-sitter-cpp==0.23.4",
38+
- "tree-sitter-rust==0.24.0",
39+
+ "tree-sitter-rust>=0.24.0",
40+
"tree-sitter-go==0.25.0",
41+
- "tree-sitter-c-sharp==0.23.1",
42+
+ "tree-sitter-c-sharp>=0.23.1",
43+
"tree-sitter-php==0.24.1",
44+
"tree-sitter-lua==0.5.0",
45+
# OpenTelemetry
46+
@@ -85,7 +85,7 @@ dependencies = [
47+
"mcp>=1.27,<2",
48+
"pathspec>=1.1.1",
49+
"grep-ast==0.9.0",
50+
- "tree-sitter-language-pack==1.13.3",
51+
+ "tree-sitter-language-pack>=1.13.3,<2.0",
52+
]
53+
54+

0 commit comments

Comments
 (0)