Skip to content

Commit e9a7997

Browse files
authored
deltalite: add build-deltalite.yml for riscv64 wheels (#1805)
* **Package**: `deltalite` * **Version**: `0.1.7` * **Source**: https://github.com/PostHog/posthog (monorepo, `rust/deltalite`) * **Docs**: https://github.com/PostHog/posthog Compiles deltalite, a pyo3/maturin streaming partition-upsert engine for Delta Lake tables built on the `deltalake` (delta-rs) crate, into an abi3 wheel. Upstream publishes no riscv64 wheel. Mirrors [upstream's `build-deltalite.yml`](https://github.com/PostHog/posthog/blob/f25833286a435bbcc6f4f25a4a34514e9e498628/.github/workflows/build-deltalite.yml). **Differs from upstream** - Sparse-checks-out `rust/` + `LICENSE` instead of the full posthog monorepo - the Cargo workspace only needs that slice. - Copies `LICENSE` into `rust/deltalite/python/` - maturin globs licence files next to `pyproject.toml`, which sits below the repo root. - Builds via `PyO3/maturin-action` targeting `riscv64gc-unknown-linux-gnu` - upstream's matrix has no riscv64 entry. - Does not run upstream's own differential parity suite: `ci-deltalite-python.yml` pins `deltalake`/`pyarrow`/`duckdb` and runs it against them, but uv (0.12.13, riscv64) rejects `deltalake==1.6.3`'s only wheel (`cp310-abi3-manylinux_2_39_riscv64`, correctly served with `data-requires-python: >=3.10`) as having "no usable wheels" whenever it's resolved by name from an index - reproduced with our registry as both the extra index and, separately, the default index, ruling out an index-priority cause. `duckdb`/`pyarrow` (exact `cp312-cp312` tags, no `abi3`) and deltalite's own local-file wheel install both succeed under the same job, isolating the break to uv's abi3-forward-compatibility tag check specifically for a registry-resolved wheel. Of the 8 files under `rust/deltalite/python/tests`, 7 import `deltalake`/`pyarrow`/`duckdb` (directly or via `harness/common.py`) and can't run without them; only `test_planner.py` is self-contained (one of its 18 tests, `test_knob_kwargs_are_accepted_by_a_real_upsert`, also imports `deltalake`/`pyarrow` inline for a real round-trip and is deselected for the same reason; the other 17 pure `plan()` unit tests run). **Matrix**: `cp312-abi3` (built once) - pyo3's `abi3-py312` feature is unconditional in `Cargo.toml`. **Testing** - `test_planner.py`, 17/18 tests (one deselected -- see above -- for the same registry-resolution reason). - Wheel-level: licence-file check. **License**: OK Built on cp312-abi3-manylinux_riscv64; results pending CI.
1 parent 66521af commit e9a7997

1 file changed

Lines changed: 147 additions & 0 deletions

File tree

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
# SPDX-FileCopyrightText: 2026 The RISE Project
2+
# SPDX-License-Identifier: MIT
3+
---
4+
# Based on PostHog/posthog's own release and test workflows for this package:
5+
# https://github.com/PostHog/posthog/blob/f25833286a435bbcc6f4f25a4a34514e9e498628/.github/workflows/build-deltalite.yml
6+
# https://github.com/PostHog/posthog/blob/f25833286a435bbcc6f4f25a4a34514e9e498628/.github/workflows/ci-deltalite-python.yml
7+
name: Build deltalite wheels (riscv64)
8+
9+
on:
10+
workflow_dispatch:
11+
inputs:
12+
version:
13+
description: 'deltalite version to build (e.g. 0.1.7)'
14+
required: true
15+
default: '0.1.7'
16+
pull_request:
17+
paths:
18+
- '.github/workflows/build-deltalite.yml'
19+
20+
concurrency:
21+
group: ${{ github.workflow }}-${{ inputs.version || '0.1.7' }}-${{ github.head_ref || github.run_id }}
22+
cancel-in-progress: true
23+
24+
permissions:
25+
contents: read # to fetch code (actions/checkout)
26+
27+
env:
28+
# `inputs.version` is empty on pull_request events; default to 0.1.7 there.
29+
DELTALITE_VERSION: ${{ inputs.version || '0.1.7' }}
30+
# deltalite lives in the PostHog/posthog monorepo and is never tagged on its
31+
# own; this is the commit whose rust/deltalite/ tree is byte-for-byte the
32+
# 0.1.7 PyPI sdist (verified by diff). Update alongside the version.
33+
DELTALITE_REF: f25833286a435bbcc6f4f25a4a34514e9e498628
34+
CARGO_INCREMENTAL: 0
35+
CARGO_NET_RETRY: 10
36+
RUSTUP_MAX_RETRIES: 10
37+
# Halves concurrent rustc processes on the 4-core riscv64 runner (gotcha 228).
38+
CARGO_BUILD_JOBS: 2
39+
40+
jobs:
41+
setup:
42+
uses: $/.github/workflows/_setup.yml
43+
44+
build_wheels:
45+
needs: [setup]
46+
name: Build deltalite ${{ inputs.version || '0.1.7' }} cp312-abi3-manylinux_riscv64
47+
runs-on: ubuntu-24.04-riscv
48+
timeout-minutes: 1440
49+
50+
steps:
51+
# posthog/posthog is a multi-GB application monorepo; rust/ (plus the
52+
# repo-root LICENSE) is the self-contained slice the Cargo workspace at
53+
# rust/Cargo.toml needs. Cone mode (the actions/checkout default) is
54+
# required here: non-cone sparse-checkout leaves the working tree in a
55+
# shape cargo's own file-walking/exclude-matching can't handle,
56+
# failing with "Failed to update the excludes stack" -- cone mode
57+
# supports mixing a directory pattern (rust) with a top-level file
58+
# pattern (LICENSE) just fine on a modern git.
59+
- name: Checkout PostHog/posthog ${{ env.DELTALITE_REF }}
60+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
61+
with:
62+
repository: PostHog/posthog
63+
ref: ${{ env.DELTALITE_REF }}
64+
sparse-checkout: |
65+
rust
66+
LICENSE
67+
persist-credentials: false
68+
69+
# maturin's licence auto-glob only looks next to pyproject.toml
70+
# (rust/deltalite/python/), not the repo root where LICENSE lives
71+
# (gotcha 146).
72+
- name: Bundle LICENSE into the wheel
73+
run: cp LICENSE rust/deltalite/python/LICENSE
74+
75+
# Same OOM guard build-deltalake.yml needs for a comparably sized Rust
76+
# build (this one links AWS-LC too).
77+
- name: Set swap space
78+
uses: pierotofy/set-swap-space@fc79b3f67fa8a838184ce84a674ca12238d2c761 # master
79+
with:
80+
swap-size-gb: 10
81+
82+
- name: Build wheel
83+
uses: PyO3/maturin-action@e83996d129638aa358a18fbd1dfb82f0b0fb5d3b # v1.51.0
84+
with:
85+
command: build
86+
target: riscv64gc-unknown-linux-gnu
87+
manylinux: '2_39'
88+
# pyo3 carries `abi3-py312` unconditionally in Cargo.toml, so
89+
# upstream ships one cp312-abi3 wheel and no free-threaded variant
90+
# (gotcha 181).
91+
args: >-
92+
-i python3.12 --release --strip --out dist
93+
--manifest-path rust/deltalite/python/Cargo.toml
94+
95+
- uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
96+
with:
97+
python-version: '3.12'
98+
activate-environment: true
99+
enable-cache: false
100+
101+
# ci-deltalite-python.yml's own parity suite installs pinned
102+
# deltalake/pyarrow/duckdb and runs the full rust/deltalite/python/tests
103+
# tree against them, but that isn't reproducible here: uv (0.12.13,
104+
# riscv64) rejects deltalake==1.6.3's only wheel
105+
# (cp310-abi3-manylinux_2_39_riscv64, correctly `data-requires-python:
106+
# >=3.10` on our index) as "has no usable wheels" when resolved BY NAME
107+
# from any index -- confirmed with our registry as both the extra index
108+
# and, separately, as the *default* index (ruling out an index-priority
109+
# cause) -- while duckdb/pyarrow (exact cp312-cp312 tag, no abi3) and
110+
# deltalite's own local-file wheel install both succeed. This isolates
111+
# the break to uv's abi3-forward-compatibility tag check specifically
112+
# for a wheel fetched from a registry rather than a local path (gotcha
113+
# 375); there is no clean workaround from this side. Of the 8 files
114+
# under tests/, 7 import deltalake/pyarrow/duckdb (directly or via
115+
# harness/common.py) and can't run without them -- only test_planner.py
116+
# is self-contained, so it's the one real test file left to run.
117+
# test_planner.py itself has one test that also imports deltalake/
118+
# pyarrow inline (a real DeltaLiteTable.upsert round-trip), deselected
119+
# for the same reason -- the other 17 pure-plan() unit tests still run.
120+
# deltalite's own wheel already installs and licenses correctly on its
121+
# own.
122+
- name: Test wheel
123+
run: |
124+
uv pip install pytest dist/*.whl --force-reinstall
125+
python -c "
126+
import importlib.metadata
127+
files = {str(p) for p in importlib.metadata.files('deltalite') if '.dist-info/' in str(p)}
128+
assert any('LICENSE' in f for f in files), files
129+
"
130+
python -m pytest rust/deltalite/python/tests/test_planner.py -q \
131+
-k "not test_knob_kwargs_are_accepted_by_a_real_upsert"
132+
133+
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
134+
with:
135+
name: deltalite-${{ env.DELTALITE_VERSION }}-cp312-abi3-manylinux_riscv64
136+
path: dist/*.whl
137+
if-no-files-found: error
138+
139+
publish:
140+
name: Publish deltalite ${{ inputs.version || '0.1.7' }}
141+
needs: [setup, build_wheels]
142+
permissions:
143+
contents: write
144+
pull-requests: write
145+
uses: $/.github/workflows/_publish-wheel.yml
146+
with:
147+
artifact-pattern: deltalite-${{ inputs.version || '0.1.7' }}-*-manylinux_riscv64

0 commit comments

Comments
 (0)