perf(array): answer a constant IN list by probing a set, and falsify it by interval by bjchambers · Pull Request #95 · spiceai/vortex

bjchambers · 2026-09-11T18:30:12Z

Addresses spiceai/spiceai#11336. The spiceai/spiceai side is a pin bump and
cannot land until this merges.

What was wrong

A constant IN (...) list arrives at Vortex as list_contains(lit(list), col).
Two separate costs grew with the list length M, and neither had anything to do
with how many rows were being read.

Per batch. Membership was answered by building one full-length Eq array per
list element and OR-reducing them. A query paid one pass over every row for every
element of the list — O(N x M) where DataFusion's own InListExpr is O(N).

Per file. Deriving the statistics predicate was worse than linear. The
falsifier emitted an AND of one term per element, and the optimize pass that
runs over it compares every pair of conjuncts, so deciding whether a file could
be skipped at all cost O(M^2) — paid per file and again per zone map, even for
a file that emits one row.

What this changes

The kernel probes a set. The list is keyed once and probed in a single pass
over the needles, for primitive, Utf8 and Binary needles. Below four elements
a comparison still wins outright, so short lists keep the OR-of-equalities form —
which is also what answers everything the probe does not cover.

The set's equality is the kernel's own: NativePType::is_eq, which compares
floats by their bits. NaN therefore matches itself and -0.0 does not match
0.0, which are the answers Operator::Eq gives. A set keyed on the value would
disagree with both, so the key is a TotalKey<T> newtype rather than T.

The falsifier describes the list as intervals. A single top-level OR: the
scope lies wholly outside the list's range, or wholly inside one of the gaps
between two adjacent sorted values — nothing orders between an adjacent pair, so
a row strictly between them cannot be in the list. The gaps are what make a
clustered list prunable at all: a list split into a low group and a high group
leaves most of a column inside its overall range, where the outer bounds prove
nothing. Staying one OR is also what keeps the pairwise search off the
derivation path.

FSST answers IN in compressed space. This adds the first implementation of
ListContainsElementKernel, which had none. Compression under a fixed symbol
table is deterministic and lossless, so two values are equal exactly when their
codes are — the same property compare_fsst_constant already relies on to answer
Eq. Compressing the list once turns membership into a test over the codes, so a
compressed string column is never decompressed to answer IN.

Extension needles are answered through their storage. An extension value
is its storage value, and Operator::Eq already compares extensions that way,
so a timestamp or date IN list becomes an integer one. Without this the most
common IN column type after integers and strings silently took the slow path,
even though Binary already had the same unwrap kernel for Eq.

VarBin is probed where it lies. VarBin holds a byte buffer and a run of
offsets — everything the probe needs — but it is not canonical, so reaching it
through the generic path first built a VarBinView: sixteen bytes of view per
row, to reach bytes that are already contiguous. For short values that costs more
than the comparison it serves. FSST's codes arrive in exactly this shape.

The list scalar is borrowed, not rebuilt. ListScalar::element_values returns
the element values in place. The probe reads them once per batch, and
materializing a Vec<Scalar> to do so cost more than the set built from it.

Correctness

vortex-array/tests/in_list_differential.rs records 39 rows of answers produced
by running it against the commit before this one, and asserts them. It covers each
needle type on both sides of the probe threshold, NaN, both signed zeros,
strings too long to inline in their view, nullable and Binary needles, integer
widths other than i64, lists holding duplicates, a null list element, and the
empty list. A change that makes list_contains faster leaves the table untouched;
one that makes it answer differently fails here.

vortex-layout/tests/list_contains_pruning.rs checks the property the interval
form rests on, which no test previously reached: for each case it derives the
predicate, prunes a zone map built from the zones' own statistics, and requires
every excluded zone to hold zero matching rows. It covers floats, string bounds
stored wider than the zone's values the way truncation stores them, duplicate
elements, the list lengths either side of the cap on interior gaps, and an
exhaustive sweep of every list over a small domain against every zone alignment.

Negative controls, run to confirm those tests can fail:

mutation	result
gap bounds made non-strict	5 of 6 pruning cases fail
list sorted the other way	the same 5 fail
outer range test made non-strict	2 fail
FSST kernel drops a matching list element	all 5 affected FSST cases fail

3028 vortex-array tests, 90 vortex-fsst, and the two test files above all
pass. (vortex-layout's layouts::table tests fail identically on the base
commit and on this one.)

Performance

Measured on a real spiced over the SQL endpoint against a 1,048,576-row Cayenne
table, base and this branch running concurrently on separate ports, interleaved
per repetition with the leading port alternated. The results cache is off and a
results-cache-status: HIT voids the run; every response's row count and payload
sum are asserted, so a failed or wrong query cannot be reported as a fast one. A
no-filter control the change cannot affect bookends the run and read 1.02 / 1.04
/ 1.01 / 0.98. Medians below; minima agree to within a few percent everywhere.

The columns the sweep filters on are stored as fastlanes.bitpacked for the
i64, vortex.fsst for the utf8, and vortex.alp for the f64 — read off
the written files' encoding trees, not assumed. So the utf8 arms do exercise
the FSST kernel: scanning one of those files with a pushed-down
label IN (64 values) filter runs it 8 times for the 17 rows it matches.

IN, query time base -> this branch:

M	`i64`	`utf8`	`f64`
32	9.8 -> 9.5 ms (1.03x)	18.3 -> 9.5 ms (1.92x)	12.3 -> 9.7 ms (1.27x)
150	17.4 -> 10.0 ms (1.73x)	51.4 -> 11.2 ms (4.58x)	26.2 -> 10.6 ms (2.47x)
512	39.5 -> 12.6 ms (3.14x)	162.5 -> 13.7 ms (11.84x)	70.5 -> 13.5 ms (5.21x)
2048	165.4 -> 21.7 ms (7.64x)	647.9 -> 24.6 ms (26.29x)	248.8 -> 23.1 ms (10.77x)
8192	1.68 s -> 57.4 ms (29.24x)	4.05 s -> 69.5 ms (58.18x)	986.6 -> 63.3 ms (15.58x)
32768	44.57 s -> 225.4 ms (197.75x)

NOT IN over the same lists: 1.05x / 1.48x / 2.09x / 3.64x / 4.78x / 6.16x for
i64, 1.78x -> 32.97x for utf8, 1.10x -> 10.43x for f64. It gains less at the
top because nearly every row survives it, so the scan is decode-bound rather than
kernel-bound — the membership test stops being the thing the query is waiting for.

The base cost is user-visible, not theoretical: a 32768-element IN list took
44.6 seconds on a table of a million rows.

Below the threshold, and just above it:

M	`i64`	`utf8`	`f64`
2	1.04x	1.01x	0.98x
3	1.06x	1.02x	0.91x
4	1.02x	1.18x	0.97x
5	0.94x	1.13x	1.00x
8	0.96x	1.25x	1.01x
16	1.02x	1.47x	1.09x
24	1.01x	1.72x	1.18x
32	1.13x	1.77x	1.23x
48	1.16x	2.31x	1.32x
64	1.22x	2.68x	1.51x

M=2 and M=3 are below the probe threshold, so the kernel runs the same code
on both builds and those rows read the noise floor of a 9 ms query: they scatter
0.91x to 1.06x. Every reading below parity above them sits inside that floor, and
the trend from M=16 upward is monotone, so the threshold needs no adjustment.
The query at these sizes is dominated by fixed per-query cost rather than by the
kernel, which is why the crossover itself is not resolvable here and is taken
from the kernel benchmark instead.

The FSST kernel, on its own

Compressing the list is paid once per batch while the decompression it avoids is
paid per row, so which wins is a ratio rather than a size. The kernel declines
below 32 rows per list element for that reason. Toggling its registration on and
off over an 8192-row batch, median of 51:

M	short words	URLs
4	2.03x	3.64x
8	2.02x	3.56x
32	1.81x	3.20x
128	1.42x	2.17x
256	1.14x	1.63x
512	1.00x	0.99x
8192	1.00x	0.99x

Wins where it applies, parity where it declines. Two things had to be fixed to
get there, both found by measurement rather than reading: without the VarBin
probe the canonicalization alone was 65% of the call and the kernel was a loss
of 1.11x to 2.73x on the short column at every length; and without the ratio
guard, re-compressing a long list every batch cost more than the decompression
it saved (6.2x at 8192 elements).

Deriving the predicate

Two rewrite rules are registered per scalar function and differ only in the
NaN proof they apply, so for any given needle one of them commonly emits nothing
— after having built a predicate it then discards. Resolving the guard before the
list is touched rather than after removes that:

M	before	after
1024	126.6 us	32.5 us	3.90x
8192	647.8 us	221.9 us	2.92x
65536	2.77 ms	1.76 ms	1.57x

Where the remaining gap is

The same kernel benchmark measures DataFusion's own hashed InListExpr over an
identical batch, which is the floor this path is trying to reach. Per 8192-row
batch of i64 needles:

M	`list_contains`	`InListExpr`
4	8.79 us	9.34 us
32	9.41 us	11.33 us
150	12.44 us	10.67 us
2048	23.13 us	11.64 us
8192	67.98 us	10.57 us

Up to a few dozen elements the probe is now the faster of the two. Past that it
falls behind, and by a factor of 6.4 at 8192, for one reason: InListExpr keys
its set once when the expression is constructed, while a kernel handed the list
as a value has nowhere to keep one and so rebuilds it for every batch. That
rebuild is the whole of the remaining distance, and closing it needs somewhere to
cache the set across batches rather than a faster probe.

Not in this change

Decimal needles still take the OR-of-equalities form. A decimal's physical
width is not fixed by its logical dtype, so two equal decimals can arrive in
different DecimalValue variants and a set keyed on one would not find the
other; keying it needs a normalizing cast first. Additive, and not a special
case in the code — the probe covers what it covers and the definition form
answers the rest. Boolean and nested needles stay on that form deliberately:
an IN list of booleans never reaches the probe threshold, and a struct or
list needle has no cheap key.
Dict needles have no kernel, so a dictionary-encoded column is expanded to
its values before the probe. The compare path already has the template for
this (evaluate against values(), rewrap the codes, canonicalize), and a
low-cardinality column would answer an IN list by probing its dictionary
rather than its rows. Measured at only 3.5 us over an 8192-row batch with a
64-value dictionary, so it is worth doing on its own merits rather than
urgently.
The set is rebuilt for every batch. DataFusion's InListExpr keys its
set once when the expression is constructed; a kernel handed the list as a
child array has nowhere to keep one. That is the whole of the remaining
distance in the table above, and it is worth under 4% for any list a person
would write by hand — the rebuild only becomes material past roughly 256
elements (-24.9% at 1024, -67.5% at 8192, measured by handing a prebuilt set
into the same entry point).
list_contains_falsify still runs twice per Expression::falsify, once
for each of the two NaN-proof rewrite rules, and for a float needle both of
them still do the whole job. The hoisted guard only short-circuits a rule that
cannot emit at all, which is every non-float needle — measured, the residual
duplication is 1.07-1.20x for i64 against 2.02-2.28x for f64. The rules'
outputs are OR-ed, and both carry the same value predicate under different
guards, so (g1 and V) or (g2 and V) could be emitted once as (g1 or g2) and V. That is a change to how the rules are registered rather than to the
falsifier, and the predicate is derived once per layout and cached, so it is
left out of this change.

…it by interval A constant `IN (...)` list was answered by building one full-length `Eq` array per list element and OR-reducing them, so a query paid one pass over every row for every element of the list. Deriving the statistics predicate was worse than linear: it emitted an `AND` of one term per element, and the optimize pass compares every pair of conjuncts, so the cost of *deciding whether a file could be skipped* grew with the square of the list length, regardless of how many rows the file held. Membership is now answered by keying a set on the list once and probing it in a single pass over the needles, for primitive, `Utf8` and `Binary` needles. The set's equality is the kernel's own — `NativePType::is_eq`, which compares floats by their bits, so `NaN` matches itself and `-0.0` does not match `0.0`, the answers the OR-of-equalities form gives. Below four elements a comparison still wins outright, so short lists keep that form, which is also what answers everything the probe does not cover. The statistics predicate becomes a single top-level `OR` describing the list as intervals: the scope lies wholly outside the list's range, or wholly inside one of the gaps between two adjacent sorted values, since nothing orders between an adjacent pair. The gaps are what make a clustered list prunable at all — a list split into a low group and a high group leaves most of a column inside its overall range, where the outer bounds prove nothing. Staying a single `OR` is also what keeps the pairwise `between` search off the derivation path. FSST gets a `ListContainsElementKernel`, the first implementation of that trait: compression under a fixed symbol table is deterministic and lossless, so two values are equal exactly when their codes are. Compressing the list once therefore answers `IN` over compressed strings without decompressing any of them — the same property `compare_fsst_constant` already relies on for `Eq`. `ListScalar::element_values` borrows the element values instead of materializing a `Vec<Scalar>`, which the probe pays once per batch. Evidence -------- `vortex-layout/tests/list_contains_pruning.rs` checks the property that makes the interval form safe: for each case it derives the predicate, prunes a zone map built from the zones' own statistics, and requires every excluded zone to hold zero matching rows. It covers floats (where `NaN` and the two signed zeros have positions under the total order), string bounds stored wider than the zone's values as truncation stores them, duplicate elements, the list lengths either side of the gap cap, and an exhaustive sweep over a small domain. Making the gap bounds non-strict fails 5 of its 6 cases; sorting the list the other way fails the same 5; loosening the outer range test fails 2. `vortex-array/tests/in_list_differential.rs` asserts 39 rows of answers recorded by running it against the commit before this one, so a change that makes `list_contains` faster leaves the table untouched and one that makes it answer differently fails.

…pelling The pair differed only in a final byte, to show that a code comparison stopping at a shared prefix would answer both alike. `strinX` carried that property but reads as a misspelling of `string`.

Copilot

🟡 Changes recommended

The extension kernel can panic for nullable lists containing null, and the FSST tests do not reach the new kernel.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Optimizes constant IN evaluation and zone pruning, addressing spiceai/spiceai#11336.

Changes:

Adds hash-set probing for primitive and binary/string arrays.
Rewrites pruning predicates as bounded interval gaps.
Adds FSST and extension-storage kernels plus regression tests.

Checks: Static code review only; tests were not run in the review environment.

File summaries

File	Description
`vortex-layout/tests/list_contains_pruning.rs`	Tests interval-pruning soundness.
`vortex-array/tests/in_list_differential.rs`	Pins existing membership semantics.
`vortex-array/src/stats/rewrite/builtins.rs`	Implements interval-based falsification.
`vortex-array/src/scalar/typed_view/list.rs`	Exposes borrowed list elements.
`vortex-array/src/scalar_fn/fns/list_contains/mod.rs`	Implements set-based membership probing.
`vortex-array/src/scalar_fn/fns/binary/compare/bytes.rs`	Reuses shared resolved views.
`vortex-array/src/arrays/varbinview/views_side.rs`	Adds resolved byte-view helper.
`vortex-array/src/arrays/varbinview/mod.rs`	Registers the view helper.
`vortex-array/src/arrays/extension/vtable/kernel.rs`	Registers extension membership kernel.
`vortex-array/src/arrays/extension/compute/mod.rs`	Enables extension implementation.
`vortex-array/src/arrays/extension/compute/list_contains.rs`	Delegates membership to extension storage.
`encodings/fsst/src/kernel.rs`	Registers FSST membership kernel.
`encodings/fsst/src/compute/mod.rs`	Enables FSST implementation.
`encodings/fsst/src/compute/list_contains.rs`	Probes compressed FSST codes.

Review details

Files reviewed: 14/14 changed files
Comments generated: 3
Review effort level: Balanced

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

An extension list and a needle only have to agree on their dtype ignoring nullability, so a nullable list can reach the extension kernel against a non-nullable column. A null element there has no storage value to unwrap, and building one panicked in `Scalar::new_unchecked`. Such a list now goes back to the generic path, which answers it, matching what the FSST kernel already does. The FSST tests were not reaching the kernel they exist for. It declines below a ratio of rows to list elements, and the fixture was eight rows, so a five-element list needed a hundred and sixty before the kernel would engage. The fixture now fills enough rows for the longest list the tests use, and asserts the ratio it needs rather than leaving it implicit. Removing a matching element from the compressed list now fails five of the seven; the two that still pass are the two that decline before compressing. The probe walks valid rows a word at a time instead of testing validity per row. Measured against each other in one binary over an 8192-row batch, the word-at-a-time form wins at every density: 1.91x at fully valid, where it fast-paths whole words, through 4.06x at a quarter valid to 28x at 0.2%.

bjchambers · 2026-09-11T18:52:03Z

Thanks — all three were real, and two of them were the important kind. Fixed in af1c5b3.

The extension kernel panic. Reproduced before fixing, exactly as described:

Scalar::new_unchecked called with incompatible dtype and value:
  Other error: non-nullable dtype i64 cannot hold a null value

A nullable list does reach the kernel against a non-nullable column, because the
dtype check ignores nullability. Such a list now goes back to the generic path,
which answers it — the same thing the FSST kernel already does with a null
element. The repro is kept as a test.

The FSST tests were not reaching the kernel. Correct, and the cause is worth
recording: the tests were written and their negative control run before the
row-to-element ratio guard existed, and I never re-ran the control afterwards.
With the guard in place the eight-row fixture needed a hundred and sixty rows for
a five-element list, so everything fell through to the generic path.

The fixture now cycles to enough rows for the longest list the tests use, and
asserts that ratio rather than leaving it implicit, so the tests fail loudly if
the guard moves again. Re-running the control — removing a matching element from
the compressed list — now fails five of the seven; the two that still pass are
the two that decline before compressing (dtype mismatch, empty list). Before this
change that same control failed none of them.

non_nullable_needles was also building its own short column, so it went through
the same fallback and skipped the compressed-equals-uncompressed assertion the
others have. It now shares the helper and gets both.

The per-row validity check. Confirmed, and by a wider margin than I expected.
Measuring the two forms against each other in one binary, interleaved, over an
8192-row batch:

valid	per-row	word-at-a-time
100%	5.96 us	3.12 us	1.91x
50%	5.71 us	2.50 us	2.28x
25%	5.58 us	1.38 us	4.06x
6.2%	6.25 us	0.50 us	12.50x
0.2%	4.67 us	0.17 us	27.95x

It wins even at fully valid, where it fast-paths whole words. The probe now takes
a row index and fills an unset buffer through for_each_set_index.

Two failing checks on this PR are not from this change, for the record:
License Check and Audit Check (advisories) is RUSTSEC-2026-0258 in a transitive
dependency, and Compat check / compat-test fails on vortex.onpair being an
unknown encoding plus one fixture-set mismatch. This diff touches no manifest and
no fixture, and both checks fail the same way on an unrelated dependency-bump PR.

Copilot

🔵 Needs a closer look

Correctness-sensitive pruning and encoded execution paths warrant final human review despite strong targeted coverage.

Review details

Files reviewed: 14/14 changed files
Comments generated: 0 new
Review effort level: Balanced

…kernel The extension kernel rebuilt the whole list as storage `Scalar`s for every batch, then wrapped them in a constant array and a fresh `ListContains` to recurse. Measured against the same query over the storage type directly, that marshalling cost ~12ns an element and did not vary with batch size: +88% at a thousand-element list over an 8192-row batch, +155% at sixty-five thousand. An extension value is its storage value — `ScalarValue` has no extension variant — so the generic probe can key on the list's values unchanged once the needle is unwrapped. Moving it there drops the per-batch cost to nothing, and the dtype check the kernel had to re-derive for itself is one the generic path has already done: executing as a parent kernel is what skipped it. It also removes the `Scalar::new_unchecked` that retagged each element with the column's storage dtype. That call had to assume the two dtypes agreed all the way down, while the check above it compares them ignoring nullability, so a list whose storage was a struct with a nullable field could be retagged against a column whose field was not. Nothing constructs a scalar now, and a storage type the probe does not cover declines to the equality form as before.

Copilot

🟢 Approval recommended

The optimized paths preserve dtype, validity, equality, and pruning invariants with comprehensive regression coverage.

Review details

Files reviewed: 11/11 changed files
Comments generated: 0 new
Review effort level: Balanced

Stable rustfmt warns that `group_imports` and `imports_granularity` are nightly-only and formats without them, so a local `cargo fmt` leaves import order untouched while CI enforces it.

Copilot

🔵 Needs a closer look

Null-bearing lists currently perform avoidable canonicalization, allocation, and compression before falling back.

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

encodings/fsst/src/compute/list_contains.rs:80

This loop may compress every preceding list value before a trailing null makes the kernel decline, after which the generic fallback repeats list processing. Since any null element unconditionally disqualifies this kernel, detect it before obtaining the compressor and building codes to avoid wasted compression for nullable IN lists.
vortex-array/src/scalar_fn/fns/list_contains/mod.rs:286
A null list element makes every probe path return None, but this is checked only after primitive needles have already been canonicalized (and after the byte set may have been allocated and populated). The caller then runs the full equality fallback, so nullable constant lists pay this extra decode/allocation on top of the old path. Reject null-bearing lists before dispatching on the needle dtype.

Files reviewed: 11/11 changed files
Comments generated: 0 new
Review effort level: Balanced

A null element is not a key on any probe path, so such a list goes back to OR-ing one equality per element. Both probes established that only once they were already committed: the primitive path after canonicalizing the needles, the byte path after allocating and populating the set, and the FSST kernel after compressing every element preceding the null. The caller then ran the equality form anyway, so a nullable list paid all of that on top of the work it did before the probe existed — the one shape this change made slower. The check now happens once, on the borrowed element values, before anything is decoded, allocated or compressed. Answers are unchanged; the differential table's null-element row is what says so.

Copilot

🔵 Needs a closer look

The cross-crate execution and pruning changes are correctness-sensitive and warrant final human validation.

Review details

Files reviewed: 11/11 changed files
Comments generated: 0 new
Review effort level: Balanced

spiceai/vortex#95 has merged, so the pin moves from that pull request's branch to the commit it landed as on `spiceai-54`, and the ledger row drops the marker that was refusing to let the branch pin land. The merge commit contains the reviewed branch head rather than a squash of it, so the patch audit the ledger asks for is the same one recorded when the branch was pinned: `spiceai-54` remains a strict ancestor, nothing is a re-cut, and no fork patch can have been dropped.

… it by interval (spiceai#14061) * perf(vortex): pick up the constant IN list probe and interval falsifier Moves the vortex pin onto spiceai/vortex#95, which answers a constant `IN (...)` list by probing a set instead of building one full-length `Eq` array per element, and derives the statistics predicate as a handful of intervals instead of one term per element. Both costs grew with the length of the list and neither had anything to do with how many rows were read: the kernel paid one pass over every row for every element, and deriving the predicate was worse than linear, because the optimize pass that runs over a wide `AND` compares every pair of conjuncts — so deciding whether a file could be skipped at all cost more than reading it, for a file that might emit one row. Measured on a real spiced over the SQL endpoint against a 1,048,576-row Cayenne table, both builds concurrent on separate ports and interleaved per repetition, results cache off with a cache hit voiding the run, row counts and payload sums asserted on every response, and a no-filter control the change cannot affect bookending the run at 1.02/1.07/1.02/0.92: IN, M=8192 i64 1.68 s -> 57.4 ms 29.2x utf8 4.05 s -> 69.5 ms 58.2x f64 987 ms -> 63.3 ms 15.6x IN, M=32768 i64 44.57 s -> 225.4 ms 197.8x The cost was user-visible rather than theoretical: a 32768-element `IN` list took forty-five seconds on a table of a million rows. The pin points at the pull request's branch so this can be reviewed alongside it; it moves to the merge commit before this lands. The benches are the instruments those numbers came from. They separate the per-batch kernel cost from the once-per-file predicate cost, which a black-box query time cannot: `in_list_kernel` measures the kernel against DataFusion's own hashed `InListExpr` over an identical batch, `in_list_pruning` the predicate derivation alone, `in_list_pruning_power` what pruning is worth by list shape, and `in_list_query_e2e` the whole query through Cayenne. * build: refuse to land a fork pinned at a pull request's branch A change that spans this repository and one of its forks is reviewed with the pin on the fork's pull request branch, which is the only way to see both halves together. That pin must not land. The branch is deleted when the fork's pull request merges, so trunk would name a revision no branch reaches, and a later clone could not resolve it — the kind of breakage that appears long after the change that caused it, in someone else's build. Nothing stopped that today. The ledger row's branch cell can now carry `(TEMPORARY: <the pull request that has to merge first>)`, and the guard fails while it is present, naming what has to merge and what to do afterwards. A pin on a branch is reviewable and un-landable at the same time, and cannot land by being forgotten. This commit marks the vortex pin, so the guard fails on this branch until spiceai/vortex#95 merges and the pin moves to the merge commit. That failure is the check working. * test(vortex): keep the balanced OR tree guard reaching the tree The pin this branch moves sends a primitive `IN` list of four elements or more through a set probe, so the eight-thousand-element `INT` list in `test_large_in_list_filter_pushdown_stays_evaluable` no longer builds the right-leaning OR tree whose loss that test exists to catch. The test still passed, on a build that never constructed the tree at all — which is the failure the fork-patch ledger is meant to make impossible. Adds an arm at the same depth over a `DECIMAL` column. The probe does not cover decimals, so that list still goes back to OR-ing one equality per element, and the guard reaches what it guards again. Confirmed by dropping an element from that fallback: the decimal assertion fails, 1023 against 1024, while the integer one above it passes — which is the same evidence that the integer arm no longer reaches it. The ledger row now says which arm keeps the guard live, so the next change that widens the probe knows what it has to leave alone. * build(vortex): move the pin to the merged revision spiceai/vortex#95 has merged, so the pin moves from that pull request's branch to the commit it landed as on `spiceai-54`, and the ledger row drops the marker that was refusing to let the branch pin land. The merge commit contains the reviewed branch head rather than a squash of it, so the patch audit the ledger asks for is the same one recorded when the branch was pinned: `spiceai-54` remains a strict ancestor, nothing is a re-cut, and no fork patch can have been dropped. * style: format the new benches They were written and verified before being formatted; `cargo fmt` is what the sign-off gate checks.

bjchambers added the changelog/performance A performance improvement label Sep 11, 2026

test(fsst): rename two fixture values the spell check reads as a miss…

aff6635

…pelling The pair differed only in a final byte, to show that a code comparison stopping at a shared prefix would answer both alike. `strinX` carried that property but reads as a misspelling of `string`.

bjchambers requested a balanced review from Copilot September 11, 2026 18:33

Copilot started reviewing on behalf of bjchambers September 11, 2026 18:34 View session

bjchambers marked this pull request as ready for review September 11, 2026 18:34