Skip to content

Commit 83935e2

Browse files
committed
Count valid rows from the validity array's Sum statistic
`valid_count` uses an exact cached `NullCount` first (`len - null count`). Otherwise, for array-backed validity, it now reads the validity array's `Sum` statistic, which is the number of valid rows and can be cached or computed by an encoding kernel, as develop did with the `sum` aggregate. It counts the bitmap only when the session registers no `Sum` aggregate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ps4mp7KF5c4p66aoSoXx5j Signed-off-by: Claude <noreply@anthropic.com>
1 parent 912f404 commit 83935e2

1 file changed

Lines changed: 10 additions & 1 deletion

File tree

‎vortex-array/src/array/erased.rs‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -360,7 +360,16 @@ impl ArrayRef {
360360
let count = match self.validity()? {
361361
Validity::NonNullable | Validity::AllValid => len,
362362
Validity::AllInvalid => 0,
363-
Validity::Array(a) => a.execute::<BoolArray>(ctx)?.to_bit_buffer().true_count(),
363+
// The number of valid rows is the number of `true` values in the validity array, which
364+
// its `Sum` statistic gives. Count the bitmap if the session has no `Sum` aggregate.
365+
Validity::Array(a) => match a
366+
.statistics()
367+
.compute_stat(Stat::Sum, ctx)?
368+
.and_then(|sum| sum.as_primitive().as_::<usize>())
369+
{
370+
Some(count) => count,
371+
None => a.execute::<BoolArray>(ctx)?.to_bit_buffer().true_count(),
372+
},
364373
};
365374
vortex_ensure!(count <= len, "Valid count exceeds array length");
366375

0 commit comments

Comments
 (0)