Skip to content

Implement a DiskArrays compute backend - #69

Open
meggart wants to merge 6 commits into
mainfrom
fg/diskarraysext
Open

meggart wants to merge 6 commits into
mainfrom
fg/diskarraysext

Conversation

@meggart

@meggart meggart commented Sep 15, 2026

Copy link
Copy Markdown
Owner

No description provided.

DiskArrays now owns the backend type (JuliaIO/DiskArrays.jl#313), so that
it can be a member of its `DynamicBackend` sum type. We only add methods.

- Type `a` in the reduction impls to resolve an ambiguity with the
  `::ComputeBackend` fallbacks in DiskArrays.
- Tests select the backend per array with `withbackend` instead of relying
  on the backend preference, and import `AccessCountDiskArray` from
  `DiskArrays.TestTypes`.
- Temporary `[sources]` entry pointing to the DiskArrays branch.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018ra7EAJeFE1dtekwf4Tzxh
asinghvi17 and others added 2 commits September 20, 2026 13:07
- Qualify `Base.mapreducedim!`, size the output from `ndims` instead of
  assuming 3 dimensions, and start the reference from `zero(R)` instead of
  uninitialized memory.
- `init=0.0` for float data, an Int `init` with `dims` throws in Base too.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018ra7EAJeFE1dtekwf4Tzxh
- Reductions over all dimensions return a scalar like in Base, instead of
  the 1x...x1 array from `compute`. Same for `mapreduce` without `dims`.
- `mapreduce` applied only `op` and never `f`.
- Without `init`, `mapreduce` got DiskArrays' `_InitialValue` sentinel to
  fill the buffers with. Use a neutral element of `op` instead.
- `value(::OnlineStats.Extrema)` is a NamedTuple `(min, max, nmin, nmax)`,
  so `maximum` with `strategy=:reduce` returned `nmax`, a count, and
  `extrema` failed to convert. Use the accessors of the stat. This fixes
  the `@test_broken` "gives all ones for some reason" in the Aggregate tests.
- `extrema` defaults to `strategy=:reduce`, the direct aggregator
  broadcasts its result into the output, which fails for a Tuple.
- Tests: scalar results for `mapreduce` without `dims`, an associative
  `op`, and regression tests for the OnlineStats path.
- Point `[sources]` to the DiskArrays branch with the `count` fallback.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018ra7EAJeFE1dtekwf4Tzxh
@coveralls

coveralls commented Sep 21, 2026

Copy link
Copy Markdown

Coverage Report for CI Build 35607162816

Coverage increased (+4.0%) to 61.916%

Details

  • Coverage increased (+4.0%) from the base build.
  • Patch coverage: 4 uncovered changes across 2 files (26 of 30 lines covered, 86.67%).
  • No coverage regressions found.

Uncovered Changes

File Changed Covered %
src/enginearrays.jl 25 22 88.0%
src/util/aggregate.jl 5 4 80.0%

Coverage Regressions

No coverage regressions found.


Coverage Stats

Coverage Status
Relevant Lines: 2140
Covered Lines: 1325
Line Coverage: 61.92%
Coverage Strength: 2097.24 hits per line

💛 - Coveralls

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants