Skip to content

refactor: extract stores and conformance tests into zarr-storage - #325

Draft
d-v-b wants to merge 2 commits into
mainfrom
codex/extract-zarr-storage
Draft

refactor: extract stores and conformance tests into zarr-storage#325
d-v-b wants to merge 2 commits into
mainfrom
codex/extract-zarr-storage

Conversation

@d-v-b

@d-v-b d-v-b commented Sep 6, 2026

Copy link
Copy Markdown
Owner

🤖 AI text below 🤖

Extract the existing storage layer into zarr-storage, with its current APIs in zarr_storage.legacy. The package includes the Store ABC and byte request types; MemoryStore, ManagedMemoryStore, GpuMemoryStore, LocalStore, ZipStore, FsspecStore, and ObjectStore; WrapperStore, LoggingStore, and LatencyStore; StorePath and store utilities; and experimental CacheStore.

Context for this extraction:

These are design and migration references; this draft is an initial extraction and does not complete the proposed API redesign or remove the remaining dependency on zarr.

Include the complete store test suite and cache-store tests, redirected to the extracted implementations. Distribute StoreTests, state machines, strategies, buffer test utilities, and assertion helpers under zarr_storage.testing, with an optional testing extra. LatencyStore is separated from the pytest-dependent module so it can be used without pytest.

This remains an initial extraction: buffers, configuration, concurrency, and some metadata/sync helpers still come from zarr. The package must not become a Zarr dependency until this reverse dependency is removed. Zarr runtime imports remain unchanged, and the extracted classes have distinct identities. Array/group integration tests use a documented, test-only re-export fixture to exercise real Zarr arrays with the extracted stores; normal imports do not alter Zarr's bindings. No new storage contract or deprecation is introduced.

Validation:

  • Built wheel, Python 3.12 with optional backends: 1,030 passed, 209 skipped, including mock-S3 tests.
  • Built wheel, Python 3.14 minimal: 847 passed, 193 skipped.
  • Source, Python 3.12 minimal: 847 passed, 193 skipped.
  • Opt-in stateful tests: 3 passed, 3 upstream capability skips.
  • Wheel and source distribution builds pass. Signature parity, normal-import isolation, and pytest-free LatencyStore imports are covered.

CI covers source and built-wheel tests on Python 3.12/3.14, with a Python 3.12 optional-backend job and an explicit stateful test step. GPU tests skip without appropriate hardware, and existing unsupported-backend-operation skips are preserved.

@d-v-b d-v-b changed the title refactor: extract legacy storage interfaces into zarr-storage refactor: extract stores and conformance tests into zarr-storage Sep 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant