Skip to content

Commit 91b38f4

Browse files
authored
Describe scheme variants in the versioning docs (#10747)
## Summary The compression section of the versioning design still described per-edition scheme configuration as planned work, and said that the decimal scheme only declares the v1 wire ID. Since #10076, the decimal, bit-packing and frame-of-reference schemes pick their v1 or v2 variant from the allowed wire IDs. The section and the compression footnote of the compatibility page now describe that behavior, with the decimal scheme as the example. A v2 variant also writes the v1 ID, so a writer must allow both IDs to use it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
1 parent 3974ee0 commit 91b38f4

2 files changed

Lines changed: 19 additions & 19 deletions

File tree

‎docs/specs/versioning/compatibility.md‎

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -76,8 +76,7 @@ For the compatibility guarantee and minimum reader versions, see [Versioning](..
7676
support for Format B, so Library 1 still writes only Format A.
7777

7878
[^compression]:
79-
The default compressor filters schemes by their declared output wire IDs, but general per-writer
80-
scheme configuration is not implemented. The planned configuration would select compatible
81-
behavior before estimation, sampling, and full compression, avoiding recompression solely to
82-
meet the edition. These cells still require the writer to construct a permitted representation.
83-
See [Compression](design.md#compression) for the current behavior and planned work.
79+
The default compressor picks its schemes and their variants from the permitted wire IDs before
80+
it compresses, so it does not need to recompress only to meet the edition. These cells still
81+
require the writer to construct a permitted representation. See
82+
[Compression](design.md#compression) for the current behavior.

‎docs/specs/versioning/design.md‎

Lines changed: 15 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -258,20 +258,21 @@ recording the minimum version.
258258

259259
## Compression
260260

261-
The default BtrBlocks compressor excludes a scheme if any of its declared output wire IDs is
262-
forbidden. This filtering also applies to schemes used for child compression.
263-
264-
The current decimal scheme produces only single-child arrays and declares the v1 wire ID, so values
265-
too wide for it remain in the standard uncompressed decimal representation. The multi-child
266-
serializer exists, but the default scheme does not construct those arrays and no declared edition
267-
permits their v2 ID.
268-
269-
### Planned scheme configuration
270-
271-
The planned improvement is to configure a scheme's behavior for the selected editions, allowing it
272-
to retain an older mode when its newer mode requires a forbidden format. That configuration needs to
273-
apply consistently to estimation, sampling, full compression, children, and fallbacks. General
274-
per-writer scheme configuration is not implemented, and its API is unsettled.
261+
The default BtrBlocks compressor uses only the schemes whose output it can write. A wire ID is
262+
allowed when the session registers it, the enabled editions permit it, and the compressor's preset
263+
does not exclude it. For example, the CUDA preset excludes some v2 IDs. The compressor excludes a
264+
scheme if any wire ID that the scheme declares is not allowed. Child arrays go through the same
265+
compressor. The writer still checks every serialized ID, so a forbidden ID in any child fails the
266+
write.
267+
268+
A scheme can have variants that write different wire IDs. The compressor picks the variant from the
269+
allowed wire IDs, so callers do not choose it directly. For example, the decimal scheme uses its v2
270+
variant, which splits values too wide for one 64-bit child into several children, only when
271+
`vortex.decimal_byte_parts.v2` is allowed. Otherwise its v1 variant constructs only single-child
272+
arrays, and values too wide for one child remain in the standard uncompressed decimal
273+
representation. The v2 variant also writes the v1 ID for arrays that fit the v1 format, so allowing
274+
only the v2 ID excludes the scheme. No declared edition permits `vortex.decimal_byte_parts.v2` yet,
275+
so a writer that enforces editions writes only single-child decimal arrays.
275276

276277
[^decimal-availability]:
277278
No declared edition currently permits the multi-child format. The

0 commit comments

Comments
 (0)