Skip to content

Replace arrow_field with a memoizing arrow_field_ref - #34

Draft
emilk wants to merge 4 commits into
release-0.6from
emilk/arrow-field-ref
Draft

Replace arrow_field with a memoizing arrow_field_ref#34
emilk wants to merge 4 commits into
release-0.6from
emilk/arrow-field-ref

Conversation

@emilk

@emilk emilk commented Aug 27, 2026

Copy link
Copy Markdown
Member

Related

  • None

What

ColumnDesc::arrow_field_ref() -> FieldRef, built once and thereafter just an Arc::clone. arrow_field is removed, as suggested — *desc.arrow_field_ref() clones an owned Field if you need one. It was only used in tests.

The cache is a &'static OnceLock<FieldRef>, not an inline OnceLock field. That is the part worth reviewing: the derive emits descriptors as pub const, so a cell stored by value would be materialized fresh at every use site and would never hit — a cache that silently does nothing. Holding a reference to a static means all uses of a const share one cell, and ColumnDesc stays Copy.

The derive attaches one per column:

pub const COLUMN_MAYBE_AGE: ColumnDesc<Option<i64>> =
    ColumnDesc::new("Typed", "maybe_age").with_field_cache({
        static FIELD: OnceLock<FieldRef> = OnceLock::new();
        &FIELD
    });

with_field_cache is public, so hand-written descriptors can do the same. Without it the field is rebuilt per call — correct, just not free. The cache does not take part in PartialEq or Debug.

OnceLock rather than LazyLock: LazyLock's initializer is part of its type and cannot capture, so it could not see the descriptor's name and metadata. get_or_init lets the descriptor supply the builder, so one method serves derived and hand-written descriptors alike.

Testing

New test covers repeat calls sharing an allocation, a separate copy of the const sharing it too (the case a by-value cell gets wrong), an uncached descriptor rebuilding each time, and the cache not affecting equality. Full CI job run locally on 1.95.0.

Compatibility

Breaking: arrow_field is gone, and ColumnDesc gains a private field.

🤖 Generated with Claude Code

emilk and others added 4 commits August 27, 2026 18:07
`arrow_field_ref` returns a `FieldRef`, built once and thereafter just an
`Arc::clone`, for descriptors that carry a cache.

The cache is a `&'static OnceLock<FieldRef>` rather than an inline cell:
descriptors are `const`, so a cell stored by value would be a fresh,
always-empty one at every use site — a cache that silently never hits.
`with_field_cache` takes the reference, so all uses of a `const` share
the one `static`. Descriptors built without it still work, rebuilding
the field per call.

`arrow_field` is gone; `*desc.arrow_field_ref()` clones an owned `Field`
if you need one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Covers the case a by-value cell would get wrong: a separate copy of the
`const` still shares the one cached `FieldRef`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`with_field_cache` becomes `#[doc(hidden)] __with_field_cache`. Nothing
about the mechanism changes — the cache still has to point at a `static`,
because a `const` is inlined at every use site and an interior-mutable
field would be a fresh, always-empty cell on each call (exactly what
`clippy::declare_interior_mutable_const` warns about). But it is no
longer something a caller is expected to reach for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant