Add conversions between the typed and dynamic column types

[emilk]

Related

• None

What

There was no way to go between Column<L>/ColumnDesc<Column<L>> and DynColumn/DynColumnDesc. Now there is:

• ColumnDesc::to_dyn() -> DynColumnDesc, plus impl From<&ColumnDesc<C>> for DynColumnDesc. The declared metadata is dropped, since DynColumnDesc does not carry any.
• Column::into_dyn(name) -> DynColumn, for L: ConcreteType. The field takes its name from the argument, its datatype and nullability from L, and its metadata from the column. Zero-copy: the array is moved.
• DynColumn::try_into_column::<L>() -> Result<Column<L>, Error>, the inverse. Validates and downcasts (zero-copy), carrying over the field metadata.

Two things worth a look:

• into_dyn needs an explicit name, because Column<L> does not store one. Passing it in beats inventing a placeholder.
• try_into_column errors use record_type: "DynColumn", matching how Column::from_record_batch_and_name uses "Column".

Testing

New tests in tests/column.rs and tests/quiver.rs cover metadata round-tripping, Option<…> as the only source of field nullability, nested inner-field nullability, and that validation follows the array's nulls rather than the field's nullable flag.

Compatibility

Additive only.

🤖 Generated with Claude Code

[emilk]

Post-merge review, as part of reviewing the 0.6 stack for #30. Clippy and tests are clean at the tip of release-0.6, so everything below is a follow-up. This closes a real hole, and the test that pins which nullability wins — the array's nulls decide, not the field's nullable flag — is the subtle case and it is asserted three ways.

into_dyn is ConcreteType-only, so the multi-encoding types can't cross. AnyList/AnyUtf8/AnyBinary are parse-only and have no L::datatype(), but a Column<AnyUtf8> has an array and that array has a concrete datatype. self.as_arrow().data_type().clone() plus L::NULLABLE works for every LogicalType, and is more accurate for the concrete ones too (it reports what is there, not what L says should be). As it stands, exactly the users who parsed an unknown encoding have to hand-build the Field.

The metadata clone is now copied five times. field.metadata().iter().map(|(key, value)| (key.clone(), value.clone())).collect() appears here, in Column::extract_named, and three times in generated code (quiver.rs:437, :627, :787). A pub fn field_metadata(field: &Field) -> BTreeMap<String, String> in quiver_types covers all five and shrinks the expansion, which is the one place the copies also cost compile time.

Minor: to_dyn() dropping the declared metadata is documented but untested — with #32's whole-descriptor equality it is a two-line assertion, and "the metadata is dropped" is the kind of documented behavior that quietly becomes false.

— Claude