`slot_derivations` at top-level of `TransformationSpecification` has no effect in `SchemaMapper`

[pkalita-lbl]

According to the TransformationSpecification model, a slot_derivations list is allowed at the top level. The model says that it specifies "instructions on how to derive a set of top level slots in the target schema".

Based on that, I would expect that with this source schema:

# source schema
name: test_source
id: http://example.com/test_source

slots:
  a_slot:
    description: This is a top-level slot definition
    range: string

classes:
  AClass:
    description: This is a top-level class definition
    slots:
     - a_slot

And this transformation spec:

# transformation spec
title: Transform test_source into test_target
source_schema: test_source
target_schema: test_target

slot_derivations:
  a_slot:
    derived_from: a_slot
    overrides:
      description: TOP-LEVEL DERIVED SLOT DEFINITION

class_derivations:
  AClass:
    populated_from: AClass
    slot_derivations:
      a_slot:

I think I would have expected that the top-level derivation of a_slot would have had an effect before the derivation of a_slot within the context of the AClass class derivation. In other words, I was expecting to get something like this as a result:

# expected target schema
name: test_source-derived
id: http://example.com/test_source-derived
default_prefix: http://example.com/test_source-derived/
classes:
  AClass:
    name: AClass
    description: This is a top-level class definition
    attributes:
      a_slot:
        name: a_slot
        description: TOP-LEVEL DERIVED SLOT DEFINITION
        alias: a_slot
        domain_of:
        - AClass
        range: string

Instead I just get the a_slot description from the source top-level slot definition in the target schema:

# actual target schema
name: test_source-derived
id: http://example.com/test_source-derived
default_prefix: http://example.com/test_source-derived/
classes:
  AClass:
    name: AClass
    description: This is a top-level class definition
    attributes:
      a_slot:
        name: a_slot
        description: This is a top-level slot definition
        alias: a_slot
        domain_of:
        - AClass
        range: string

[amc-corey-cox]

Design analysis

We've done a thorough review of this issue and the related is_a/mixins inheritance on ClassDerivation. Here's where things stand:

Current state

• Top-level slot_derivations: defined in the model since 54e917f3 (Feb 2023, cmungall) but never wired up in runtime code — not in SchemaMapper, ObjectTransformer, or engine.py.
• is_a/mixins on ClassDerivation: model added in the same commit, runtime merge logic added in e5a3bd87 (Jan 2024). The merge is shallow dict-key replacement — if a child defines a slot_derivation by name, it completely replaces the parent's. No field-level merging.
• Both were likely intended to mirror the LinkML metamodel (top-level slots vs class-level attributes, class inheritance).

Possible design

If implemented, these could work together naturally:

1. Top-level slot_derivations as defaults — if a class doesn't define a slot derivation for a given slot, inherit from the top level.
2. is_a/mixins parent slot_derivations as defaults at class level — same semantic.
3. Resolution order: class-level → is_a/mixins parents → top-level spec.
4. Field-level merge (not whole-SlotDerivation replacement): start with the inherited default, overlay the child's non-None fields. This matches how LinkML slot usage works — a class inherits a top-level slot's range/description/etc. and can override individual fields.

Motivating use case

With the slot() function and hide features (PR #200), top-level defaults become powerful for shared derivation patterns:

slot_derivations:
  visit_id:
    expr: "uuid5('VisitURI', slot(_hidden_visit_identifier), slot(participant_id))"
  participant_id:
    populated_from: dbGaP_id

class_derivations:
  MeasurementObservation:
    populated_from: pht001450
    slot_derivations:
      _hidden_visit_identifier:
        expr: "f'{phv00098771}_{phv00098800}'"
        hide: true

  DrugExposure:
    populated_from: pht002100
    slot_derivations:
      _hidden_visit_identifier:
        expr: "f'{phv00099450}_{phv00099470}'"
        hide: true

Each class defines only the table-specific intermediate value. The shared UUID construction logic and participant_id mapping are inherited from the top level. This also brings trans-specs into intuitive alignment with the LinkML metamodel — users familiar with LinkML's top-level slots and class inheritance would find these semantics unsurprising.

Why we're not implementing now

The gains are real but small relative to the surface area introduced:

• Field-level merge adds resolution complexity and implicit context — "where did this field come from?" isn't answerable by reading the class derivation alone.
• Most real-world specs (especially generated ones) have few slots that would benefit from top-level defaults.
• The is_a/mixins mechanism already provides structural reuse for hand-authored specs, even with the current shallow merge.
• The motivating use cases save a repeated line or two per class, which doesn't justify the debugging/testing cost yet.

Leaving this open for future consideration if the pattern causes real pain at scale. The design above is sound if we decide to proceed.