expand and constrain axis `unit` values (UDUNITS-2 controlled vocabulary, MUST)

[thewtex]

Summary

This is a request for preliminary feedback, not yet an RFC.

Two parts are proposed:

1. Expand the enumerated, UDUNITS-2-valid unit vocabularies beyond space and time,
   so that other axis types in the specification (and likely future ones) have canonical units.
2. Tighten the requirement: unit MAY be omitted, but if present its value MUST be
   one of the strings enumerated for that axis type — rather than the current SHOULD — so
   that it is machine-verifiable in the JSON Schema.

Related: ome/ngff-spec#192, ome/ome-ngff-validator#76, ome/ome-ngff-validator#52

Current state

Since 0.4, an axis "SHOULD contain the field unit [...] The value SHOULD be one of the following
strings, which are valid units according to UDUNITS-2", with enumerations provided only for
space and time axes. In 0.6.dev4, axes moved under coordinateSystems and the recommended
type vocabulary grew to array, space, time, channel, coordinate, displacement, plus
arbitrary custom types, but the unit lists were not revisited and still cover only space and
time.

Because the requirement is SHOULD, and the JSON Schemas encode MUST requirements, the schemas
validate only that unit is a string and never validate its value. That is the gap reported in
ome/ngff-spec#192.

This was already flagged during RFC-5 review. In response to a reviewer question about units, the
authors noted that units were out of scope for RFC-5 and that "revisiting units would be good
topic for a new RFC [...] since they seem not to have been reconsidered since they [were introduced]".
This issue picks up that deferred item.

Why this matters: FAIR interoperability

The value of an enumeration is not the list itself. It is that the same string always denotes the
same quantity, and that any consumer can decide membership without parsing, guessing, or
normalizing.

• "Valid UDUNITS-2" is a weaker constraint than the enumeration. UDUNITS-2 deliberately supports
  plurals, symbols, prefixes, aliases, and compound expressions. micrometer, micron, microns,
  um, µm (U+00B5 MICRO SIGN), and μm (U+03BC GREEK SMALL LETTER MU) are one physical unit and
  six incompatible strings — the last two are visually identical and byte-different. The enumeration
  is what carries the interoperability guarantee today; the UDUNITS-2 reference supplies provenance
  and conversion semantics.
• FAIR I1/I2. A controlled vocabulary drawn from an external, well-defined reference gives units
  a formal, shared, machine-actionable representation instead of free text.
• Cheap, exact, cross-language validation. An enum is validated identically by every JSON
  Schema implementation, in every language, with no unit-parsing dependency. It also surfaces errors
  at write time rather than as silent misinterpretation at read time.
• Equality and comparison semantics. Canonical strings let readers compare, merge, and register
  datasets by string equality, and let scale factors be converted safely.

Failures already observed

• feat: check for "units" field ome-ngff-validator#76: a sample in the wild used "units" instead of "unit". It passed
  validation while conveying nothing, because unknown keys are permitted and unit is optional.
  Non-canonical values such as "µm" are the same class of failure: data that looks fine and is
  quietly wrong or quietly ignored. The validator had to add heuristic checks outside the schema
  precisely because the spec language is SHOULD (see Validate unit, e.g. "micrometer" not "µm" ome-ngff-validator#52).
• Error, parsing the spatial unit in an OME-Zarr. google/neuroglancer#780: Neuroglancer errors parsing OME-Zarr spatial units, including "pixel",
  which OME-TIFF supports but NGFF does not enumerate — with the observation that the spec permits
  values outside the list.

Every downstream implementation (ome-ngff-validator, ome-zarr-models-py, pydantic-ome-ngff,
ngff-zarr, MoBIE/n5-ij, Neuroglancer, vizarr) currently invents its own tolerance policy. That is
the definition of a non-interoperable field.

Precedent in the spec family

RFC-4 already uses this pattern for orientation, with an explicit permissible-values table and a
controlled vocabulary. Per ome/ngff-spec#192, the RFC-4 orientation schema needs the same enum
treatment that axis unit does — which argues for fixing both with one shared artifact.

Proposed changes (for discussion)

1. Normative strength. An axis MAY contain unit; if present, its value MUST be one of the
strings enumerated for that axis type. Retain the SHOULD recommendation that physical axes carry
a unit. Optionality is preserved deliberately, since 0.6.dev4 supports unitless array coordinate
systems.

2. Expanded per-type vocabularies — canonical, singular, lowercase, UDUNITS-2-resolvable names.
Illustrative candidates only; the point of this issue is to find out which domains need what:

• space, time — as today.
• angle — degree, radian, arcminute, arcsecond (polarization imaging, tomography tilt
  series, light-sheet/SCAPE skew).
• spectral / channel — nanometer, micrometer, angstrom (wavelength); hertz, terahertz,
  kayser (frequency/wavenumber, Raman/FTIR); electronvolt, kiloelectronvolt (energy, EM/X-ray).
• temperature — kelvin, degree_celsius.
• Possibly mass (dalton), concentration, magnetic field strength (tesla), pressure — driven by
  whoever shows up in this thread.
• coordinate / displacement — clarify that these take the unit of the space they map into.

Note that angle and wavelength axis types were already raised and deferred during the 0.4 axes
discussion, so this expansion was anticipated.

3. Dimensionless / unknown. Decide explicitly between (a) omitting unit — the current de facto
meaning, and the RFC-5 authors' stated preference that "having no units key reflects that 'there are
no units' better than a placeholder" — and (b) an explicit sentinel such as dimensionless, which a
reviewer requested during RFC-5. Either way, pixel/index-style values should be handled by the
array/unitless coordinate-system mechanism already in 0.6.dev4, not by adding them to the space
enum.

4. Extension escape hatch. As with custom axis types, define how a domain uses a unit not in
the list. Candidates: a reserved namespaced prefix, a companion unitSystem/unitReference field,
or a documented "MUST be UDUNITS-2 parseable, MUST NOT be an alias of an enumerated unit" fallback
validated by pattern.

5. Machine-readable artifact. Publish the vocabularies as a single versioned schema file (e.g.
units.schema.json) that other schemas $ref, plus a plain data file (unit name, UDUNITS-2
expression, applicable axis types, description). Implementations then generate identical enums
instead of hand-maintaining lists, and ome/ngff-spec#192 is resolved uniformly across RFC-4
orientation and axis unit.

Anticipated disagreements

Raising these up front, because they are the real design tensions:

• Closed enum vs. UDUNITS-2 grammar. An enum is trivially validatable but must be revised for
  each new community. Accepting any UDUNITS-2 expression (um, mm/s, 1e-6 m) is maximally
  expressive but requires a unit-parsing dependency in every reader and reintroduces the aliasing
  problem the enum exists to prevent. A hybrid (enum = canonical/RECOMMENDED, pattern =
  permitted-with-warning) is a plausible landing zone, but it weakens the guarantee.
• MUST may invalidate existing writers. Some converters emit "µm" or "pixel" today. Is a
  MUST acceptable at 0.6, or should this be SHOULD plus a mandatory validator warning?
• UDUNITS-2 vs. UCUM/QUDT/OM. UDUNITS-2 is the incumbent and CF/geoscience-aligned; UCUM is
  stronger in clinical/DICOM contexts, QUDT is more ontology-friendly for RDF-based FAIR tooling.
  Reference a second vocabulary, or provide a crosswalk table rather than switching?
• Scope relative to RFC-4/RFC-5. Units interact with coordinateSystems, array axes, and
  RFC-4 orientation. Small self-contained "units" RFC layered on RFC-5, or folded into the axes
  work?
• channel semantics. Attaching a wavelength unit to a channel axis may indicate the need for
  a distinct wavelength/spectral axis type.

What we are asking for now

Preliminary thoughts only — specifically:

1. Which axis types and units does your domain need, ideally with a concrete dataset?
2. Enum vs. grammar vs. hybrid: what can your implementation realistically enforce?
3. MUST vs. SHOULD, and whether a shared units.schema.json would be adopted by your library.
4. Is anyone relying on unit strings outside the current lists, beyond the "pixel" case in
   Error, parsing the spatial unit in an OME-Zarr. google/neuroglancer#780?

We recognize there may be substantive disagreement, which is exactly why we would like to socialize
this and gather endorsements and objections here before drafting an RFC PR. The next step is an RFC following the template, with the enumerated values, schema
changes, an alias-mapping table, and compatibility/migration behavior.

[d-v-b]

worth also asking if explicit axis types are necessary when the units alone can convey timeness, spaceness, etc

[jni]

worth also asking if explicit axis types are necessary when the units alone can convey timeness, spaceness, etc

imho the answer is a definitive yes. The units don't convey all the information about the type. Wavelength and Space are very different concepts. timedeltas and time are slightly different concepts, but different nonetheless. For FLIM data, users/implementations probably want to differentiate between time and "microtime" (the lifetime measurement). Angular altitude (angle to the horizon) is different from a light polarisation angle. Etc.

[d-v-b]

For FLIM data, users/implementations probably want to differentiate between time and "microtime" (the lifetime measurement). Angular altitude (angle to the horizon) is different from a light polarisation angle. Etc.

agreed but I'm not sure an explicit list of axis types is a good way to model these differences.

maybe this is consistent with your point, but in the examples you gave, the information the unit fails to provide is information about how a signal was acquired, or the meaning of a signal in the context of an experiment. A small set of classes like "space" won't convey that. You might want labels instead of classes, like the cf standard names. But I would look carefully at that design, it's old and there are almost certainly things that can be done better.

[lubianat]

If we are touching it, may we also forbid units? A key units MUST NOT be present.

Some code at ome/ome-ngff-validator#76 is needed also because nothing blocks the "units" key from existing

[d-v-b]

If we are touching it, may we also forbid units? A key units MUST NOT be present.

If you want readers to treat additional keys are an error, then I would do this for all additional keys, not a particular one. IIRC "units" vs "unit" arose because of some confusion over the spec text vs the JSON schema, but presumably that confusion has been resolved. If the spec documents are clear on the structure of the document, then you probably don't need to put special keys on a blocklist.

That being said, there are reasons for making the set of keys closed (i.e., an additional key is an error that halts a reader), I think this makes sense when there are interactions between keys and so any new key is basically introducing an new data model that old readers can't handle.
But there are also reasons for keeping the set open: an open set allows future versions of the spec to include additional, optional keys without breaking old readers. Otherwise, readers for version X will error on perfectly readable data if version X + 1 adds a new, optional key.

IMO this is a broader issue that the spec should probably answer decisively in one place , and then that decision can be re-used. You may end up in a situation where some metadata allows extra keys, but others do not, based on how they are used / how they are expected to evolve.

see #209