Storing only a label mask image

[tischi]

@joshmoore @constantinpape

With the current specs, is it possible to only store labels without an image?

└── image.zarr                 
    │
    └── labels
        │
        ├── .zgroup
        │
        ├── .zattrs        

[joshmoore]

I believe so though I don't think it's clearly stipulated. @DragaDoncila and @jni may already be doing this and can say more. I think the primary downside in the current spec is that if you do that, it would be difficult to add the image back into the same fileset at a later date. Of course, if you're going to use the (future) remote links (#13) then it would be fine.

[DragaDoncila]

Sorry for the delayed reply on this! You can indeed store label files exclusively. We were doing this more than 6 months ago now, so the metadata structure may have changed, but we were storing a multiscale label image without the labels subdirectory, specifically:

└── label_image.zarr                 
    │
    └── 0
    │
    └── 1
    |
    └── ...
    │
    ├── .zgroup
    │
    ├── .zattrs 

With .zattrs specifiying the `"multiscales" and "image-label" members.

[tischi]

Thanks @DragaDoncila!
@joshmoore so the labels sub-directory is not mandatory?

[joshmoore]

For an image to be a label, no. For an image to have labels, currently yes.

[lubianat]

Hi,

Thank you all for the discussion here. As part of an effort to keep the NGFF issue tracker manageable, we're closing tickets that haven't seen activity in several years and whose discussion may be out of date.

If this issue is still relevant, feel free to reopen it with details in the current context of NGFF.

Best,
Tiago

[jni]

@lubianat I think this probably should be reopened. It is still unclear in the spec whether this is allowed, even though many folks have used it. At a minimum, the spec needs to be updated to be clearer on the expectations. And it may require a new RFC if there are conflicting implementations in the wild.

[lubianat]

Hi @jni, thanks! I will reopen it. It would be great if someone could provide an update on what is needed given the current spec/RFCs.

It could be a new issue to supersede this one, for example.

[jo-mueller]

I would say with 0.6rc0, you have multiple options at your disposal.

At the very least, what @DragaDoncila pointed out above still is very much correct: The presence of the image-label key signals that the group is not just a multiscale image, but a multiscale label image, irrespective of the context or location in a zarr hierarchy it appears at. An implementation would still be able to load the image without that key, but it wouldn't necessary know that it has to do something label-specific with the image.

If the multiscale appears under image/labels/..., that would signal to a reader to treat the image as a label image, irrespective of the presence of the image-label key. You as a user could make use of that to store label images, with or without the image-label key present because the location of the label image in the hierarchy (under /labels/...) would tell a reader what it needs to know.

Lastly, in 0.6(rc0), you could also put an image and a label in a scene, which is useful if the labels only contain a section of the whole image (i.e., from image annotations), or if the segmentation has been done at a different resolution level. The transformation in the parent group's scene field could then contain the necessary transformation between the label image and its corresponding image. In this case, the label-image would have to declare that it is a label image by containing the image-label field.

@dstansby opened a PR, that would have required a label image to declare its kind irrespective of where or how it appears, but that didn't become part of 0.6 because RFC8 will probably come up with a cleaner way of declaring an image's "kind", anyway.

[jni]

@jo-mueller I think the key issue here:

The presence of the image-label key signals that the group is not just a multiscale image, but a multiscale label image, irrespective of the context or location in a zarr hierarchy it appears at.

is that this is not clear at all in the spec text. (I am not sure about the schema.)

In OME-Zarr, Zarr arrays representing pixel-annotation data are stored in a group called "labels". [...] The "labels" group is nested within an image group

So to consider this issue closed, I think we need to update the text so that it is clear that it is valid to have a single image with the image-label key.

That's the quick and easy fix. A better but more intrusive fix (which probably means RFC) is #203 (comment), making labels a type of image.

These are not mutually exclusive. The first fix can really be considered just correctly documenting the status quo, so it should be done regardless.

[jo-mueller]

Yeah, ultimately I think it would nice to have a field for a "kind" of image, which would indicate the type of the image more clearly ("intensity/raw", "labels", "probability"), which would be a borderline ontological project, but a desirable one.

I agree though that the labels section is probably one of the more underdeveloped parts of the spec.