Skip to content

The truth readers open a zarr without the guard written for it #231

Open

Description

@chrisfinlay

tabascal/truth.py has three readers that take a path and open it bare:

def read_true_vis_ast(zarr_path, data_col=None):   # truth.py:144
    xds = xr.open_zarr(zarr_path)
def read_true_vis_rfi(zarr_path, data_col=None):   # truth.py:153
    xds = xr.open_zarr(zarr_path)
def read_true_gains(zarr_path):                    # truth.py:162
    xds = xr.open_zarr(zarr_path)

A store that is missing, unreadable or corrupt gives whatever xarray raises, from inside component setup, without naming the configuration option that asked for the truth.

There was a helper for exactly this

#230 removed _open_zarr, which was dead:

def _open_zarr(config: Dict):
    """Open the sim zarr, raising a clear TruthError if it cannot be read."""
    path = _zarr_path(config)
    if not path or not os.path.exists(path):
        raise TruthError(f"No tab-sim truth available: simulation zarr not found at {path!r}.")
    try:
        return xr.open_zarr(path)
    except Exception as e:
        raise TruthError(f"Could not open simulation zarr at {path!r}: {e}") from e

It took a config; the readers take a path, so it could never have been their caller. It was written for this and never wired in, which is why it read as dead to a reachability sweep. Deleting it was right; the gap it was meant to close is still there.

Why it is not simply reachable today

require_truth runs before TabConfig setup and fails fast when a config needs truth the zarr does not provide — it checks the path exists and that the required variables are present. So the common cases are caught early and well.

What is not caught: a store that exists and passes available_truth but fails on read — truncated, permissions, a partially written zarr, a store on a filesystem that goes away between the preflight and the read. available_truth opens it in a try that returns all-False on failure, so an unreadable store makes require_truth report the truth as missing, which is a reasonable message; but a store that opens for the preflight and fails later reaches the readers bare.

What to do

  • Give the three readers the same treatment: a TruthError naming the path and the underlying error. A shared path-taking helper rather than the config-taking one, so the readers can actually use it.
  • Decide whether available_truth's silent except Exception: return flags should distinguish "no store" from "store I could not read" — right now a corrupt zarr is reported to the user as an absent one, and those want different fixes.
  • Worth a test with a deliberately truncated store, which nothing currently covers.

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions