Skip to content

Fix MDPDatastore raising KeyError for category="static" when absent - #733

Open
AshNicolus wants to merge 2 commits into
mllam:mainfrom
AshNicolus:fix/mdp-datastore-missing-static-category
Open

AshNicolus wants to merge 2 commits into
mllam:mainfrom
AshNicolus:fix/mdp-datastore-missing-static-category

Conversation

@AshNicolus

Copy link
Copy Markdown
Contributor

Describe your changes

MDPDatastore.get_dataarray's own docstring says "'state' is the only required category, for other categories, the method will return None if the category is not found in the datastore" - matching the abstract base class contract (BaseDatastore.get_dataarray: "'forcing' and 'static' are optional (in which case the method should return None)").

But the implementation only special-cased "forcing":

if category not in self._ds and category == "forcing":
    warnings.warn("no forcing data found in datastore")
    return None

This exact pattern was copy-pasted across four methods (get_vars_units, get_vars_names, get_vars_long_names, get_dataarray), never handling "static". An mllam-data-prep config with state+forcing but no static output category - a legitimate configuration per the documented interface - raised an unhandled KeyError: "No variable named 'static'" instead of gracefully returning None/[].

Fix: extend the guard from category == "forcing" to category != "state" in all four places, matching the documented "only state is required" contract, with a category-aware warning message so it still says the right thing for either missing category.

Added a regression test (bypassing the full mllam-data-prep machinery with a synthetic _ds, same pattern as other lightweight datastore unit tests) - verified it fails with the original KeyError before the fix and passes after. Full tests/test_datastores.py suite (55 tests) passes with real network access.

Issue Link

closes #732

Type of change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📖 Documentation (Addition or improvements to documentation)

Checklist before requesting a review

  • My branch is up-to-date with the target branch - if not update your fork with the changes from the target branch (use pull with --rebase option if possible).
  • I have performed a self-review of my code
  • For any new/modified functions/classes I have added docstrings that clearly describe its purpose, expected inputs and returned values
  • I have placed in-line comments to clarify the intent of any hard-to-understand passages of my code
  • I have updated the README to cover introduced code changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have given the PR a name that clearly describes the change, written in imperative form (context).
  • I have requested a reviewer and an assignee (assignee is responsible for merging). This applies only if you have write access to the repo, otherwise feel free to tag a maintainer to add a reviewer and assignee.

AshNicolus added a commit to AshNicolus/neural-lam that referenced this pull request Aug 25, 2026
@sadamov

sadamov commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

Thanks @AshNicolus, this is the right fix and overlaps with #231 (fixes #125), where I proposed the same change across all four accessors in varunsiravuri#2 but it never landed. Are you still interested in getting this merged once v0.7.0 is released? If so I would rebase on main and replace the object.new test with a proper state-only datastore config like the one in #231. And plese restore the full PR template

@AshNicolus

Copy link
Copy Markdown
Contributor Author

Yes, still interested in merging after v0.7.0. I'll rebase on main, replace the test with a proper state-only datastore config, and restore the full PR template.

get_dataarray's own docstring (and the abstract base class contract)
say only "state" is required, with "forcing" and "static" both
optional and returning None/[] when missing. The guard in
get_vars_units, get_vars_names, get_vars_long_names and get_dataarray
only special-cased "forcing", so a static-less mllam-data-prep config
raised an unhandled KeyError instead.

Extend the guard to category != "state", matching the documented
contract, with a category-aware warning message.

Closes mllam#732
AshNicolus added a commit to AshNicolus/neural-lam that referenced this pull request Sep 14, 2026
@AshNicolus
AshNicolus force-pushed the fix/mdp-datastore-missing-static-category branch from 30280da to 8a4ac89 Compare September 14, 2026 14:06
@AshNicolus
AshNicolus force-pushed the fix/mdp-datastore-missing-static-category branch from 8a4ac89 to 889200b Compare September 14, 2026 14:06
@AshNicolus

Copy link
Copy Markdown
Contributor Author

Rebased on latest main and updated the test comment. Ready for merge after v0.7.0.

@sadamov

sadamov commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

@AshNicolus you really need to fix the PR template in all your open PR descriptions. we need to the full template with all checkboxes to merge into main

@sadamov sadamov self-assigned this Sep 14, 2026
@sadamov
sadamov self-requested a review September 14, 2026 14:08
@sadamov sadamov added the bug Something isn't working label Sep 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MDPDatastore raises KeyError for category="static" when no static data is present

2 participants