Fix/cfradial2 global attrs assignment - #384
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #384 +/- ##
==========================================
+ Coverage 94.23% 94.25% +0.02%
==========================================
Files 29 29
Lines 6452 6478 +26
==========================================
+ Hits 6080 6106 +26
Misses 372 372
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
@chfer Thanks for starting this. This may need a bit of discussion here. This export function was not meant to add these attributes to the original dataset, but only to add or overwrite these attributes to be written to disk. So if you do something like this with your approach: dtree.to_cfradial2(fname2)
dtree.to_cfradial1(fname1)would have |
|
I believe @kmuehlbauer is suggesting something along these lines: dtree.xradar.to_cf2(fname2)
dtree.xradar.to_cf1(fname1) |
|
Thank you for bringing this to my attention. Indeed my proposed solution is too simple and causes undesirable side-effects. After conversion to cfradial2 the original DataTree should be preserved. The first thing that comes to my mind is to create a (deep) copy of the DataTree upon entering the function to_cfradial2 , so that the original DataTree will always be preserved: def to_cfradial2(dtree, filename, engine=None, timestep=None):
"""
docstring ...
"""
# let the local dtree reference a copy and not the original DataTree
dtree = dtree.copy(deep=True)
# now safely modify dtree
...
# write DataTree
dtree.to_netcdf(filename, engine=engine)Maybe a shallow copy of dtree is sufficient, I still have to find this out. |
|
In this last commit, the In principle, a shallow copy would be sufficient: |
|
@chfer Is this still draft? We usually wait for the PR author to set this as ready for review. |
dae89d4 to
c75a777
Compare
* A test “tests/io/test_cfradial2.py::test_to_cfradial2_global_attrs” was added to check if the global attributes of a CfRadial2 file created by the to_cfradial2 function are present and have the expected values. * Former test needs a minimal DataTree, which is also needed by test_to_cfradial2_selects_default_engine. To avoid repetition a pytest fixture minimal_dtree wass added in “tests/io/test_cfradial2.py. * The test “tests/io/test_cfradial2.py::test_to_cfradial2_global_attrs” demonstrates that the to_cfradial2 function fails to set global attributes correctly.
* In the function to_cfradial2 (file xradar/io/export/cfradial2.py, line 79), the original statement 'root = dtree[/].to_dataset()' was replaced with 'root = dtree[/].ds'.This change matters because DataTree.to_dataset() returns a copy of the dataset, while DataTree.ds gives a direct reference to the dataset stored in the tree. Using .ds ensures that modifications affect the actual DataTree content instead of a detached copy. * The test “tests/io/test_cfradial2.py::test_to_cfradial2_global_attrs” demonstrates now that the to_cfradial2 function succeeds in setting the global attributes as expected. * Add a history.md entry for global attribute assignement fix in to_cfradial2.
* ADD: In to_cfradial2, create a copy of the input DataTree and modify only that copy for CfRadial2 export. The original DataTree remains unchanged and can still be used after the function returns. * TST: Add test_to_cfradial2_preserves_input_dtree to verify that to_cfradial2 does not mutate the input DataTree. * DOC: Add a history entry documenting that to_cfradial2 now preserves the input DataTree.
…on.md * set first_dim=time when creating dtree3 for CfRadial2-consistent dimension ordering * set optional=False so dtree3 matches variables written by to_cfradial2 * make the roundtrip comparison validate real equivalence instead of relying on prior in-place mutation behavior * updated history.md
c75a777 to
7614f25
Compare
|
The notebook rendered from CfRadial1_Model_Transformation.md started failing after commit 32d8c37. The failure occurs in the section “Roundtrip with xradar.io.to_cfradial2”, where the workflow is:
On main, this test passed, but it was not validating the intended behavior: xd.io.to_cfradial2 mutated dtree3 in place, so dtree3 and dtree4 were effectively guaranteed to match. After commit 32d8c37, xd.io.to_cfradial2 uses a deep copy instead of mutating the input. This exposed real differences between dtree3 and dtree4, causing the assertion to fail for two reasons:
With both conditions applied, the roundtrip assertion succeeds:
|
…ng to CfRadial2 * In model.py, the dictionaries required_global_attrs and optional_root_attrs were adapted to allow type checking of inserted global attributes and, where applicable, to check whether the given values are allowed. * These dictionaries were also made immutable by using MappingProxyType, with allowed values enumerated as a tuple instead of a list. * In cfradial2.py, the function to_cfradial2 was adapted so it now accepts a global_attrs parameter, through which global attributes to add or override can be specified. * A pytest function, test_to_cfradial2_global_attrs_override, was added to test overriding/manually inserting global attributes in to_cfradial2. * Before being applied, the contents of the global_attrs parameter are validated by a dedicated function, validate_global_attrs, in model.py. A warning is issued for unknown attributes; exceptions are raised when the type or value of a given global attribute is invalid. * A pytest function, test_validate_global_attrs_valid_and_invalid_cases, was added for the validate_global_attrs function. * Updated history.md.
Add support for manually setting global attributes when exporting to CfRadial2SummaryWhen exporting to CfRadial2/FM301 not all global attributes can be derived from the imported DataTree. Sometimes some global attributes have to be overridden. This commit adds a global_attrs parameter to to_cfradial2(), allowing callers to add or override root-group global attributes at export time (e.g. Conventions, title), with validation against the CfRadial2/FM301 attribute schema. Changesmodel.py
cfradial2.py
Tests
No breaking changes: global_attrs defaults to None, preserving existing behavior when unused. |
* Remove the unsupported ReadOnly attribute from the global attribute schema. * Update history.md.
|
In commit 4c0c109 several tests involving model.py failed under Python 3.11 because the ReadOnly type qualifier is only available from Python 3.13 onward. This is now corrected in commit f601cfc : the unsupported ReadOnly qualifier was removed from GlobalAttrSpec. |

This PR fixes a bug in the to_cfradial2 function (located in xradar/io/export/cfradial2.py, line 79).
At that line, the code used:
The intention is for root to reference the root Dataset so that the following lines can assign the global attributes of the CfRadial2 file. However, DataTree.to_dataset() returns a copy of the dataset, not the live dataset stored in the tree. As a result, the global attributes were being written to this temporary copy and never persisted in the actual DataTree.
The first commit in this branch (fix/cfradial2-global-attrs-assignment, 0eaf495) adds a pytest, tests/io/test_cfradial2.py::test_to_cfradial2_global_attrs, which demonstrates that the global attributes produced by to_cfradial2 do not match the expected values for a CfRadial2 file.
The second commit (3585b62) fixes the issue by replacing:
with:
Using .ds ensures that the function operates on the actual dataset stored in the DataTree, making the global attribute assignment effective.