Repository navigation
DOC: Show every radarx function in the end-to-end workflow notebook - #199
Merged
Merged
Conversation
Extend Radar_Workflow with the sounding utilities and wind-profile parameters, the VAD profile, the Bayesian DSD retrieval, the other gridding and plotting functions, ERA5 on the grid, network bias and evaporation, and add Radar_Workflow_Advanced (fundamentals, disdrometers, raindrop trajectories, surface stations, lightning, tornado and biological echo, single-Doppler winds, the diabatic Lagrangian analysis and the ML interface) on small open or synthetic data. A function index lists every public name with its section, and tests/test_workflow_covers_api.py fails when a function or accessor method is missing from the notebooks.
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 0 |
| Duplication | 0 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #199 +/- ##
=======================================
Coverage 98.58% 98.58%
=======================================
Files 72 72
Lines 14367 14367
=======================================
Hits 14163 14163
Misses 204 204
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Every public radarx function and every
.radarxaccessor method now appears in a worked example, in a sensible pipeline order with short explanatory text, a realistic call and a figure or printed result.docs/notebooks/Radar_Workflow.md(core pipeline on the KGWX squall line, 30 March 2022) gets these additions, each next to the step it belongs to:get_s3_client,list_available_filesecho_mask,apply_mask,dealias_velocity,estimate_kdp,hid,qvp,llsd,azimuthal_shear,radial_divergence), checked against each othernearest_station,station_list,read_sounding,era5_profile,open_sounding_file, humidity and thermodynamic helpers,interpolate_profile,mean_wind) and wind-profile parameters (bulk_shear,layer_mean_wind,bunkers_storm_motion,storm_relative_wind,storm_relative_helicity, hodograph)vad_profile) against ERA5 and the radiosondedsd_prior,forward_grid,dsd_bayesian) against the deterministic retrievalgrid_cones,grid_radar,make_3d_grid,create_cappi,stack_data,gate_corners,plot_ppi,plot_cappi,plot_rhi,plot_maxcappiand the sevenhvplot_*functionsgrid_radars;era5_column,radar_geometry,network_bias,merge_radarsevaporation,integrate_evaporation)New
docs/notebooks/Radar_Workflow_Advanced.md(small open data or synthetic data with a known answer; no third-party field-campaign data is committed, the file formats are written on the fly):radarx.fundamentals,radarx.core, one call per function)read_parsivel,read_pips_netcdf,disdrometer_qc,raupach_berne_correction,fit_gamma,radar_from_dsd,match_radar, ...)dsd_spectrumrain_trajectories,size_sorting,surface_dsd,rain_source_points,trajectory_matched_times)read_sticknet,read_pips,read_mrr,read_wind_profiler,cold_pool_*,rkw_ratio,baroclinic_generation)read_lma,cluster_flashes,grid_lightning,cell_flash_rate,lightning_jump)tornet_inputs,tornado_probability,rotation_couplets,biological_echo; the MIT-licensed upstream weights are downloaded on first use as in the tornado notebook)single_doppler_winds,radar_geometry)trajectories,diabatic_lagrangian, closures,microphysical_rates,fall_speed)radarx.ml, with a tiny ONNX model built in the notebook)The notebook was split in two because the single notebook would have run well beyond the 600 s limit of
nb_execution_timeouton a slow connection; the API-coverage test spans both.tests/test_workflow_covers_api.pyfails when a name of the__all__ofradarx.retrieve,grid,io,ml,vis,fundamentalsorcore, or an accessor method, is not called in the code cells of the two notebooks (imports do not count) or is missing from the function index. Documented exceptions: the deprecated IMD reader (read_sweep,read_volume,to_cfradial2,to_cfradial2_volumes) and the helpers ofradarx.testing, which are shown through xradar in the IMD notebook.Runtime
Executed with
pytest docs/notebooks/<name>.mdon a laptop (Apple silicon, compiled kernels): Radar_Workflow 112 s of CPU, Radar_Workflow_Advanced 93 s of CPU. The wall time (9.4 and 3.1 min) was dominated by a slow connection to the AWS archive and the ERA5 store on the day of the run; both notebooks are well below the 600 s limit with a normal connection.Every figure was inspected for overlapping text and text on top of data.
Filed while writing this: #198 (a user-supplied
axis closed byplot_*(show_figure=False)). The fundamentals issues #167, #168, #194 and #195 and themake_3d_gridissue #169 are already open.Machine-learning examples on real data, PERiLS citations
ML_KDPfits a linear network to the CSAPR2 processor KDP on azimuth sectors outside 120 to 240 degrees and tests it on that sector, next to the classical estimators; the target is a reference estimate, not the truth.ML_Single_Dopplerfits a linear network to the KGWX and KBMX dual-Doppler wind of two volume pairs (23:33 and 23:46 UTC) and tests it on the 23:59 UTC pair, next to the variational retrieval.Radar_Workflow_Advancedsmooths the real KGWX sweep with the ONNX example; the box filter inMachine_Learningis marked as a plumbing example."perils2022"DSD prior cites the PIPS data set (Dawson et al. 2025, doi:10.26023/HFBG-7W5M-WA00) and Kosiba et al. (2024) in the docstring,README_dsd.mdnext to the CSV and the notebooks; the examples use the generic prior by default. No PIPS, StickNet or LMA files are in the repository.