Skip to content

DOC: Show every radarx function in the end-to-end workflow notebook - #199

Merged
syedhamidali merged 5 commits into
mainfrom
doc-workflow-all-functions
Oct 10, 2026
Merged

syedhamidali merged 5 commits into
mainfrom
doc-workflow-all-functions

Conversation

@syedhamidali

@syedhamidali syedhamidali commented Oct 9, 2026 •

Copy link
Copy Markdown
Owner

What

Every public radarx function and every .radarx accessor method now appears in a worked example, in a sensible pipeline order with short explanatory text, a realistic call and a figure or printed result.

docs/notebooks/Radar_Workflow.md (core pipeline on the KGWX squall line, 30 March 2022) gets these additions, each next to the step it belongs to:

  • 1: get_s3_client, list_available_files
  • 2, 4, 5, 6, 8, 9: the functions behind the accessors next to the accessor calls (echo_mask, apply_mask, dealias_velocity, estimate_kdp, hid, qvp, llsd, azimuthal_shear, radial_divergence), checked against each other
  • 3: sounding utilities (nearest_station, station_list, read_sounding, era5_profile, open_sounding_file, humidity and thermodynamic helpers, interpolate_profile, mean_wind) and wind-profile parameters (bulk_shear, layer_mean_wind, bunkers_storm_motion, storm_relative_wind, storm_relative_helicity, hodograph)
  • 4: VAD wind profile (vad_profile) against ERA5 and the radiosonde
  • 7: Bayesian DSD retrieval (dsd_prior, forward_grid, dsd_bayesian) against the deterministic retrieval
  • 10: grid_cones, grid_radar, make_3d_grid, create_cappi, stack_data, gate_corners, plot_ppi, plot_cappi, plot_rhi, plot_maxcappi and the seven hvplot_* functions
  • 11, 12: accessor forms of the advection functions and grid_radars; era5_column, radar_geometry, network_bias, merge_radars
  • 13: the evaporation section that was a placeholder (evaporation, integrate_evaporation)
  • Function index: every public name with its module and the section where it is used, plus the accessor methods, and a References section generated from Crossref

New docs/notebooks/Radar_Workflow_Advanced.md (small open data or synthetic data with a known answer; no third-party field-campaign data is committed, the file formats are written on the fly):

  1. radar equation, beam geometry, Doppler and unit helpers (radarx.fundamentals, radarx.core, one call per function)
  2. disdrometers (read_parsivel, read_pips_netcdf, disdrometer_qc, raupach_berne_correction, fit_gamma, radar_from_dsd, match_radar, ...)
  3. Bayesian DSD retrieval with calibration check, dsd_spectrum
  4. raindrop trajectories and size sorting (rain_trajectories, size_sorting, surface_dsd, rain_source_points, trajectory_matched_times)
  5. surface stations, profilers and cold pools (read_sticknet, read_pips, read_mrr, read_wind_profiler, cold_pool_*, rkw_ratio, baroclinic_generation)
  6. lightning (read_lma, cluster_flashes, grid_lightning, cell_flash_rate, lightning_jump)
  7. tornado detection and biological echo (tornet_inputs, tornado_probability, rotation_couplets, biological_echo; the MIT-licensed upstream weights are downloaded on first use as in the tornado notebook)
  8. single-Doppler winds (single_doppler_winds, radar_geometry)
  9. diabatic Lagrangian analysis (trajectories, diabatic_lagrangian, closures, microphysical_rates, fall_speed)
  10. machine-learning plumbing (radarx.ml, with a tiny ONNX model built in the notebook)

The notebook was split in two because the single notebook would have run well beyond the 600 s limit of nb_execution_timeout on a slow connection; the API-coverage test spans both.

tests/test_workflow_covers_api.py fails when a name of the __all__ of radarx.retrieve, grid, io, ml, vis, fundamentals or core, or an accessor method, is not called in the code cells of the two notebooks (imports do not count) or is missing from the function index. Documented exceptions: the deprecated IMD reader (read_sweep, read_volume, to_cfradial2, to_cfradial2_volumes) and the helpers of radarx.testing, which are shown through xradar in the IMD notebook.

Runtime

Executed with pytest docs/notebooks/<name>.md on a laptop (Apple silicon, compiled kernels): Radar_Workflow 112 s of CPU, Radar_Workflow_Advanced 93 s of CPU. The wall time (9.4 and 3.1 min) was dominated by a slow connection to the AWS archive and the ERA5 store on the day of the run; both notebooks are well below the 600 s limit with a normal connection.

Every figure was inspected for overlapping text and text on top of data.

Filed while writing this: #198 (a user-supplied ax is closed by plot_*(show_figure=False)). The fundamentals issues #167, #168, #194 and #195 and the make_3d_grid issue #169 are already open.

Machine-learning examples on real data, PERiLS citations

  • ML_KDP fits a linear network to the CSAPR2 processor KDP on azimuth sectors outside 120 to 240 degrees and tests it on that sector, next to the classical estimators; the target is a reference estimate, not the truth.
  • ML_Single_Doppler fits a linear network to the KGWX and KBMX dual-Doppler wind of two volume pairs (23:33 and 23:46 UTC) and tests it on the 23:59 UTC pair, next to the variational retrieval.
  • Radar_Workflow_Advanced smooths the real KGWX sweep with the ONNX example; the box filter in Machine_Learning is marked as a plumbing example.
  • The "perils2022" DSD prior cites the PIPS data set (Dawson et al. 2025, doi:10.26023/HFBG-7W5M-WA00) and Kosiba et al. (2024) in the docstring, README_dsd.md next to the CSV and the notebooks; the examples use the generic prior by default. No PIPS, StickNet or LMA files are in the repository.

Extend Radar_Workflow with the sounding utilities and wind-profile parameters,
the VAD profile, the Bayesian DSD retrieval, the other gridding and plotting
functions, ERA5 on the grid, network bias and evaporation, and add
Radar_Workflow_Advanced (fundamentals, disdrometers, raindrop trajectories,
surface stations, lightning, tornado and biological echo, single-Doppler winds,
the diabatic Lagrangian analysis and the ML interface) on small open or
synthetic data. A function index lists every public name with its section, and
tests/test_workflow_covers_api.py fails when a function or accessor method is
missing from the notebooks.
@codacy-production

codacy-production Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 0 complexity · 0 duplication

Metric Results
Complexity 0
Duplication 0

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@codecov

codecov Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.58%. Comparing base (74f16ed) to head (2271e43).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #199   +/-   ##
=======================================
  Coverage   98.58%   98.58%           
=======================================
  Files          72       72           
  Lines       14367    14367           
=======================================
  Hits        14163    14163           
  Misses        204      204           
Flag Coverage Δ
notebooktests 19.04% <ø> (ø)
unittests 97.65% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@syedhamidali
syedhamidali merged commit 1d986e2 into main Oct 10, 2026
30 of 31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant