Fix tracksFromOPMD Nit_min/Nit_max codepath - #41
Open
berceanu wants to merge 2 commits into
Open
Conversation
Two bugs lived in this branch since at least 7e0d82e ("improved converter verbosity"). Together they made the iteration-window arguments unusable: 1. The chained-filter form iteration_ind = np.arange(iterations.size) iteration_ind = iteration_ind[iterations >= Nit_min] # shrinks iteration_ind = iteration_ind[iterations <= Nit_max] # IndexError IndexError'd whenever both bounds were given: the second filter tried to apply a full-length boolean mask to the already-shrunken `iteration_ind`. Rewrote as a single combined boolean mask, which is also clearer. 2. Even when only one bound was set (so the filter quietly succeeded), the trajectory arrays were never sliced to the selected window — the slice was sitting commented-out a few lines below ("temporal patch to select iterations"): #for var in var_list: # TC[var] = TC[var][:, iteration_ind] Without this slice, `TC[var]` keeps shape (N_selected, ts.iterations.size) (the FULL series — `ts.iterate` walks `ts.iterations` regardless), while the windowed `iterations` / `t` that we just built suggest a smaller series. The downstream `it_start` indexing inside `split_track_by_nans` then references the wrong column count and either silently produces wrong-length tracks or IndexError's on `t[it_start:it_start+z.size]`. Restored the slice (gated on either bound being set so the no-window code path is unaffected) and added a comment explaining why it has to exist. Tested locally on a 6-node FBPIC `BackTransformedParticleDiagnostic` extract (3001 snapshots) with `Nit_min=2500`, `Nit_max=3000`: produces the expected windowed `tracks.h5`. Without the patch, both bug paths trigger in our pipeline; we currently work around (1) by mutating `ts.iterations` / `ts.t` before the call.
Removed outdated comments regarding iteration filtering.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two interlocked bugs make the
Nit_min/Nit_maxarguments oftracksFromOPMDunusable in the currentdevHEAD (7e0d82e):Boolean-mask shape mismatch. The chained-filter form
raises
IndexErrorwhenever both bounds are set: the second filterapplies a full-length boolean mask to the already-shrunken
iteration_ind. Replaced with a single combined boolean mask,which is also clearer.
Trajectory arrays never sliced to the window. The slice that
should follow the iteration filter sits commented out a few lines
below (
# temporal patch to select iterations):ts.iteratewalks the fullts.iterationsregardless of thefilter, so each
TC[var]returns with shape(N_selected, ts.iterations.size)— but the windowediterations/tarraysbuilt just above suggest the smaller series. Downstream
it_startindexing insidesplit_track_by_nansthen referencesthe wrong column count and produces wrong-length tracks (or
IndexError's on
t[it_start:it_start+z.size]whenz_is_xi=True).Restored the slice, gated on either bound being set so the
no-window code path stays untouched, and added a comment explaining
why it has to live there (after the list-of-lists → ndarray
conversion).
Context
Hit while integrating SynchRad into a hybrid LWFA→PWFA betatron
pipeline at ELI-NP (FBPIC's
BackTransformedParticleDiagnostic→tracksFromOPMD→SynchRad.calculate_spectrum). Was workingaround bug (1) downstream by mutating
ts.iterations/ts.tbeforethe call; this PR makes the upstream API match its docstring so the
workaround can be removed.
Test plan
Syntax check —
python -c "import ast; ast.parse(open('synchrad/converters.py').read())"⇒ okEnd-to-end equivalence on a real FBPIC extract. Ran the
same iteration window twice over a 3001-snapshot
BackTransformedParticleDiagnostictree (FBPIC pure-N₂ run, ~1.94Mcandidate particles, sequential sampling of 200):
ts.iterations-mutationworkaround (
Nit_min/Nit_maxnot passed totracksFromOPMD).tracksFromOPMDwithNit_min=2950, Nit_max=3000passed directly,ts.iterationsleft untouched.Result (200 common track IDs out of 200 written by each):
PR-patched output is bit-exact with the workaround's output.
No-window code path (
Nit_min=Nit_max=None) — the new gateat line 109 (
if Nit_min is not None or Nit_max is not None:) onlyfires when at least one bound is set, so the no-window path is
untouched.
Verification script + raw output available on request.