Skip to content

Fix tracksFromOPMD Nit_min/Nit_max codepath - #41

Open
berceanu wants to merge 2 commits into
hightower8083:devfrom
berceanu:fix/nit-min-max-codepath
Open

Fix tracksFromOPMD Nit_min/Nit_max codepath#41
berceanu wants to merge 2 commits into
hightower8083:devfrom
berceanu:fix/nit-min-max-codepath

Conversation

@berceanu

@berceanu berceanu commented May 5, 2026

Copy link
Copy Markdown
Contributor

Summary

Two interlocked bugs make the Nit_min/Nit_max arguments of
tracksFromOPMD unusable in the current dev HEAD (7e0d82e):

  1. Boolean-mask shape mismatch. The chained-filter form

    iteration_ind = np.arange(iterations.size)
    iteration_ind = iteration_ind[iterations >= Nit_min]   # shrinks
    iteration_ind = iteration_ind[iterations <= Nit_max]   # IndexError

    raises IndexError whenever both bounds are set: the second filter
    applies a full-length boolean mask to the already-shrunken
    iteration_ind. Replaced with a single combined boolean mask,
    which is also clearer.

  2. Trajectory arrays never sliced to the window. The slice that
    should follow the iteration filter sits commented out a few lines
    below (# temporal patch to select iterations):

    #for var in var_list:
    #    TC[var] = TC[var][:, iteration_ind]

    ts.iterate walks the full ts.iterations regardless of the
    filter, so each TC[var] returns with shape (N_selected, ts.iterations.size) — but the windowed iterations / t arrays
    built just above suggest the smaller series. Downstream
    it_start indexing inside split_track_by_nans then references
    the wrong column count and produces wrong-length tracks (or
    IndexError's on t[it_start:it_start+z.size] when z_is_xi=True).

    Restored the slice, gated on either bound being set so the
    no-window code path stays untouched, and added a comment explaining
    why it has to live there (after the list-of-lists → ndarray
    conversion).

Context

Hit while integrating SynchRad into a hybrid LWFA→PWFA betatron
pipeline at ELI-NP (FBPIC's BackTransformedParticleDiagnostic
tracksFromOPMDSynchRad.calculate_spectrum). Was working
around bug (1) downstream by mutating ts.iterations / ts.t before
the call; this PR makes the upstream API match its docstring so the
workaround can be removed.

Test plan

  • Syntax checkpython -c "import ast; ast.parse(open('synchrad/converters.py').read())" ⇒ ok

  • End-to-end equivalence on a real FBPIC extract. Ran the
    same iteration window twice over a 3001-snapshot
    BackTransformedParticleDiagnostic tree (FBPIC pure-N₂ run, ~1.94M
    candidate particles, sequential sampling of 200):

    • Mode A — stock @ 7e0d82e + the local ts.iterations-mutation
      workaround (Nit_min/Nit_max not passed to tracksFromOPMD).
    • Mode B — PR-patched tracksFromOPMD with Nit_min=2950, Nit_max=3000 passed directly, ts.iterations left untouched.

    Result (200 common track IDs out of 200 written by each):

    check A B Δ
    N_particles 200 200 0
    it_range [0, 51] [0, 51] match
    cdt 6.023597e-06 6.023597e-06 0.00e+00
    shape match (20-track sample) 20/20
    it_start match (20-track sample) 20/20
    max |Δ| per field — x, y, z, ux, uy, uz, w 0.000000e+00 (all)

    PR-patched output is bit-exact with the workaround's output.

  • No-window code path (Nit_min=Nit_max=None) — the new gate
    at line 109 (if Nit_min is not None or Nit_max is not None:) only
    fires when at least one bound is set, so the no-window path is
    untouched.

Verification script + raw output available on request.

berceanu added 2 commits May 5, 2026 12:56
Two bugs lived in this branch since at least 7e0d82e ("improved
converter verbosity").  Together they made the iteration-window
arguments unusable:

1. The chained-filter form

       iteration_ind = np.arange(iterations.size)
       iteration_ind = iteration_ind[iterations >= Nit_min]   # shrinks
       iteration_ind = iteration_ind[iterations <= Nit_max]   # IndexError

   IndexError'd whenever both bounds were given: the second filter
   tried to apply a full-length boolean mask to the already-shrunken
   `iteration_ind`.  Rewrote as a single combined boolean mask, which
   is also clearer.

2. Even when only one bound was set (so the filter quietly succeeded),
   the trajectory arrays were never sliced to the selected window —
   the slice was sitting commented-out a few lines below ("temporal
   patch to select iterations"):

       #for var in var_list:
       #    TC[var] = TC[var][:, iteration_ind]

   Without this slice, `TC[var]` keeps shape (N_selected,
   ts.iterations.size) (the FULL series — `ts.iterate` walks
   `ts.iterations` regardless), while the windowed `iterations` / `t`
   that we just built suggest a smaller series.  The downstream
   `it_start` indexing inside `split_track_by_nans` then references
   the wrong column count and either silently produces wrong-length
   tracks or IndexError's on `t[it_start:it_start+z.size]`.

Restored the slice (gated on either bound being set so the no-window
code path is unaffected) and added a comment explaining why it has to
exist.

Tested locally on a 6-node FBPIC `BackTransformedParticleDiagnostic`
extract (3001 snapshots) with `Nit_min=2500`, `Nit_max=3000`: produces
the expected windowed `tracks.h5`.  Without the patch, both bug paths
trigger in our pipeline; we currently work around (1) by mutating
`ts.iterations` / `ts.t` before the call.
Removed outdated comments regarding iteration filtering.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant