Skip to content

fix(analysis): refuse non-integral md.csv step values instead of truncating (AUD-001) - #76

Merged
hydrogen1222 merged 3 commits into
mainfrom
fix/aud-001-fractional-step-truncation
Nov 2, 2026
Merged

hydrogen1222 merged 3 commits into
mainfrom
fix/aud-001-fractional-step-truncation

Conversation

@hydrogen1222

@hydrogen1222 hydrogen1222 commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner

Tracking: #75 (Living code audit & exhaustive scientific test campaign)

AUD-001 — trajectory step integerity / data integrity

Severity: P0 hypothesis → confirmed data-integrity defect (no scientific
miscomputation demonstrated; see "Impact" below).

Defect (red, reproduced on baseline main @ 1ef9a1be)

TrajectoryDataset.load read the finite raw/md.csv step column and cast it
with astype(int) without checking integrality:

values = np.asarray(thermo["step"], dtype=float)
if np.all(np.isfinite(values)):
    steps = values.astype(int)          # 0.5, 1.5, 2.5, 3.5 -> 0, 1, 2, 3

Consequences reproduced:

  • dataset.steps was silently fabricated: a file with 0.5, 1.5, 2.5, 3.5
    loaded as 0, 1, 2, 3.
  • The fabricated axis was re-exported by thermodynamic_diagnostics
    (columns["step"]), so an output artifact disagreed with its input.
  • analysis_view() / slice_frames() recompute frame_stride_steps from
    self.steps, so a fabricated integral column can override the trusted
    metadata stride.
  • A step value beyond the int64 range wrapped (astype) with a numpy
    RuntimeWarning: invalid value encountered in cast.
  • TrajectoryDataset.from_frames(steps=[0.5, 1.5, ...]) had the same
    np.asarray(..., int) truncation.

The module already guards this elsewhere: _uniform_integer_step returns
None for non-integral values. The two astype(int) sites were the gap.

Fix (green)

_validated_steps() requires finite, integral, in-range values and returns
int64; it is used by both cast sites. Invalid input now fails closed with a
clear ValueError naming the offending values, instead of inventing a step
axis that is not in the file (AGENTS.md rules 2 and 3).

Non-finite md.csv step columns keep the pre-existing "step axis unknown"
fallback; only the audited finite-truncation path changed.

Tests

Red before the fix, green after (tests/mliport/mliport/analysis/test_dataset_validation.py):

Test Before After
test_fractional_md_csv_steps_are_rejected_not_truncated FAIL (DID NOT RAISE) PASS
test_out_of_range_md_csv_steps_are_rejected FAIL (DID NOT RAISE, cast warning) PASS
test_from_frames_rejects_fractional_steps FAIL (DID NOT RAISE) PASS
test_integral_float_md_csv_steps_still_load (guard) PASS PASS

Evidence

  • Baseline main @ 1ef9a1bea5033dc1a1664d8450f4b4f9edfbd91d, Python 3.12.13,
    pytest 9.1.1, numpy 2.5.4, CPU-only .venv-dev.
  • Baseline full suite: 1649 passed, 33 skipped.
  • With this branch: 1653 passed, 33 skipped (pytest tests -q -p no:cacheprovider -rsx).
  • Focused: pytest tests/mliport/mliport/analysis -q → 130 passed, 23 skipped.
  • ruff check mliport/mliport tests scripts → All checks passed;
    ruff format --check → clean.
  • scripts/check_repository_hygiene.py → passed.
  • scripts/generate_docs.py --check → up to date.
  • test_docs_contract.py → 56 passed.

DATA IMPACT

No scientific record invalidated. No stored trajectory, md.csv or evidence
file is rewritten or regenerated. Only the handling of an already-invalid step
column changes, from silent truncation to an explicit rejection.

Residual / not covered

  • Real-backend GPU runs were not exercised; this is a pure CPU data-contract fix.
  • Non-finite (NaN/Inf) step columns still fall back to
    info/arange(nframes) * stride rather than failing closed. That is a
    separate, pre-existing behaviour and was left untouched to keep this PR minimal.

Rev 2 — review follow-up (commit ff71072b, rebased onto main @ 77a94afd)

Addresses the "precision edge" review. Integer input is never routed through
float again, and the CSV step column is validated on its text:

  • integer arrays keep their exact value; only the asymmetric int64 range is
    enforced (-2**63 accepted, 2**63 rejected);
  • float arrays must be finite, integral and within |v| <= 2**53;
  • _parse_step_column reads each CSV cell with decimal.Decimal, so
    "9007199254740993" and "9007199254740993.0" both keep their exact value;
  • blank / unparseable / non-finite cells keep the pre-existing "axis unknown"
    fallback (still a tracked follow-up, not expanded here).

Regressions added: test_from_frames_preserves_large_integer_steps_exactly,
test_from_frames_accepts_int64_min_and_rejects_overflow,
test_md_csv_huge_integer_step_text_is_parsed_exactly,
test_md_csv_float_literal_step_keeps_its_exact_integer_value,
test_md_csv_unparseable_step_makes_the_axis_unknown. Full suite: 1661
passed, 33 skipped
; analysis suite 135 passed / 23 skipped; ruff clean; CI
green on 3.10–3.13.


Rev 3 — round-2 review follow-up (commit 2adfc2ef)

Fixes the remaining precision hole: a heterogeneous Python sequence such as
[2**53 + 1, 2.0] was promoted to float64 by np.asarray and the integer was
rounded to 2**53 before any dtype check could see it.

  • np.ndarray input keeps the vectorised dtype path (_validated_step_array).
  • Any other sequence is validated element by element via _step_index()
    (Python int first — exact, never through float — then float with
    finite / integral / |v| <= 2**53), and the int64 array is only built
    afterwards.
  • Hardening: _parse_step_column compares the Decimal to the int64 bounds
    before int(integral). On the previous commit 1e999999999 made the
    process hang (the regression run timed out at 300 s); it is now rejected in
    < 1 ms.

Regressions: test_from_frames_preserves_mixed_integer_and_float_steps_exactly,
test_from_frames_rejects_invalid_mixed_sequences (4 cases),
test_md_csv_astronomical_step_exponent_is_rejected. Full suite: 1667
passed, 33 skipped
; analysis suite 141 passed / 23 skipped; ruff clean.

@hydrogen1222

Copy link
Copy Markdown
Owner Author

Additional real-artifact evidence (not synthetic fixtures):

Loaded every real mliport run under runs/ with this branch (26 run directories
with raw/md.csv, including the MACE t6-mace/search cycles and the
speed-nod3 / speed-d3 baselines):

run dirs with md.csv: 26
loaded=26 failed=0

All of them have integral step columns (e.g. [0, 100, 200], 81 frames) and
still load, so the new integrality gate does not reject any real mliport
artifact. The gate only fires for a fractional or out-of-range step column,
which mliport's own writer (int(frame["step"])) never produces — i.e. it
targets corrupt or externally produced md.csv files. Script:
mliport-audit/repro/verify_real_runs.py (scratch, not committed).

@hydrogen1222 hydrogen1222 left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Audit review — changes requested before merge (precision edge, not caught by CI).

The finite/fractional validation is a real improvement and the red→green tests are persuasive. However _validated_steps begins with np.asarray(values, dtype=float), so TrajectoryDataset.from_frames(steps=[9007199254740993, ...]) silently changes the valid integer 2**53+1 into 2**53 before the integral/range checks. The CSV parser _read_thermodynamics likewise converts the step column through float before validation. Also -2**63 is a representable signed int64 value but the symmetric abs >= 2**63 bound rejects it (consider whether negative steps are allowed at all).

Please add a failing regression showing that an integer input is either preserved exactly or explicitly rejected (never silently rounded); validate text step columns before lossy parsing, or document/reject values beyond the exactly representable float range. Keep the fix narrow; the separately identified nonfinite-CSV fallback can remain a follow-up if explicitly tracked. CI is green, but this is a scientific identity contract.

…cating

AUD-001 (tracker #75).  TrajectoryDataset.load cast the finite raw/md.csv
"step" column with astype(int), so steps 0.5, 1.5, 2.5, 3.5 loaded as
0, 1, 2, 3.  The fabricated indices were then re-exported as the step axis
(thermodynamic_diagnostics "step" column) and used to recompute
frame_stride_steps in analysis_view()/slice_frames().  A step beyond the
int64 range wrapped as well, with a numpy "invalid value encountered in
cast" RuntimeWarning.

Step indices are integers by contract: mliport writes int steps and
_uniform_integer_step already treats non-integral values as unknown.  Add
_validated_steps(), which requires finite, integral, in-range values, and
use it in both cast sites (TrajectoryDataset.load and
TrajectoryDataset.from_frames).  Invalid input now fails closed with a
clear ValueError instead of being silently reinterpreted.

Tests (red before the fix, green after):
  test_fractional_md_csv_steps_are_rejected_not_truncated
  test_out_of_range_md_csv_steps_are_rejected
  test_from_frames_rejects_fractional_steps
  test_integral_float_md_csv_steps_still_load (guard)

DATA IMPACT: no scientific record invalidated.  No stored trajectory is
rewritten; only the handling of an already-invalid step column changes,
from silent truncation to an explicit error.

Tracking: #75
…float

AUD-001 review follow-up (tracker #75, PR #76).

_validated_steps() began with np.asarray(values, dtype=float), so an
integer input such as steps=[2**53 + 1] silently became 2**53 before the
integrality/range checks could see it, and the md.csv step column was
parsed through float() for the same reason.  The symmetric abs() >= 2**63
bound also rejected the representable int64 minimum -2**63.

Dispatch on the input dtype instead:

  * integer input is never routed through float; only the asymmetric
    int64 range is enforced (so -2**63 is accepted, 2**63 is rejected);
  * float input must be finite, integral and within the exactly
    representable float range (|v| <= 2**53);
  * a CSV step column is parsed exactly from its text with Decimal, so
    "9007199254740993" and "9007199254740993.0" both keep their exact
    integer value.  A finite non-integral cell is rejected; blank,
    unparseable or non-finite cells keep the pre-existing "axis unknown"
    fallback (unchanged, tracked separately).

Regressions added (red before this commit, green after):
  test_from_frames_preserves_large_integer_steps_exactly
  test_from_frames_accepts_int64_min_and_rejects_overflow
  test_md_csv_huge_integer_step_text_is_parsed_exactly
  test_md_csv_float_literal_step_keeps_its_exact_integer_value
  test_md_csv_unparseable_step_makes_the_axis_unknown

DATA IMPACT: no scientific record invalidated.  No stored trajectory is
rewritten; only the handling of a step column changes.

Tracking: #75
@hydrogen1222
hydrogen1222 force-pushed the fix/aud-001-fractional-step-truncation branch from 6e7cab9 to ff71072 Compare November 1, 2026 13:17
@hydrogen1222

Copy link
Copy Markdown
Owner Author

Review follow-up pushed as ff71072b (branch rebased onto main @ 77a94afd after #79 merged).

Integer input is no longer routed through float. _validated_steps now dispatches on the input dtype:

  • integer arrays keep their exact value — only the asymmetric int64 range is enforced, so steps=[2**53 + 1, ...] survives verbatim, -2**63 is accepted, and 2**63 is rejected;
  • float arrays must be finite, integral and within the exactly representable range (|v| <= 2**53);
  • bool/object/text input is rejected rather than coerced.

The CSV step column is validated on its text. _read_thermodynamics no longer runs float() over it: _parse_step_column reads each cell with decimal.Decimal, so "9007199254740993" and "9007199254740993.0" both keep their exact integer value. A finite non-integral cell is rejected; blank / unparseable / non-finite cells keep the pre-existing "axis unknown" fallback, which remains a tracked follow-up as you asked.

New regressions (verified red on the previous commit, green now):

test before
test_from_frames_preserves_large_integer_steps_exactly [2**53, 2**53+2, 2**53+4]
test_from_frames_accepts_int64_min_and_rejects_overflow -2**63 rejected
test_md_csv_huge_integer_step_text_is_parsed_exactly 9007199254740993 → 2**53
test_md_csv_float_literal_step_keeps_its_exact_integer_value 9007199254740993.0 → 2**53
test_md_csv_unparseable_step_makes_the_axis_unknown guard for the fallback

Evidence: pytest tests → 1661 passed, 33 skipped; analysis suite → 135 passed, 23 skipped; ruff check / format --check clean; CI green on 3.10–3.13.

@hydrogen1222 hydrogen1222 left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round-2 independent code review — one precision hole remains; please fix before merge.

Rev 2 correctly preserves homogeneous integer inputs and parses CSV step text exactly. However _validated_steps still starts with array = np.asarray(values). A heterogeneous Python sequence with a large integer and an otherwise legitimate integral float silently rounds before the dtype checks:

import numpy as np
steps = [2**53 + 1, 2.0]
array = np.asarray(steps)
assert array.dtype == np.float64
assert array[0] == 2**53             # integer identity was already lost
# _validated_steps(steps, source="test") returns [2**53, 2]
# without raising an error

Please add a red→green regression for TrajectoryDataset.from_frames(steps=[2**53+1, 2.0]) (use 2 frames) and preserve exact integer values or reject mixed numeric input before coercion. For array inputs, distinguish np.ndarray numeric integer vs float dtypes without rebuilding a mixed array. This is precisely the silent data-identity failure AUD-001 is intended to prevent.

A secondary hardening note (not necessarily blocking): in _parse_step_column, compare the finite Decimal to the int64 bounds before int(integral), so an input like 1e999999999 is rejected cheaply without attempting a massive integer conversion.

All current GitHub checks are green; this is a source-reviewed uncovered edge, not a claim that tests failed. Please keep #76 open until the regression is added.

…coercion

AUD-001 round-2 review follow-up (tracker #75, PR #76).

_validated_steps still began with np.asarray(values), so a heterogeneous
Python sequence such as [2**53 + 1, 2.0] was promoted to float64 and the
integer was silently rounded to 2**53 before any dtype check could see it
-- exactly the data-identity failure AUD-001 exists to prevent.

* An np.ndarray keeps the vectorised dtype path (_validated_step_array).
* Any other sequence is validated element by element via the new
  _step_index(), which checks Python int first (exact, never through
  float), then float (finite / integral / |v| <= 2**53), and rejects
  everything else.  The int64 array is only built afterwards, so numpy
  promotion can no longer round a value.

Secondary hardening: _parse_step_column now compares the Decimal against
the int64 bounds before int(integral).  This is a real robustness fix, not
just a micro-optimisation -- on the previous commit int(Decimal("1e999999999"))
tried to build a ~10**9-digit integer and the process hung.

Regressions (red before, green after):
  test_from_frames_preserves_mixed_integer_and_float_steps_exactly
  test_from_frames_rejects_invalid_mixed_sequences (4 parametrised cases)
  test_md_csv_astronomical_step_exponent_is_rejected

DATA IMPACT: no scientific record invalidated.  No stored trajectory is
rewritten; only the handling of a step axis changes.

Tracking: #75
@hydrogen1222

Copy link
Copy Markdown
Owner Author

Round-2 follow-up pushed as 2adfc2ef.

Heterogeneous sequences are validated element-wise, before any coercion. _validated_steps now dispatches:

  • an np.ndarray keeps the vectorised dtype path (_validated_step_array), exactly as you asked — no rebuilding of a mixed array;
  • any other sequence is validated element by element through the new _step_index(), which checks Python int first (exact, never routed through float), then float (finite / integral / |v| <= 2**53), then rejects anything else. The int64 array is only built after every element passed, so np.asarray promotion can no longer round an integer.

Your example now preserves the identity:

steps = [2**53 + 1, 2.0]
-> [9007199254740993, 2]      # was [9007199254740992, 2]

Secondary hardening done. _parse_step_column compares the Decimal against the int64 bounds before int(integral). This turned out to be more than cosmetic: on the previous commit int(Decimal("1e999999999")) attempted to build a ~10⁹-digit integer and the process hung — my first attempt to run the regression against ff71072b timed out at 300 s. It is now rejected in < 1 ms.

Regressions (verified red on ff71072b, green now):

test before
test_from_frames_preserves_mixed_integer_and_float_steps_exactly [9007199254740992, 2] — the exact rounding you quoted
test_from_frames_rejects_invalid_mixed_sequences (4 cases) guards: fractional / int64-overflow / NaN / non-numeric element
test_md_csv_astronomical_step_exponent_is_rejected hung on the old code

Evidence: pytest tests → 1667 passed, 33 skipped; analysis suite 141 passed / 23 skipped; ruff check / format --check clean.

@hydrogen1222 hydrogen1222 left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round-3 source/CI review of head 2adfc2ef: the previously blocking AUD-001 precision reproductions are resolved. Specifically, ordinary mixed Python numeric inputs [2**53 + 1, 2.0] are now element-validated before array coercion and preserve 9007199254740993; the exact-text CSV Decimal parser bounds the value before integer allocation, with a regression for 1e999999999. Full test matrix, lint, docs and package build all report success on this head, including Python 3.10–3.13. No further blocking concern for the documented ordinary Python int/float and CSV-text use cases; owner may consider merging #76. No real model GPU job was performed in this review.

A non-blocking, uncommon follow-up for tracker #75: from_frames currently wraps steps in list(steps), so the _validated_step_array vectorized branch is not reached through that public entry point, and the scalar _step_index casts np.longdouble to Python float. On platforms where longdouble has extended mantissa, a list/array of np.longdouble(2**53+1) may thus still lose integer identity. If such nonstandard dtype inputs are within the supported contract, add a separate regression/fix; do not re-expand this already substantial PR without an actual requirement. Likewise float16/float32 input contracts should be spelled out separately rather than assuming a float64 exactness threshold. These do not affect standard int/float CSV pathways fixed here.

@hydrogen1222
hydrogen1222 merged commit ad44f57 into main Nov 2, 2026
13 checks passed
@hydrogen1222
hydrogen1222 deleted the fix/aud-001-fractional-step-truncation branch November 2, 2026 00:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant