Make calphy work with LAMMPS executables - #270
Merged
Merged
Conversation
Freeze current (pylammpsmpi) behavior as reference data before the ExecutableRunner refactor. - tests/baselines/BASELINE_INFO.md: env, test-suite status (105 passed, 2 skipped), per-scenario results, and deferral notes. - tests/baselines/B1-B10: report.yaml + calphy.log + input for 10 Cu EAM scenarios (fe solid/liquid, ts, tscale, pscale, alchemy; fix_lattice and finite-pressure paths). Physics cross-checks: B1~B9, B6~B8, B4~B5~B7. - tests/baselines/inputs/B1-B12.yaml: all 12 scenario inputs (FCC Cu 5x5x5, 4 cores). B11/B12 inputs kept but deferred. - tests/fixtures/: reference data files for the Part 2 mock tests. - scoped .gitignore negations so the *.dat/*.data/traj.*/*log* contract files are trackable. Deferred (not captured): - B11 (fe-qtb): this LAMMPS build lacks the QTB package. - B12 (prescan): pre-existing bug -- calphy/phase.py imports calphy.range_scan which does not exist; phase_transition_detection.mode != none crashes.
Reduce the LAMMPS interface calphy uses to command() only, so the driver
object can later be swapped for the executable runner.
- phase.py check_if_melted / check_if_solidfied: use self.natoms (set in
__init__ from calc._natoms) instead of the library read lmp.natoms -- the
only non-command reads calphy made.
- liquid.py rattle_structure / melt_structure / run_averaging: replace the
magic-attribute calls lmp.velocity(...) and lmp.run(...) with explicit
lmp.command("velocity all create %f %d" ...) / lmp.command("run %d" ...).
- helpers.py: drop the unused `from lammps import lammps` import.
Verified: full suite 105 passed / 2 skipped (== part 0); B4 (melt) and B5
(rattle) end-to-end reproduce their baselines within 5e-4 eV/atom.
Freeze the exact LAMMPS command stream each driver path emits, on the current (pylammpsmpi) code, as the contract the refactor must preserve (Part 5.9). - tests/conftest.py: RecordingRunner (records command() strings, executes nothing), make_calc / recorded_job fixtures (seed np.random, monkeypatch create_object onto a shared recorder reusing the real init-command emission, stub find_solid_fraction, stage avg.dat/msd.dat), and assert_golden (write-on-first-run, byte-compare after). normalize_command strips the absolute simfolder path so goldens are machine/run independent. - tests/test_golden_streams.py: 18 goldens covering solid/liquid/alchemy averaging + integration, ts forward/backward/uniform-temperature/mc-swaps, tscale, pscale, qtb averaging+integration, overlay potential. prescan is a skipped placeholder (blocked on the missing calphy.range_scan module). - tests/golden/*.txt: the frozen streams (portable, deterministic across two runs, no repr/path leakage). - tests/test_command_vocabulary.py: every golden's first token is in the Appendix A vocabulary; goldens non-empty and repr-clean. Convergence loops are forced to cycle 1 via loosened tolerances (a Python-side check, absent from the command stream); box/spring stand-ins feed the integration/sweep methods. Full suite: 160 passed, 3 skipped.
New, self-contained module (no wire-in yet) that drives an external LAMMPS
binary in restart-continued segments.
- errors.py: LammpsExecutionError, RunnerStateError.
- runner.py:
* resolve_lammps_executable / resolve_mpi_executable
(input key -> $CALPHY_*_EXECUTABLE -> name on PATH; clear ValueError).
* SessionState: tracks the sticky state replayed after each read_restart
(init header, pair_block, groups, computes, variables [position kept on
redefine], fixes [+ fix_modify], dumps, thermo). Classifies every command
per Appendix A; unknown token -> RunnerStateError.
* ExecutableRunner: command() buffers + classifies; sync() flushes one
segment (seg 0 verbatim + write_restart; seg k>0 = replay header per 3.4
ordering + read_restart + buffer + write_restart), runs
[mpirun -np N] lmp -in seg -log seglog -screen none, and fails on nonzero
rc / timeout / ERROR-in-log with a LammpsExecutionError excerpt. Replayed
fix `file X` -> `file X.seg<k>`; replayed fix print / append -> error;
live dump or immediate-eval variable crossing a boundary -> error.
close(), rotate_logs(), logical_commands, dry_run.
* read_timeseries: concatenates avg.dat + avg.dat.seg* across segments;
single-file case reproduces np.loadtxt exactly.
- __init__.py: defer the LAMMPS-backend driver imports when the backend is
absent, so calphy.runner / calphy.errors import without pylammpsmpi
(re-raising any non-backend ImportError).
- tests/test_runner.py: 73 tests (state machine, exact segment-1 header,
execution layer via monkeypatched subprocess, resolution, read_timeseries,
no-pylammpsmpi import). runner.py coverage 97%. Full suite 233 passed.
Verify the LAMMPS binary provides every style a calculation needs, before the
first segment runs (wired in at Part 5).
- runner.py:
* get_binary_styles(binary): parse the `* Pair/Fix/Compute styles:` sections
of `lmp -h` (tolerant of column wrapping); cached per (path, mtime).
* required_styles(calc): derive needed pair/fix/compute styles from the calc
(always nve/nvt/npt/langevin/ave/time/print/momentum + msd/temp/com/pair
and the user pair styles; ti/spring for solid fe/ts/tscale; ufm +
hybrid/scaled for liquid / melting_temperature; hybrid/scaled for
ts/tscale/overlay; atom/swap for MC swaps; qtb for fe-qtb).
* preflight(calc, binary): raise ValueError listing each missing style with
its LAMMPS package. Honors CALPHY_SKIP_PREFLIGHT=1. If the binary cannot
report styles at all, log a warning and skip rather than false-flag
everything (some builds crash on `-h` -- observed rc=139 on the conda-forge
lammps wheel in this env, which is exactly why the skip path exists).
- tests/test_preflight.py: 12 tests over canned full/minimal `-h` fixtures
(parser, pass/fail per mode, MC-swap and QTB requirements, per (path,mtime)
cache, skip-hatch, unreadable-`-h` skip-with-warning).
No changes to phase/solid/liquid/alchemy. runner.py coverage 97%.
Full suite 245 passed, 3 skipped.
calphy now drives the external LAMMPS binary through ExecutableRunner; the pylammpsmpi library path and the entire script_mode / staged-CLI machinery are removed. Every Part 2 golden command stream is byte-identical through ExecutableRunner(dry_run=True).logical_commands, so no physics command changed. - helpers.create_object(calc, directory) -> configured ExecutableRunner (binary/mpi resolution + preflight); init-command emission split into emit_init_commands(); LammpsScript class and the pylammpsmpi import deleted; all 12 call sites updated. - 4 sync() points (dump_current_snapshot + the three convergence loops); the prescan sync is deferred with the missing range_scan module. - cumulative avg.dat/msd.dat reads -> ph.read_timeseries (stitches .seg* files). - deleted all run_minimal_* twins + dispatchers (iterative bodies renamed to the stable dispatcher names) and process_averaging_results; deleted every script_mode branch and lmp.write script-emit block. - removed the lmp=/self._lmp injection; lammps_close -> lmp.close(); every os.rename(log.lammps -> <stage>.log.lammps) -> lmp.rotate_logs(<stage>) (including a close+rotate for pressure_scaling, which never closed lmp before). - input.py: script_mode: True now raises a clear migration error; deleted savefile / save_job / load_job. - clitools: dropped the four staged-CLI functions; pyproject dropped the four staged entry points and the pylammpsmpi dependency; environment.yml dropped pylammpsmpi (lammps stays -- it ships the lmp binary calphy now drives). - tests/conftest.py: recorded_job drives an ExecutableRunner(dry_run=True) and compares normalized logical_commands. Full suite 245 passed / 3 skipped. calphy imports and its CLIs run without pylammpsmpi installed.
First end-to-end validation of the segmented ExecutableRunner against the real lmp binary (tests/test_integration_lammps.py, @pytest.mark.lammps, skipped when no lmp resolves). Continuity core: - T1 exact-restart PE continuity across a segment boundary (<= 1e-6 relative). - T2 Nose-Hoover chain restoration (no volume/pressure kick at the boundary). - T3 fix ave/time file -> file.seg<k> redirection; read_timeseries stitches a strictly-increasing step column. - T4 group/compute/variable (MSD) replay after read_restart. End-to-end matrix: B1-B10 re-run through calphy_kernel reproduce the frozen baseline free energies (tol floor 5e-3 eV/atom -- the tiny baseline settings scatter ~3-5e-3 run-to-run, larger than the switching-hysteresis error bar and inherent to the settings, not the refactor). Spring constant is informational only (bistable from 2-decimal MSD rounding + unseeded RNG in the old code too). Error paths: broken pair_coeff -> LammpsExecutionError with the ERROR excerpt; resolution ValueError names all three lookup steps; fe-solid at 5000 K -> MeltedError + melted_error.log.lammps. MPI: T1 continuity holds under -np 2. Two real bugs found and fixed in runner.py: - read_timeseries crashed on usecols=None across segment files with differing row counts (single-row files squeeze to 1D); now ndmin=2 + vstack + squeeze. - preflight style parser missed the "* Fix styles" heading (this binary omits the trailing colon that Pair/Compute headings carry), so it reported every fix style missing; heading match is now colon-tolerant. pyproject: register the lammps / slow markers.
calphy v2 runs an external lmp binary; update the docs and env files to match (no CI / CHANGELOG / version bump in this commit). - gettingstarted.md: rewrite the LAMMPS story. calphy no longer needs LAMMPS as a python library; it drives an lmp binary resolved as lammps_executable -> $CALPHY_LAMMPS_EXECUTABLE -> lmp on PATH (mpirun the same way for cores > 1). New "Installing LAMMPS" section: a package-requirement table (EXTRA-FIX ti/spring, EXTRA-PAIR ufm/hybrid-scaled, MC atom/swap, QTB fix qtb) and three routes (conda-forge, HPC module/existing binary, manual build of the executable with the right cmake flags), plus a preflight note and a `lmp -h` check. - inputfile.md: script_mode is now a removal notice; lammps_executable / mpi_executable apply to every mode with the documented resolution order; cmdargs documented as appended to the lmp argv. - environment-docs.yml, environment-nolammps.yml: drop pylammpsmpi / mpi4py. - calphy.def: drop pylammpsmpi / mpi4py; build the lmp executable (add MC + QTB packages) and put it on PATH instead of building a shared library + make install-python.
report.yaml now records the switching dissipation that was previously
computed and discarded:
- results.dissipation (fe/alchemy): mean 0.5*(W_fwd+W_bwd), a switching
irreversibility metric
- results.ts_dissipation (ts/tscale): max energy dissipation along the
reversible-scaling sweep (folded in via Phase._amend_report, since the
base report is written before the sweep runs)
Every *.dat output now carries a '#'-commented column legend with units:
- temperature_sweep.dat / pressure_sweep.dat via numpy header
- forward/backward, ts.*, ps.*, avg.dat, msd.dat via LAMMPS title/title2
Golden command streams regenerated: the only change is the added title
strings (verified 29/29 diffs are title-only insertions).
Add docs/source/outputfiles.md describing every output file (report.yaml blocks, free-energy curves, switching/averaging data, trajectories, logs) with column/unit tables, wired into the toctree and cross-linked with the input-file keys both ways. Clarify in inputfile.md that lambda_schedule changes only the sampling density along the temperature axis, not the computed free energy, and note the default stays 'linear'.
Restores calphy/range_scan.py (ported from the main branch), the pure-numpy
detector behind Phase.scan_temperature_range. Its RangeScan / plot_scan /
ScanResult API already matched what phase.py expected, and all LAMMPS driving
stays in phase.py (executable-runner native), so no backend changes were
needed. matplotlib is imported lazily via the Agg backend (headless-safe).
- prescan.forward.dat gains a column-legend header, consistent with the
other data files
- un-skip test_prescan; add the prescan.txt golden
- BASELINE_INFO.md: B12 flipped from BLOCKED to RESOLVED
- document prescan.forward.dat / prescan_signals.png in outputfiles.md
Validated end-to-end against real LAMMPS (B12): ramp 400->700 K, RangeScan
correctly reports no transition for solid Cu in-range, prescan_signals.png
written, ts sweep runs unchanged.
… runner A runner now owns all traffic across the LAMMPS boundary: command strings in through command(), results back through data accessors. BaseRunner carries the command vocabulary check, logical_commands recording, and a file-based read_timeseries default; ExecutableRunner subclasses it and keeps its segment machinery private. process_pressure, finalise_pressure and analyse_spring_constants take the runner and read through lmp.read_timeseries(), so a future backend can swap file reads for direct library calls without touching driver code.
Drives a live pylammpsmpi.LammpsLibrary session behind the BaseRunner contract: commands forward immediately, sync() is a no-op (no segments, no restart replay, no .seg files), rotate_logs rotates a scratch -log target and tolerates the drivers' close-before-rotate ordering. pylammpsmpi is imported lazily inside the constructor with an error pointing at pip install calphy[library]; importing the module never needs it. Unit-tested over a fake pylammpsmpi.
…kend-equivalence goldens New input key execution_mode: executable | library (default executable, validated, propagated through read_yamlfile). create_object branches once: library mode builds a LibraryRunner (lazy import); preflight and binary/MPI resolution stay executable-only. pyproject gains the calphy[library] extra; the dead ImportError guard in __init__ is removed (nothing imports pylammpsmpi at import time anymore). Unified-backend guarantee: recorded_job gained runner='library', and the library backend is asserted against the same immutable goldens the executable backend froze (solid_fe_averaging, liquid_fe_averaging_meltcycle) - byte-identical logical command streams. Verified against real pylammpsmpi (calphy2 env): B1 solid fe end-to-end in library mode gives -3.64465 eV/atom vs baseline -3.64386 (within the 5e-3 e2e tolerance). Decision 5 recorded as a plan addendum; docs updated.
The executable path stays the primary route; a new optional section at the end of the installation page covers execution_mode: library setup: pip install calphy[library], conda-forge lammps (covers both backends in one install), the BUILD_LIB=ON compile recipe with make install-python, and a warning that the lammps python module and liblammps must be the same LAMMPS version - a mismatch hangs library-mode runs at startup instead of erroring cleanly. Cross-linked from the execution_mode input key reference.
…ndency floors Ports the upstream changes onto the two-backend architecture: - box tilt large dropped from the init commands (ICAMS#262: the box command is deprecated in LAMMPS since 22Dec2022 and its arguments are ignored). The box token stays in the runner vocabulary so md.init_commands overrides keep validating. All 19 goldens regenerated; verified the only diff in every file is the removed box line. - MLIAP activation (ICAMS#260) ported into LibraryRunner: when the lammps python build ships lammps.mliap, activate_mliappy (or the kokkos variant when -k/-kokkos is in cmdargs) is called on the live session. On pylammpsmpi <= 0.4.1, which lacks the activation proxies, warn and skip instead of crashing - the calphy[library] extra now pins pylammpsmpi>0.4.1 to match upstream. - pydantic>=2.3.0, numpy>=2.0.0 floors adopted; pylammpsmpi stays out of the core dependencies (optional library extra only). Version 1.8.5. Verified: 269 unit tests green; all 18 real-LAMMPS integration tests (executable backend) green; B1 solid-fe e2e in library mode gives -3.64428 eV/atom vs baseline -3.64386 (tol 5e-3).
pylammpsmpi is no longer a core dependency, so the upgrade steps in the main test matrix are gone. Since environment.yml provides the conda-forge lammps binary, the @pytest.mark.lammps integration tests (executable backend) now run in CI as well. A new ubuntu job installs calphy[test,library] and drives a real pylammpsmpi session through LibraryRunner (commands, read_timeseries, close-then-rotate) to cover the optional library backend.
Unknown input keys are now hard errors instead of being silently ignored
- a typo like n_iteration previously ran the calculation with the default
n_iterations and nothing ever surfaced it. All input models share a
_StrictInput base (pydantic extra=forbid plus a before-validator), so the
error carries the best available hint:
- did-you-mean via difflib against the block's own keys
- wrong-category: a real key in the wrong block points at where it
belongs (timestep at the calculation level -> 'md:' block), derived
from a lazily built field->block map with no per-model code
- removed-in-v2 keys (savefile/save_job/load_job) get a migration
message
Top-level file keys other than 'calculations:' are rejected too. The
fast path (read_inputfile(validate=False), important for inputs with
many calculations) is unchanged and skips all validation as before.
The pre-v1 legacy input converter is removed (_convert_legacy_inputfile,
the calphy_convert_input entry point); legacy-format files now get a
clear error pointing at 'pip install calphy<2' for one-time conversion.
The sweep this enabled found real bugs in shipped inputs, now fixed:
examples 01/02 (and their docs mirrors) had equilibration_control inside
md: where it was silently ignored - those examples never used the
Nose-Hoover control they intended; docs example_02 set a
peak_threshold key that is not wired to anything; tests/inp1.yaml had
two typo'd damping keys. A regression test validates every shipped
example input stays key-clean.
md.thermostat_damping/barostat_damping declared Union[float, 2-list] but
Field(gt=0) made the list branch fail validation ('Unable to apply
constraint gt'), and the drivers never handled a user-supplied list
either (Phase.__init__ would have nested it while building the runtime
[equilibration, integration] pair). The docs only ever documented the
scalar. Declare them as the scalar they are; a list input now gets a
clean type error. Equilibration-stage damping stays configurable via the
nose_hoover/berendsen/quantum_thermal_bath blocks, so no functionality
is lost.
All 34 random draws in the drivers (velocity create, langevin, MC atom/swap, fix qtb seeds, composition-scaling atom picks) come from the np.random stream, which Phase.__init__ now seeds from the new md.seed input key. When the key is unset (default) a fresh seed is drawn per run, preserving statistical independence of repeated runs; either way the seed actually used is logged and backfilled into the simfolder copy of input_file.yaml BEFORE serialisation - so every run, seeded or not, can be reproduced exactly by rerunning that file. The quantum_thermal_bath seed key is removed: it was declared with a default of 880302 but never consumed anywhere - the qtb fixes always drew a random seed. Setting it now errors with a pointer to md.seed. Golden command streams are unchanged: the test fixtures pin md.seed=42, which reproduces the historical np.random.seed(42) stream exactly. A fixed seed makes the LAMMPS command stream deterministic; bitwise trajectory reproducibility additionally needs the same LAMMPS build and core count (documented in the new inputfile.md entry).
…e 04 notebook text
…n pycalphad to_tdb now writes the build_calphad_surface 6-term L_k(T) polynomials verbatim for full-range phases (L_temperature_form='poly6', default), so the TDB free energy is identical to calphy's CALPHAD surface and a pycalphad binplot matches calculate(calphad_surface=True) inside the validity range. The legacy ridge-regularised a+bT refit stays available as 'linear' (it drifted up to ~20 meV/atom and narrowed the melting lens). Limited-range ordered phases get a reworked bounded fit: - fit against dense pseudo-data from the per-T polynomial fits (get_phase_free_energy) instead of the raw composition points, which keeps the well-minimum location faithful near the dome top - RK order auto-selected (3-6) by matching the dissolution temperature - 3-term L(T) = a + b*T + c*T*lnT - convexity constraint inside the data window (no W-shaped wells / spurious ICAMS#2 composition sets) - G >= G_host + 1 meV outside the window, with constraint points clustered toward the pure endpoints - weak scaled ridge (1e-8) to suppress huge cancelling L magnitudes that dipped below the host between constraint samples from_tdb parses the new output unchanged; metadata records the L form and per-phase fit settings. Validated on CuAu LDA/GGA/rSCAN MD data: dissolution temperatures within ~5 K of the data, melting lens exact.
…e stale gitlab config
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Significant changes: