Split pylcm into a public `lcm/` package and a private `_lcm/` package by hmgaudecker · Pull Request #361 · OpenSourceEconomics/pylcm

hmgaudecker · 2026-05-18T19:38:01Z

Summary

I found the source code increasingly hard to navigate and to predict what might be the public interface and what we should be free to change (case in point that came up in the process: validate_transition_probs, which was removed in #361). I do not want to imagine how that may look for a user who inspects the source code for guidance of what she may use). After a couple of iteration rounds, I think pytask's strategy is the right one: Shallow public modules and all the hard work in _lcm. See the docs page explaining the internal architecture.

After this PR, pylcm's source is two packages with a hard public/private boundary:

src/lcm/ — the public surface. Everything a user constructs or consumes: the user-facing classes, the @categorical decorator, as_leaf, the public type aliases, and the exception classes. lcm/__init__.py re-exports the public symbols.
src/_lcm/ — the private implementation. The build pipeline, the canonical engine dataclasses, the JAX-traced solve / simulate machinery, validators, I/O plumbing, and the engine-side type aliases and protocols.

Public surface — `src/lcm/`

__init__.py, ages.py, categorical.py, exceptions.py, grids.py, model.py, params.py, persistence.py, processes.py, regime.py, result.py, transition.py, typing.py. Users keep writing from lcm import Model, Regime, ... — the import path is unchanged.

Private implementation — `src/_lcm/`

Every engine internal, plainly named (no leading underscore): engine.py, model_processing.py, pandas_utils.py, state_action_space.py, variables.py, dtypes.py, transition_checks.py, the bootstrap modules (jaxtyping_patch.py, beartype_conf.py, config.py), typing.py, and the grids/, processes/, persistence/, regime/, regime_building/, solution/, simulation/, params/, utils/ subpackages.

Typing split

lcm/typing.py — model-authoring aliases (jaxtyping array shapes, Period, Age) and the User* boundary aliases. Imports nothing from _lcm.
_lcm/typing.py — engine-side string labels, compound mapping aliases, canonical post-processing forms, and the structural Protocol classes.

Exceptions

The PyLCMError subclasses stay public in lcm/exceptions.py and are re-exported from lcm — both from lcm.exceptions import InvalidParamsError and except lcm.InvalidParamsError work. format_messages, internal validation plumbing, lives in _lcm/utils/error_messages.py.

Params

lcm/params.py exposes as_leaf and re-exports MappingLeaf, UserMappingLeaf, SequenceLeaf, UserSequenceLeaf. The leaf-class definitions and the engine params machinery live in _lcm/params/.

Bootstrap

_lcm/__init__.py applies the jaxtyping "..."-sentinel patch. lcm/__init__.py imports _lcm first, then registers beartype's package claw on both _lcm and lcm before any submodule loads.

Renames

interfaces.py has always been confusing me as it sounds too much like API/UI IMO. Went for engine.py after a brainstorming session, but happy to adjust that.
The "shocks" has become too narrow at least since we have been allowing for means and variances to differ from 0/1. I think the correct vocabulary for what those objects describe are (approximations to) continuous stochastic processes. The PR updates the docs to make clear that these objects bundle grids and transitions, e.g. first sentence here. The engine vocabulary has thus become process in place of shock: the seven *Process classes (UniformIIDProcess, NormalIIDProcess, LogNormalIIDProcess, NormalMixtureIIDProcess, TauchenAR1Process, RouwenhorstAR1Process, TauchenNormalMixtureAR1Process), VariableInfo.is_process, Variables.process_names, and the ProcessName typing alias. Each *Process class bundles a discretization grid and a transition mechanism — instances go in Regime(states={...}). All of these are directly importable from lcm now.
Piece has been renamed to PiecewiseGridSegment (looking at the public surface told me this needed explanation) and the corresponding keywords from pieces to segments (we were using that in prose, anyhow)

Build

hatch-vcs writes the generated version file to src/_lcm/version.py.

Migration guide for downstream code

from lcm import Model, Regime, AgeGrid, UniformIIDProcess, ... — unchanged.
from lcm.params import MappingLeaf, as_leaf — unchanged.
from lcm.typing import FloatND, ScalarInt, Period, Age, ... — unchanged for the model-authoring aliases.
from lcm.exceptions import InvalidParamsError, ... — unchanged.
from lcm._grids... import ... → from _lcm.grids... import .... The grid / process ABCs (Grid, ContinuousGrid, _ContinuousStochasticProcess) are private; import the public leaf classes from lcm.
Engine internals — lcm.engine, lcm.model_processing, lcm.regime_building, lcm.solution, lcm.simulation, lcm.variables, lcm.pandas_utils, lcm.state_action_space, lcm.dtypes, lcm.utils — are now under _lcm.*.
Explanation notebooks that demonstrate _lcm internals must import lcm before importing any _lcm submodule, so the package bootstrap completes first.

Test plan

pixi run -e tests-cpu tests -n 4 — 1004 passed, 10 skipped
pixi run ty — all checks pass
prek run --all-files — all hooks pass
pixi run explanation-notebooks — all notebooks execute
pixi lock — re-locked
Verify Auto-validate state transitions; make log_level the runtime-validation policy #360 is merged before merging this PR

Move user-facing type aliases (UserAge, UserParams, UserInitialConditions, UserFunction, UserFacingParamsTemplate, plus the private _UserParamsLeaf) from lcm/typing.py to a new lcm/api/typing.py. lcm/typing.py keeps the canonical / engine-side aliases and adds a bottom-of-file shim re-exporting the User* names so existing `from lcm.typing import UserParams` keeps working. Adjust the TYPE_CHECKING-only imports in `lcm.params.mapping_leaf` and `lcm.params.sequence_leaf` to source `_UserParamsLeaf` from its new home. First step of `Phase 2 — api Reorganisation.md`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…ariableInfo Rename `lcm/interfaces.py` to `lcm/engine.py` (canonical / engine-side dataclasses consumed by the DP machinery). Hoist `Variables` and `VariableInfo` dataclasses from `lcm/variables.py` into `engine.py`. `variables.py` retains the factories (`from_regime`, `get_grids`, `_raw_variable_info`, `_ordered_state_action_names`, `_bind_forward_refs`), which now import `Variables` / `VariableInfo` from `engine.py`. `Variables.from_regime` classmethod becomes the module-level `lcm.variables.from_regime` factory; src and test call sites are updated. All `from lcm.interfaces` imports are rewritten to `from lcm.engine`. Second step of `Phase 2 — api Reorganisation.md`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…mulateFunctionPair Move `lcm/user_regime.py` → `lcm/api/regime.py`. The user-facing `Regime` class (still defined as `class Regime`), `MarkovTransition`, `_default_H`, `_IdentityTransition`, the Phase 1 absorbed validators, and the slimmed `validate_transition_probs` ride along. Relocate `SolveSimulateFunctionPair` from `lcm/engine.py` to `lcm/api/regime.py` — it's user-facing (listed in `__all__`), constructed by users for `Regime.functions` values. Its natural home is alongside `Regime`. Update src and test importers. Sed-rewrite `from lcm.user_regime` → `from lcm.api.regime` across src/ and tests/. Update stale docstring references. Third step of `Phase 2 — api Reorganisation.md`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Move `lcm/model.py` → `lcm/api/model.py`. Heavy build machinery (`build_regimes_and_template`, `_validate_param_types`, `_resolve_fixed_params`, etc.) stays in `lcm/model_processing.py`. The `Model` class + tight privates (`_merge_derived_categoricals`, `_validate_log_args`) ride along. Rewrite `lcm.model.Model` annotations (forward refs in `api/regime.py`) to `lcm.api.model.Model`, plus the `TYPE_CHECKING` import. Sed-update `from lcm.model import` / `import lcm.model` across src and tests. Fourth step of `Phase 2 — api Reorganisation.md`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

- `lcm/ages.py` → `lcm/api/ages.py` (AgeGrid + small validators). - `lcm/persistence.py` → `lcm/api/persistence.py` (SolveSnapshot, SimulateSnapshot, load_*, save_* + I/O helpers). - `lcm/simulation/result.py` → `lcm/api/result.py` (SimulationResult + _compute_metadata). Sed-update `from lcm.ages` / `from lcm.persistence` / `from lcm.simulation.result` imports across src and tests. `tests/test_persistence.py` had a `from lcm import persistence as _persistence` form requiring a hand edit to `from lcm.api`. Fifth step of `Phase 2 — api Reorganisation.md`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Create thin re-export modules `lcm/api/grids.py` and `lcm/api/categorical.py` covering the user-facing leaf grid classes (`LinSpacedGrid`, `LogSpacedGrid`, `IrregSpacedGrid`, `DiscreteGrid`, `PiecewiseLinSpacedGrid`, `PiecewiseLogSpacedGrid`, `Piece`) and the `@categorical` decorator. The internal `lcm.grids` package is unchanged for this PR — the deeper restructure (`lcm/grids/` → `lcm/_grids/`, ABCs / validators / coordinates split into `_base.py` / `_validators.py` / `_coordinates.py`) is deferred to a follow-up; this PR is already large. Sixth step of `Phase 2 — api Reorganisation.md` in slimmed form. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…ss names Create `lcm/api/processes.py` exposing the user-facing leaf process classes under their canonical `<Distribution><Kind>Process` names. The names are import-time aliases of the internal `lcm.shocks.{iid,ar1}` classes (`Uniform`, `Tauchen`, ...). The full rename (`lcm/shocks/` → `lcm/_processes/`, `_ShockGrid` → `_ProcessGrid`, `is_shock` → `is_process`, `shock_names` → `process_names`, plus renaming the underlying classes) is deferred to a follow-up — this PR is already large and downstream user code (lcm_examples, tests) would need a coordinated update. Seventh step of `Phase 2 — api Reorganisation.md` in slimmed form. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…ss names Wire `lcm/__init__.py` to import every public symbol via `lcm.api.*`: grid classes via `lcm.api.grids`, `@categorical` via `lcm.api.categorical`, and add the seven new `*Process` aliases from `lcm.api.processes` (UniformIIDProcess, NormalIIDProcess, LogNormalIIDProcess, NormalMixtureIIDProcess, TauchenAR1Process, RouwenhorstAR1Process, TauchenNormalMixtureAR1Process). Extend `__all__` with the new names. The old shock aliases (`Uniform`, `Tauchen`, ...) remain reachable via `lcm.shocks` during the deprecation grace period; the next phase removes them. Smoke test (per the plan): python -c "from lcm import AgeGrid, DiscreteGrid, ..., UniformIIDProcess, TauchenAR1Process, ...; print('Public API intact')" passes. Ninth step of `Phase 2 — api Reorganisation.md`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

read-the-docs-community · 2026-05-18T19:40:58Z

Documentation build overview

📚 pylcm | 🛠️ Build #32805556 | 📁 Comparing 1e702fd against latest (629ac44)

🔍 Preview build

34 files changed · + 2 added · ± 31 modified · - 1 deleted

+ Added

± Modified

- Deleted

shocks/index.html

github-actions · 2026-05-18T20:38:07Z

Benchmark comparison (main → HEAD)

Comparing 629ac442 (main) → 1e702fd8 (HEAD)

Benchmark	Statistic	before	after	Ratio	Alert
aca-baseline	execution time	26.281 s	14.886 s	0.57
	peak GPU mem	639 MB	688 MB	1.08
	compilation time	297.53 s	278.77 s	0.94
	peak CPU mem	7.40 GB	11.54 GB	1.56	❌
Mahler-Yum	execution time	4.679 s	4.369 s	0.93
	peak GPU mem	529 MB	529 MB	1.00
	compilation time	14.15 s	12.77 s	0.90
	peak CPU mem	1.71 GB	1.69 GB	0.99
Precautionary Savings - Solve	execution time	46.3 ms	25.1 ms	0.54
	peak GPU mem	101 MB	101 MB	1.00
	compilation time	2.66 s	2.12 s	0.79
	peak CPU mem	1.14 GB	1.12 GB	0.98
Precautionary Savings - Simulate	execution time	120.2 ms	95.9 ms	0.80
	peak GPU mem	349 MB	349 MB	1.00
	compilation time	5.01 s	4.85 s	0.97
	peak CPU mem	1.34 GB	1.31 GB	0.98
Precautionary Savings - Solve & Simulate	execution time	159.7 ms	124.2 ms	0.78
	peak GPU mem	586 MB	586 MB	1.00
	compilation time	7.22 s	6.29 s	0.87
	peak CPU mem	1.30 GB	1.28 GB	0.98
Precautionary Savings - Solve & Simulate (irreg)	execution time	288.0 ms	262.8 ms	0.91
	peak GPU mem	2.20 GB	2.20 GB	1.00
	compilation time	7.53 s	6.71 s	0.89
	peak CPU mem	1.36 GB	1.33 GB	0.98

`lcm/grids/` is internal grid infrastructure (ABCs, validators, coordinate helpers, the leaf classes whose user-facing copies live in `lcm/api/grids.py`). The leading underscore signals "private — don't import from user code"; users keep reaching for grid classes through `from lcm import LinSpacedGrid` or `from lcm.api.grids import LinSpacedGrid`. Pure rename via `git mv` to preserve blame. Sed-rewrite of `from lcm.grids` / `import lcm.grids` / `lcm.grids.` across src and tests. Update docstring references in `api/grids.py` and `api/categorical.py`. Step A of `Phase 2 — api Reorganisation.md`'s deferred internal restructure. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…_ProcessGrid* `lcm/shocks/` is internal process infrastructure (the `_ShockGrid` ABC hierarchy plus the seven leaf distribution / discretization classes). The leading underscore matches the `lcm/_grids/` rename from step A — these are private packages users shouldn't reach into directly. Renames: - `lcm/shocks/` → `lcm/_processes/` - `_ShockGrid` → `_ProcessGrid` - `_ShockGridIID` → `_ProcessGridIID` - `_ShockGridAR1` → `_ProcessGridAR1` Sed-rewrite of `from lcm.shocks` / `import lcm.shocks` / `lcm.shocks.` and the three internal class names across src, tests, and lcm_examples. `lcm/__init__.py` drops `from lcm import shocks` and the `"shocks"` entry from `__all__` — the public surface is now exclusively `from lcm import UniformIIDProcess` (etc.) via `api/processes.py`. `lcm_examples/precautionary_savings.py` and `lcm_examples/mahler_yum_2024/_model.py` had bare-attribute access to `lcm.shocks.{iid,ar1}.{Normal,Uniform,Rouwenhorst,Tauchen}`; those rewrite to top-level imports (`NormalIIDProcess`, `UniformIIDProcess`, `RouwenhorstAR1Process`, `TauchenAR1Process`) because `lcm._processes` is private and ruff (rightly) flags the underscore access. The internal class names themselves (`Uniform`, `Tauchen`, ...) are renamed in step D; today the `*Process` names are aliases declared in `api/processes.py`. Step B of `Phase 2 — api Reorganisation.md`'s deferred internal restructure. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…ShockName → ProcessName Rename the derived attribute and type-alias names so the codebase speaks one language: `process` for any stochastic-process state. - `VariableInfo.is_shock` → `VariableInfo.is_process` - `Variables.shock_names` → `Variables.process_names` - `typing.ShockName` → `typing.ProcessName` - `non_shock_names` local var in `api/regime.py` → `non_process_names` - `test_shock_names_filters_is_shock` → `test_process_names_filters_is_process` Sed-rewrite across src and tests. Step C of `Phase 2 — api Reorganisation.md`'s deferred internal restructure. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…cess, ...) Rename the seven user-facing process leaf classes in their definition files to the canonical `<Distribution><Kind>Process` names. `api/processes.py` becomes a plain re-export without the `as ...` aliases. Tests that previously reached for the classes via `lcm._processes.iid.X` qualified access now import them from the top-level `lcm` namespace. Last step of `Phase 2 — api Reorganisation.md`'s deferred internal restructure. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

for more information, see https://pre-commit.ci

review-notebook-app · 2026-05-19T10:05:00Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

…ayout Map the post-Phase-2 source tree (api/, _grids/, _processes/, engine.py, model_processing.py, regime_building/, solution/, simulation/, params/, utils/) and explain the organising principle: proximity to user input versus proximity to JAX-traced DP machinery. Covers: - The api/ boundary and why physical separation, not just naming - Why _grids/ and _processes/ are leading-underscore packages - The engine.py (canonical / engine-side) vs api/ (boundary) split - The two-step build pipeline (model_processing.py → regime_building.processing.process_regimes) - Static (process-time) vs runtime (solve / simulate) checks - Boundary form vs canonical form in params/ - The User* typing aliases in api/typing.py vs engine-side aliases in typing.py - A suggested reading order for new contributors. Wired into the Explanations index and the myst.yml TOC. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Move validators, default Bellman aggregator, and validate_transition_probs helpers behind a leading underscore so lcm.api.regime is a thin layer of class definitions plus the deprecated public validate_transition_probs. New: lcm/_regime/{_helpers,_validation,_transition_probs}.py Moved: _IdentityTransition → regime_building/transitions.py (colocated with _make_identity_fn). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Move STEP_UNITS, PSEUDO_STATE_NAMES, parse_step (→ _parse_step), and the range/values/grid validators behind a leading underscore. api/ages.py is now just the user-facing AgeGrid class. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…nternals Move I/O helpers (atomic_dump → _atomic_dump, _save_pkl, _save_h5, _load_h5, _get_platform, _next_counter, _enforce_retention, _strip_V_arr_from_result, …) and the snapshot writers (save_solve_snapshot → _save_solve_snapshot, save_simulate_snapshot → _save_simulate_snapshot) to lcm/_persistence/. api/persistence.py now exposes only: - SolveSnapshot / SimulateSnapshot dataclasses - load_snapshot / save_solution / load_solution - _bind_forward_refs (delegator) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Extract DataFrame assembly, metadata computation, and additional-targets computation behind a leading underscore. api/result.py is now just the user-facing SimulationResult class. - simulation/_result_metadata.py — ResultMetadata (renamed from SimulationMetadata), _compute_metadata, _get_output_dtypes (renamed from get_simulation_output_dtypes; private). - simulation/_result_dataframe.py — flat-DataFrame assembly and categorical conversion helpers. - simulation/_additional_targets.py — _resolve_targets, _compute_targets, and supporting DAG helpers for to_dataframe(additional_targets=...). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Update the Internal Architecture page to reflect the new private siblings of api/: _ages.py, _regime/, _persistence/, simulation/_result_*.py, and simulation/_additional_targets.py. Add a "Private siblings of api/" section that lists what each one contains and why the split exists. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…n_checks phase-2 renamed interfaces.py → engine.py and moved model.py, regime.py, result.py into api/. The phase-1b merge brings in the runtime-check split: - api/model.py imports validate_regime_transitions_all_periods and validate_state_transitions_all_periods from lcm._transition_checks. - solve_brute.py imports validate_V from lcm.solution.validate_V. - validate_V.py: its StateActionSpace import follows the phase-2 rename to lcm.engine. - user_regime.py (deleted on phase-2): its phase-1b docstring tweak is dropped; the same deprecation note already lives on api/regime.validate_transition_probs, now pointing at lcm/_transition_checks.py. - docs/explanations/architecture.md: drop runtime_checks.py from the regime_building/ tree, add _transition_checks.py and solution/validate_V.py, and rewrite the static/runtime split prose. - Test files renamed and imports updated; no behavioural changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

aca-model 8338fbe adopts the api/ reorganisation: drops the `lcm.shocks.*` namespace, renames the process leaf classes (`Rouwenhorst` → `RouwenhorstAR1Process`, `Normal` → `NormalIIDProcess`), and reaches into `lcm._grids.continuous` for the abstract `ContinuousGrid` type (no public alternative). Without the bump the benchmarks env still resolves to the previous aca-model and fails to import.

Add a "Where transitions live — and why" subsection to the user guide transitions page: the functional vs parametric split that decides whether a transition goes in `state_transitions` or is bundled into a continuous stochastic process placed in `states`, and why there is no `DiscreteProcess`. Rename the stale "Shock Grids" subsection to "Continuous Stochastic Processes" and update it to the current `*Process` class names. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

PR 361 replaced the `lcm.shocks.*` modules with `*Process` classes exported from `lcm`. Rewrite the user guide shocks page and the AGENTS.md grid-system notes to the current API: `NormalIIDProcess`, `TauchenAR1Process`, and the other `*IIDProcess` / `*AR1Process` classes, imported directly from `lcm` and placed in `states`. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Rename the private stochastic-process base classes so the names state what they are: _ProcessGrid → _ContinuousStochasticProcess, _ProcessGridAR1 → _AR1Process, _ProcessGridIID → _IIDProcess. Rename the user guide page shocks.md → continuous_stochastic_processes.md, retitle it, and repoint every reference. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Renames the prose/docstring vocabulary to match the class names. Also drops a redundant "(importable from `lcm`)" parenthetical. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Replace the inconsistent leading-underscore-module convention with a hard binary: `src/lcm/` holds the public surface (the user-facing classes, the `@categorical` decorator, `as_leaf`, the public type aliases, and the exception classes), and `src/_lcm/` holds the private implementation. The package underscore carries the entire "private" signal — modules inside `_lcm/` are plainly named. - `lcm/api/` ceases; its modules move up to `lcm/`. - Every engine internal moves into `_lcm/`, dropping per-module underscores. - `typing.py` splits: model-authoring + `User*` aliases in `lcm/typing.py`, engine-side aliases and protocols in `_lcm/typing.py`. - `exceptions.py`: the `PyLCMError` subclasses stay public in `lcm/exceptions.py`; `format_messages` moves to `_lcm/utils/error_messages.py`. - `lcm/params.py` exposes `as_leaf` and re-exports the leaf classes; the definitions and engine params machinery live in `_lcm/params/`. - `_lcm/__init__.py` applies the jaxtyping patch; `lcm/__init__.py` registers the beartype claw on both packages. - hatch-vcs version file moves to `src/_lcm/version.py`. Pure relocation plus import rewrite — no behavior changes. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

for more information, see https://pre-commit.ci

aca-model `fbf1722` adapts its imports to pylcm's public `lcm/` + private `_lcm/` package split. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

hatch-vcs writes the generated version file to `src/_lcm/version.py` after the package restructure. The benchmark workflows regenerated it at the stale `src/lcm/_version.py` path, which `.gitignore` no longer covers — the untracked file left the worktree dirty and the dirty-tree guard aborted the benchmark run. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

The package-wide claw covers both `lcm` and `_lcm`; the `INTERNAL_CONF` comment named only `lcm`. The architecture doc called `beartype_conf.py` home to "two" configurations when it defines six. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

`_wrap_regime_transition_probs` builds a wrapper that converts a `next_regime` function's probability array into a regime-name → prob `MappingProxyType`. It copied the *return* annotation off the wrapped `next_regime` function (a bare probability array, e.g. `FloatND`), so the wrapper advertised an array return while actually returning a mapping. A beartype check on the wrapper rejects its genuine return value with `BeartypeCallHintReturnViolation`. Set the return annotation to `MappingProxyType[RegimeName, FloatND]`, matching what the wrapper returns, and drop the inherited annotation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

The internal-architecture page is the most advanced of the explanations; placing it last lets the conceptual pages build up to it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

The `_bind_forward_refs` explanation sat above the unrelated `__version__` import; relocate it directly above the `_bind_*_forward_refs` calls. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

`invert_regime_ids` is an engine-internal container helper used only by `_lcm.simulation`; `architecture.md` already documents it as internal. Remove it from `lcm`'s exports so the public surface is constructors and consumers only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

`Piece` was too generic for the flat `lcm` namespace. `PiecewiseGridSegment` disambiguates it ("a segment of a piecewise grid") and groups with the `Piecewise*Grid` classes that consume it. Cascading rename, no deprecation alias (pre-1.0): - class `Piece` → `PiecewiseGridSegment` - `PiecewiseLin/LogSpacedGrid` field `pieces` → `segments` - validator/helper params and cached `_segment_*` arrays follow suit - "segment" replaces "piece" uniformly in prose, error messages, tests, and docs; the wrong positional constructor examples are fixed to the keyword-only form Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

`_lcm` submodules import shared aliases, exceptions, and boundary classes from `lcm.*`; reaching any `lcm.*` module boots `lcm/__init__.py`. When an `_lcm` submodule is the entry point (as the benchmark suite does via `lcm_examples`), that boot re-entered the half-initialized engine module and raised a circular `ImportError`. `_lcm/__init__.py` now applies the jaxtyping patch and then imports `lcm`, so any `_lcm`-first entry point runs the full canonical bootstrap before its own body. The patch-then-`import lcm` order is load-bearing — pinned with `# isort: off` so ruff cannot reorder it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

for more information, see https://pre-commit.ci

Without `[tool.ruff.lint.isort] known-first-party`, ruff auto-detects which packages are first-party from the environment. That detection differs between a local checkout and pre-commit.ci's isolated env, so ruff grouped the `_lcm.*` / `lcm.*` imports in `lcm/__init__.py` differently in each — producing the back-and-forth `[pre-commit.ci] auto fixes` reformatting. Pinning the project's own packages makes the grouping identical everywhere; `prek` and pre-commit.ci now agree. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

hmgaudecker · 2026-05-21T16:03:58Z

 from scipy.interpolate import interp1d

-import lcm
+from _lcm.utils.dispatchers import productmap


Let's not worry about this, refactor is on its way.

…st names.

The _lcm/regime/ package held only validation.py after _default_H moved to lcm.regime. Collapse it to a single flat module. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

"Process transitions" read as verb+noun; reword to "Transitions of continuous stochastic processes". Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

The constant is only used by initial-conditions construction and the pandas bridge. It has nothing to do with the AgeGrid step machinery in _lcm/ages.py, where `age`/`period` go through different internals. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Introduce `AgeStep` in `lcm.typing` and annotate every `step` slot with it. Rewrite the `AgeGrid` docstring to cover both construction modes and the `(\d+)?[YQM]` step grammar. Reorder `STEP_UNITS` to Y, Q, M (descending duration). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

`lcm/categorical.py` was a one-line re-export module. Fold `categorical` into `lcm.grids` next to `DiscreteGrid` (the grid it builds codes for). `validate_category_class` leaves the public surface — it stays in `_lcm.grids.categorical` as an internal validator. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

The seven domain string aliases (RegimeName, StateName, ActionName, StateOrActionName, ProcessName, FunctionName, TransitionFunctionName) are user-facing vocabulary; define them in lcm.typing and re-export from _lcm.typing so engine-internal imports are unchanged. Restore the self-documenting key types on UserInitialConditions and UserFacingParamsTemplate. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Thin lcm/model.py: `_merge_derived_categoricals` moves to `_lcm.model_processing.merge_derived_categoricals`, `_contains_nan` to `_lcm.solution.validate_V.contains_nan`, and the two transition-probability checks are grouped behind `_lcm.transition_checks.validate_transitions`, which solve and simulate call once each. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

aca-model 2e5e3a6 adapts to `Piece` → `PiecewiseGridSegment` and the `PiecewiseLinSpacedGrid` `segments` argument rename. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

A with_signature wrapper around a function with explicit named parameters (not a bare *args/**kwargs forwarder) must stay callable under the beartype package claw. The wrapper advertises a permissive forwarder via its __annotations__, so the claw enforces nothing against the synthetic, annotation-free __signature__. The test builds get_argmax_and_max_Q_over_a and calls it to confirm no parameter is checked against the inspect.Parameter.empty sentinel. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

`compile_all_simulate_functions` lowered each simulate function with single-device `jnp.zeros` placeholders, so the AOT-compiled programs required `SingleDeviceSharding` inputs. When a regime distributes a grid, `build_initial_states` scatters the per-subject state arrays and `solve` scatters the value-function arrays across the device mesh, so calling the compiled programs raised a sharding mismatch. Build the lower-args with the same shardings runtime dispatches: per-subject arrays via the shared `subject_array_sharding` helper (extracted from `build_initial_states`), and `next_regime_to_V_arr` via `solve`'s V-topology helpers. Adds a multi-CPU regression test covering the AOT path. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

hmgaudecker and others added 8 commits May 18, 2026 21:19

hmgaudecker and others added 4 commits May 19, 2026 08:30

hmgaudecker mentioned this pull request May 19, 2026

Phase 2b: rename grids/shocks packages and process leaf classes #362

Closed

5 tasks

hmgaudecker changed the title ~~Phase 2: carve out lcm/api/ for the user-facing surface~~ Phase 2: api/ surface + internal restructure (_grids, _processes, *Process classes) May 19, 2026

[pre-commit.ci] auto fixes from pre-commit.com hooks

9caa879

for more information, see https://pre-commit.ci

hmgaudecker and others added 6 commits May 19, 2026 12:08

hmgaudecker changed the title ~~Phase 2: api/ surface + internal restructure (_grids, _processes, *Process classes)~~ api/ surface + internal restructure (_grids, _processes, *Process classes) May 19, 2026

hmgaudecker and others added 2 commits May 19, 2026 14:52

hmgaudecker mentioned this pull request May 20, 2026

Validation cleanup — delete error_handling.py and validation.py #359

Open

hmgaudecker and others added 2 commits May 20, 2026 09:35

hmgaudecker and others added 4 commits May 20, 2026 20:06

Drop "process grid" terminology for "continuous stochastic process"

3153deb

Renames the prose/docstring vocabulary to match the class names. Also drops a redundant "(importable from `lcm`)" parenthetical. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

a7912d5

for more information, see https://pre-commit.ci

hmgaudecker changed the title ~~api/ surface + internal restructure (_grids, _processes, *Process classes)~~ Split pylcm into a public lcm/ package and a private _lcm/ package May 20, 2026

hmgaudecker and others added 11 commits May 21, 2026 06:33

Bump benchmarks aca-model rev for the lcm/_lcm restructure

d099c92

aca-model `fbf1722` adapts its imports to pylcm's public `lcm/` + private `_lcm/` package split. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Move architecture explanation to the end of the explanations section

14aeb28

The internal-architecture page is the most advanced of the explanations; placing it last lets the conceptual pages build up to it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Move forward-ref comment to the calls it describes

1a11ee1

The `_bind_forward_refs` explanation sat above the unrelated `__version__` import; relocate it directly above the `_bind_*_forward_refs` calls. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

4f208d1

for more information, see https://pre-commit.ci

hmgaudecker commented May 21, 2026

View reviewed changes

hmgaudecker and others added 9 commits May 21, 2026 18:38

Move _default_H into lcm.regime and rename "shock" to "process" in te…

90f426f

…st names.

Flatten _lcm/regime into _lcm/user_regime_validation.py

513d604

The _lcm/regime/ package held only validation.py after _default_H moved to lcm.regime. Collapse it to a single flat module. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Clarify process-transition comment in regime processing

c32a744

"Process transitions" read as verb+noun; reword to "Transitions of continuous stochastic processes". Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Bump benchmarks aca-model rev for the Piece grid rename

4489748

aca-model 2e5e3a6 adapts to `Piece` → `PiecewiseGridSegment` and the `PiecewiseLinSpacedGrid` `segments` argument rename. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

hmgaudecker requested review from mj023 and timmens May 21, 2026 19:00

hmgaudecker and others added 2 commits May 22, 2026 07:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Split pylcm into a public `lcm/` package and a private `_lcm/` package#361

Split pylcm into a public `lcm/` package and a private `_lcm/` package#361
hmgaudecker wants to merge 60 commits into
feat/phase-1b-auto-state-transition-validationfrom
refactor/phase-2-api-reorganisation

hmgaudecker commented May 18, 2026 •

edited

Loading

Uh oh!

read-the-docs-community Bot commented May 18, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 18, 2026 •

edited

Loading

Uh oh!

review-notebook-app Bot commented May 19, 2026

Uh oh!

hmgaudecker May 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

hmgaudecker commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Public surface — src/lcm/

Private implementation — src/_lcm/

Typing split

Exceptions

Params

Bootstrap

Renames

Build

Migration guide for downstream code

Test plan

Uh oh!

read-the-docs-community Bot commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation build overview

Uh oh!

github-actions Bot commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmark comparison (main → HEAD)

Uh oh!

review-notebook-app Bot commented May 19, 2026

Uh oh!

hmgaudecker May 21, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

hmgaudecker commented May 18, 2026 •

edited

Loading

Public surface — `src/lcm/`

Private implementation — `src/_lcm/`

read-the-docs-community Bot commented May 18, 2026 •

edited

Loading

github-actions Bot commented May 18, 2026 •

edited

Loading