Skip to content

models/ directory convention for MODEL_PATH: fail-closed lookup, MLIPORT_ROOT, doctor, launchers (R2) - #105

Merged
hydrogen1222 merged 3 commits into
mainfrom
feat/models-directory
Nov 17, 2026
Merged

hydrogen1222 merged 3 commits into
mainfrom
feat/models-directory

Conversation

@hydrogen1222

@hydrogen1222 hydrogen1222 commented Nov 15, 2026 •

Copy link
Copy Markdown
Owner

Tracking: #61 — delivery 2 of the "任务文件夹 + INCAR + models/" split (spec: #61 (comment)). Independent of #104.

What this PR does (R2)

Requirement Implementation
R2.1 file name → models/ New mliport/model_lookup.py. A MODEL_PATH without a path separator is looked up in <MLIPORT_ROOT>/models/<name>, then <MLIPORT_ROOT>/models/*/<name> (exactly one level). Two matches raise ModelLookupError listing both; the user must choose, because a different checkpoint is a different model identity.
R2.2 path values unchanged A value containing a path separator is resolved exactly as before and is never redirected into models/. A same-named file next to the declaring file (INCAR folder / settings.ini dir / CWD) still wins over models/.
R2.3 root resolution MLIPORT_ROOT env var → else parent of sys.prefix when that is an installer-created .venv-<engine> (names from install/compatibility.py) → else unknown, and a bare file name is refused with instructions (never a silent fallback). A non-absolute MLIPORT_ROOT is refused. The generated bin/mliport-<engine> launchers export MLIPORT_ROOT (POSIX and Windows).
R2.4 missing-model error Lists every location searched (<declaring dir>/<name>, <root>/models/<name>, <root>/models/*/<name>) and the files models/ actually contains, including one level of subdirectories.
R2.5 provenance The resolver applies the lookup inside _resolve_declared_paths, so resolved_config.json and the existing model-identity record contain the resolved absolute path; no provenance format change.
R2.6 doctor Two new checks: MLIPORT_ROOT (path, or not determined → warn with instructions) and models/ (path + file listing, warn when absent). Present in both the text and --json reports.
R2.7 hygiene models/ is still git-ignored and the tracked-weight refusal in scripts/check_repository_hygiene.py is untouched; a test asserts git ls-files models is empty.

Tests

tests/mliport/mliport/test_models_directory.py (16 tests): direct and one-level-subdirectory lookup, two-match refusal, path values never redirected, local-file precedence, absolute paths, env and .venv-<engine> roots, unknown/relative root failures, searched-location error content, resolved_config.json absolute path, doctor text + JSON, launcher export (run through a stub venv), git-ignore.

Existing call sites that passed a never-existing bare model name now pass an explicit relative path (./m.model); this preserves the previous semantics exactly. Behaviour change: a bare name that is not next to the declaring file and not in models/ now fails at configuration resolution with the searched-location error instead of failing later inside the backend. That is the fail-closed requirement of R2.3/R2.4; no working configuration is broken.

Rebased onto the current main by merging it in (no rewrite of the two reviewed commits): only CHANGELOG.md and _resolve_declared_paths conflicted, because #104 added the INCAR OUTPUT_DIR rule to the same function. The resolution keeps both rules in one pass - the output_dir early-out first, model_path through resolve_model_reference, every other declared path unchanged.

Local verification on the merged head: full suite 2243 passed, 48 skipped; ruff check / format --check clean; scripts/check_repository_hygiene.py passes; scripts/generate_docs.py --check up to date; mkdocs build --strict and the docs contract tests pass. GitHub CI (3.10-3.13, lint, build, installer, wheel smoke) green on 86d76c24.

DATA IMPACT: no scientific record invalidated. Nothing regenerated. The resolved model path recorded for a run keeps its format; only the lookup that produces it is new.

Not merged without owner confirmation; review first.

…ry (#61 R2)

`MODEL_PATH = mace-omat-0-medium.model` had no meaning beyond "join this with
the declaring file's directory": there was no single place for weights, the
`models/` directory in the checkout was invisible to the tool, and a missing
model surfaced much later as a framework import or checkpoint error.

What this changes:

- New `mliport/model_lookup.py` implements the convention. A `MODEL_PATH`
  without a path separator is a file name: it is looked up next to the
  declaring file (the historical behaviour, unchanged), then in
  `<MLIPORT_ROOT>/models/<name>`, then in `<MLIPORT_ROOT>/models/*/<name>` -
  exactly one level of subdirectories. Two matches are an error, because a
  different checkpoint is a different model identity. A value that contains a
  path separator is a path and nothing else: it keeps its previous resolution
  and is never redirected into `models/`.
- `MLIPORT_ROOT` comes from the environment, else from the parent of an
  installer-created `.venv-<engine>` prefix (names taken from the
  compatibility registry). With no root, a bare file name fails closed with
  instructions - it is never resolved against a guess. A missing model names
  every location that was searched and lists the files `models/` does contain.
- The resolver applies this in `_resolve_declared_paths`, so the resolved
  absolute path is what `resolved_config.json` and the model identity record
  already contain; no new provenance format.
- The generated `bin/mliport-<engine>` launchers now export `MLIPORT_ROOT`
  (POSIX and Windows), which is what makes the convention work from a bare
  terminal command.
- `mliport doctor` reports the resolved root, the `models/` directory and the
  model files it holds, and warns with instructions when the root is unknown.

Existing test call sites that passed a never-existing bare model name now pass
an explicit relative path, which keeps exactly the previous semantics; the
behaviour change only affects a file name that really was meant to be found in
`models/`.

Tests: `tests/mliport/mliport/test_models_directory.py` (16 tests) covers the
lookup order, the one-level subdirectory search, the two-match refusal, path
values being untouched, the local-file precedence, both root derivations, the
unknown-root and relative-root failures, the searched-location error, the
recorded absolute path, the doctor report and the launcher export. Full suite:
2195 passed, 48 skipped; ruff and the repository hygiene check are clean.

Docs: `docs/configuration.md` (+ `.zh.md`) gains a "Model files: models/"
section and the `MLIPORT_ROOT` row in the environment table; the CHANGELOG
records the new lookup and the stricter missing-model error.

DATA IMPACT: no scientific record invalidated. No default, unit, PBC semantic,
model task/head or dtype changed, and no existing result or trajectory is
rewritten. The only behaviour change is the lookup of a name-only `MODEL_PATH`
and the earlier, explicit error when such a name cannot be found; the resolved
path recorded for a run is unchanged in format.

Tracking: #61
…cher scope

Review findings on the R2 delivery:

- GRACE's MODEL_PATH is a SavedModel *directory*. The lookup only accepted
  files, so the documented `--model saved_model` GRACE command started failing
  once the value became a name: "Model file 'saved_model' was not found".
  `resolve_model_reference` now accepts any existing path (file or directory)
  before consulting models/, `lookup_model_file_name` accepts directory
  matches, and the listings show a top-level directory that is itself a model.
- `MLIPORT_ROOT` was derived from the parent of any `.venv-<engine>` prefix, and
  `.venv` is UMA's environment name but also the name of any project-local
  virtualenv: running from `/home/<user>/.venv` would have made `/home/<user>`
  the root and silently resolved models out of an unrelated directory. The
  derivation now also requires a checkout marker (`models/`, a `bin/mliport-*`
  launcher or `mliport/pyproject.toml`), otherwise the root stays unknown and a
  bare name fails closed with instructions.
- The generated launchers overwrote an explicitly set `MLIPORT_ROOT`, which
  defeated the documented precedence and broke a shared model store. They now
  set it only when unset (POSIX `: "${MLIPORT_ROOT:=...}"`, Windows
  `if not defined`).
- `MODEL_PATH`'s one-line meaning now states the file/directory and the
  local-then-`models/` rule instead of "Path to the model checkpoint/file".

Tests: three new cases (model directory next to the INCAR, model directory found
in models/, venv prefix without a checkout marker) and a launcher case that runs
with `MLIPORT_ROOT` already set. Suite: the models/config/doctor/install tests
are green; ruff clean.

DATA IMPACT: no scientific record invalidated. This restores dropped behaviour
(GRACE directory models) and removes a silent-root failure mode; no default,
unit, PBC semantic, model task/head or dtype changed.

Tracking: #61
@hydrogen1222

Copy link
Copy Markdown
Owner Author

Review round 1 — a GRACE regression and two silent-fallback holes fixed

An independent review of the delivery found four defects; all are fixed in 110df2fe:

  1. Blocker: every directory-valued MODEL_PATH broke. GRACE's model is a SavedModel directory, and the lookup only accepted files, so the documented mliport --model saved_model --model-type grace command started failing with "Model file 'saved_model' was not found". resolve_model_reference now accepts any existing path before consulting models/, lookup_model_file_name accepts directory matches, and the listings show a top-level directory that is itself a model.
  2. MLIPORT_ROOT could be derived from an unrelated .venv. .venv is UMA's environment name and the name of any project-local virtualenv, so running from ~/.venv made the home directory the root and could silently resolve a model from there. The derivation now also requires a checkout marker (models/, a bin/mliport-* launcher, or mliport/pyproject.toml); otherwise the root stays unknown and a bare name fails closed with instructions.
  3. The launcher overrode an explicit MLIPORT_ROOT, defeating the documented env-var-first precedence and breaking a shared model store. It now only sets the variable when unset (POSIX : "${MLIPORT_ROOT:=...}", Windows if not defined).
  4. MODEL_PATH's meaning now states the file-or-directory rule and the local-then-models/ lookup instead of "Path to the model checkpoint/file".

New tests: model directory next to the INCAR, model directory found in models/, venv prefix without a checkout marker, and a launcher run with MLIPORT_ROOT already set (the old launcher test stripped the variable, so it could not catch 3).

CI green after the change.

main (#104) landed its own change in `_resolve_declared_paths`: a relative
`OUTPUT_DIR` declared in the INCAR resolves against the task folder, and
`OUTPUT_DIR = .` opts out of the `results/` default before path resolution
runs. This branch added the `MODEL_PATH` `models/` lookup in the same
function, so the two conflicted.

Resolution keeps both rules in one pass: the `output_dir` early-out stays
first, `model_path` goes through `resolve_model_reference`, and every other
declared path keeps the historical base-directory resolution.

The CHANGELOG conflict keeps both Unreleased sections, newest first.
hydrogen1222 added a commit that referenced this pull request Nov 17, 2026
#105 defines what a `MODEL_PATH` without a path separator means: it is a file
name, resolved next to the declaring file first and then in
`<MLIPORT_ROOT>/models` (one level of subdirectories, two matches refused).
The unified option table still carried the pre-#105 wording, which the
generated key reference in `docs/incar.md` (+ `.zh.md`) repeated for every
calculation type.

The description now comes from #105, and the generated pages are regenerated
from it, so the table, `mliport config schema`, the templates and the
reference page keep agreeing.
hydrogen1222 added a commit that referenced this pull request Nov 17, 2026
The TUI wizard (this branch) needs both parents: the `models/` lookup for
`MODEL_PATH` (#105) and the generated INCAR templates (#106). Both are
rebased onto the same `main`, so this merge only resolves the CHANGELOG:
the Unreleased section keeps the newest round first, i.e. this branch, then
the option table, then `models/`, then the task folder round that is already
in `main`.
hydrogen1222 added a commit that referenced this pull request Nov 17, 2026
The wizard on this branch needs both stacks: the `models/` lookup for
`MODEL_PATH` (#105) and the generated INCAR templates (#106). Both are
rebased onto the same `main`, so the merge only has to settle the unchanged
files; the CHANGELOG conflict does not arise because each round keeps its
own anchor in the Unreleased section.
@hydrogen1222
hydrogen1222 merged commit 576907a into main Nov 17, 2026
14 checks passed
@hydrogen1222
hydrogen1222 deleted the feat/models-directory branch November 17, 2026 11:48
hydrogen1222 added a commit that referenced this pull request Nov 17, 2026
#105 and #106 are merged, but this branch carried their pre-merge commits, so
`main` and the branch had two merge bases and GitHub reported the pull request
as conflicting even though the trees agreed. Merging `main` in brings the
merge base down to one commit; the resulting tree is unchanged (the branch
already contained every file of #105 and #106 verbatim).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant