Skip to content

fix: Do not autofill default_model_filename for Python-runtime PyTorch models - #518

Merged
mattwittwer merged 5 commits into
mainfrom
mwittwer/fix_python_runtime_autofill
Aug 13, 2026
Merged

fix: Do not autofill default_model_filename for Python-runtime PyTorch models#518
mattwittwer merged 5 commits into
mainfrom
mwittwer/fix_python_runtime_autofill

Conversation

@mattwittwer

@mattwittwer mattwittwer commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

What does the PR do?

Since #484, AutoCompleteBackendFields autofills default_model_filename: "model.pt" for every backend: "pytorch" model with an empty platform — including models using the Python-based runtime (runtime: "model.py"). For those models a sibling model.pt is the weights companion to the model.py class, and the runtime gives default_model_filename top priority, so it calls torch.jit.load() on the weights file and fails at load with PytorchStreamReader failed locating file constants.pkl.

Restores the pre-#484 guard — skip filename autofill when runtime is "model.py" — and extends it to the AOTI platform branch (model.pt2), which the Python runtime cannot load either. Output is unchanged for every other config. #484's stated goal (correct .pt2 selection for torch_aoti) is preserved and now pinned by tests.

Adds AutoCompleteBackendFieldsTest to repo_agent_test (which already compiles model_config_utils.cc): the two Python-runtime regression cases plus six cases pinning existing autofill and version-dir sniffing behavior. The regression cases fail without the fix and pass with it.

Checklist

  • PR title reflects the change and is of format <commit_type>: <Title>
  • Changes are described in the pull request.
  • Related issues are referenced.
  • Populated github labels field
  • Added test plan and verified test passes.
  • Verified that the PR passes existing CI.
  • Verified copyright is correct on all changed files.
  • Added succinct git squash message before merging ref.
  • All template sections are filled out.
  • Optional: Additional screenshots for behavior/output changes with before/after.

Commit Type:

Check the conventional commit type
box here and add the label to the github PR.

  • build
  • ci
  • docs
  • feat
  • fix
  • perf
  • refactor
  • revert
  • style
  • test

Related PRs:

triton-inference-server/server#8913

Where should the reviewer start?

Test plan:

Test case added here

  • CI Pipeline ID:

60326198

Caveats:

Background

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • closes GitHub issue: #xxx

@greptile-apps

greptile-apps Bot commented Jul 30, 2026

Copy link
Copy Markdown

Greptile Summary

The PR narrows PyTorch model filename autofill and adds regression coverage.

  • Skips default_model_filename autofill for Python-runtime PyTorch models on both LibTorch and AOTI platform paths.
  • Preserves existing platform selection, explicit filenames, and default-runtime autofill behavior.
  • Adds table-driven tests covering Python runtime regressions and existing version-directory inference behavior.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
src/model_config_utils.cc Adds a shared guard preventing filename autofill when the configured PyTorch runtime is model.py.
src/test/repo_agent_test.cc Adds table-driven coverage for Python-runtime exclusions and existing PyTorch autofill and inference cases.

Reviews (5): Last reviewed commit: "fix pre-commit" | Re-trigger Greptile

@mattwittwer mattwittwer changed the title draft: autofill model name fix fix: Do not autofill default_model_filename for Python-runtime PyTorch models Aug 3, 2026
@mattwittwer
mattwittwer requested review from Vinya567 and yinggeh August 3, 2026 23:05
Comment thread src/model_config_utils.cc Outdated
@mattwittwer
mattwittwer requested a review from yinggeh August 5, 2026 21:39
Comment thread src/test/repo_agent_test.cc Outdated
// and a weights "model.pt" in the version directory must not have "model.pt"
// autofilled as the default model filename, otherwise the runtime attempts
// torch.jit.load() on the weights file.
TEST_F(AutoCompleteBackendFieldsTest, PythonRuntimeLibTorchKeepsFilenameEmpty)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to merge test cases into a for loop?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, updated

@mattwittwer
mattwittwer requested a review from yinggeh August 11, 2026 22:10
@mattwittwer
mattwittwer merged commit bb85687 into main Aug 13, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

PR: fix A bug fix

Development

Successfully merging this pull request may close these issues.

2 participants