fix: bump transformers floor to 5.15.0 for DeepSeek-V4-Pro config parsing

[debermudez]

Summary

• Bumps the transformers dependency floor from >=4.56.0 to >=5.15.0 — the current floor (resolving to 5.6.2) raises AttributeError: 'PreTrainedConfig' object has no attribute 'max_position_embeddings' while parsing DeepSeek-V4-Pro's config during RoPE param standardization.
• Removes the deepseek_v32 config-alias shim from tokenizer.py (and its test file): transformers 5.15.0 registers deepseek_v32 natively, so the shim is now a permanent no-op, matching the removal condition documented in its own docstring.
• Updates docs/reference/tokenizer-auto-detection.md to drop the now-stale shim documentation.

Test plan

• Ran the exact repro from AIP-1140 against the in-repo mock server (aiperf profile --tokenizer deepseek-ai/DeepSeek-V4-Pro ...) — completes cleanly with transformers==5.15.0, no error.
• uv run pytest tests/unit/ -n auto — 17999 passed, 93 skipped, 3 xfailed.
• uv run pytest tests/unit/property/ -n auto — 59 passed (NaN/Inf invariants).
• pre-commit run --all-files equivalent on changed files — clean.

Summary by CodeRabbit

• Documentation

  • Removed documentation for the legacy DeepSeek V3.2 configuration alias.

• Compatibility

  • Updated the minimum supported Transformers version to 5.15.0.
  • Tokenizer loading now relies on native Transformers support for DeepSeek configurations.

• Maintenance

  • Removed obsolete compatibility handling and related tests.

[github-actions]

Try out this PR

Quick install:

pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@0ee204746499ed034b0b89aac88ded5d6055da72

Recommended with virtual environment (using uv):

uv venv --python 3.12 && source .venv/bin/activate
uv pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@0ee204746499ed034b0b89aac88ded5d6055da72

---

Last updated for commit: 0ee2047 • Browse code

[github-actions]

Fern Docs Preview: https://nvidia-preview-01a02115-ea49-742c-be7a-933936548dc4.docs.buildwithfern.com/aiperf/dev

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5335dfb4-d5a8-426c-bfbf-487091ffb23f

📥 Commits

Reviewing files that changed from the base of the PR and between 324bb05 and 0ee2047.

📒 Files selected for processing (4)

• docs/reference/tokenizer-auto-detection.md
• pyproject.toml
• src/aiperf/common/tokenizer.py
• tests/unit/common/test_tokenizer_deepseek_v32.py

💤 Files with no reviewable changes (3)

• src/aiperf/common/tokenizer.py
• docs/reference/tokenizer-auto-detection.md
• tests/unit/common/test_tokenizer_deepseek_v32.py

Included review availability: Your plan provides up to 12 included reviews per hour; 10 remain after this review.

---

Walkthrough

The project now requires transformers 5.15.0 and relies on native DeepSeek configuration support. The tokenizer compatibility shim, its tests, and its documentation are removed.

Changes

DeepSeek-V3.2 shim removal

Layer / File(s)	Summary
Raise Transformers requirement pyproject.toml	The minimum transformers version changes from 4.56.0 to 5.15.0.
Remove compatibility registration src/aiperf/common/tokenizer.py, tests/unit/common/test_tokenizer_deepseek_v32.py, docs/reference/tokenizer-auto-detection.md	Tokenizer loading no longer registers the deepseek_v32 configuration alias. The related unit tests and documentation section are removed.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 0ee20

The dependency floor and obsolete tokenizer shim are updated together, with the reported reproduction and test suites passing; no actionable merge-blocking risk remains after normal checks and review.

Poem

I’m a rabbit with ears held high,
The old shim hops away nearby.
Native configs now lead the way,
With newer Transformers in play.
No alias burrows in today!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the primary change: raising the transformers dependency floor to fix DeepSeek-V4-Pro configuration parsing.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

---

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[ajcasagrande]

I'm worried about the compatibility issues this may lead to with customers, especially in light of this Issue: #1278

[debermudez]

I there a better strategy for this?
we cant stay pinned forever but i see your point.

[debermudez]

closing as we have a different approach in the works.