docs(routing): requests over a deployment's max_parallel_requests get an immediate 429, no queue

[devin-ai-integration]

Summary

Documents the behavior change in BerriAI/litellm#41555. A deployment's rpm or tpm derives a per-process concurrency cap (max_parallel_requests, then rpm, then int(tpm / 1000 * 6) with a minimum of 1, then the router's default_max_parallel_requests). Until now a request over that cap waited in the proxy for a slot and came back as a slow 200. After the code PR it comes back as an immediate 429 that names the deployment and its cap, raised before any provider call

The Max Parallel Requests section of the routing page now spells out that precedence, states that there is no wait queue and nothing to configure for one, explains that the 429 does not cool the deployment down and that router retries and fallbacks treat it like any other 429 (num_retries: 0 surfaces it to the caller right away), and calls out the change from the earlier queueing behavior together with what to do about it (raise the cap or the rpm/tpm it comes from, or handle the 429). A proxy YAML example with an rpm-derived cap and the resulting 429 body, taken from a live proxy on the code PR's tip, round it off

The default_max_parallel_requests_queue_size row an earlier revision of this PR added to the router_settings reference in proxy/config_settings.md is gone again, since the code PR no longer ships a queue or any setting for it. npm run build completes with Generated static files in "build"

Linear ticket

Resolves LIT-7675

Link to Devin session: https://app.devin.ai/sessions/fd9d476571c64999a915a8312aca8389
Open in Devin Desktop: https://app.devin.ai/desktop/session/fd9d476571c64999a915a8312aca8389?variant=devin
Requested by: @yassin-berriai

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
litellm	Ready	Preview	Sep 17, 2026 6:36pm UTC

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 0cf07a5. Configure here.}