Repository navigation
feat: honor and surface rate-limit retry deadlines - #433
Open
LarsArtmann wants to merge 1 commit into
Open
LarsArtmann wants to merge 1 commit into
LarsArtmann wants to merge 1 commit into
Conversation
1 of 2 tasks
LarsArtmann
force-pushed
the
push-xlrnwtqnkpyk
branch
from
October 24, 2026 10:12
011b525 to
188c18c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a provider answers 429 with a
retry-afterbeyond 60 seconds, the retry middleware silently discards it:getRetryDelayInMstreats any header delay at or beyond a hardcoded 60s cap as unreasonable, falls back to exponential backoff, and the error a caller finally receives says "too many requests" with no hint of when a retry can actually succeed.Subscription quotas are the worst case. They renew on a billing schedule — hours or days out — so the middleware burns its whole retry budget against a locked door and then reports nothing actionable. Synthetic's quota-exhausted 429 (body:
{"error":"You've exceeded your subscription rate limits. Upgrade, or try again later. ..."}) is the concrete case that motivated this; nothing in the response names a retry moment.What changed
ProviderError.RetryAt: the earliest retry moment a provider declared, parsed fromretry-after(delay-seconds or HTTP-date) andretry-after-msby the newfantasy.RetryAtFromHeaders, case-insensitively. Wired into the openai (incl. openai-compat) and anthropic error constructors.Error()appends; retry at 2026-10-25T14:36:14Z (in 23h59m)when a future deadline is known, so even a giving-up caller learns when to come back.ProviderError.QuotaExhausted: marks a billing/subscription quota hit, set viafantasy.IsQuotaExhaustedMessage(matches "exceeded your subscription"; per-minute throttles say "rate limit" without it and keep their classification). A quota exhaustion with no known deadline reportsIsRetryable() == false: no automatic budget can outwait a billing cycle.RetryOptions.MaxRetryDelay(default 60s, the previous hardcoded cap): bounds server-declared waits. Within it, the middleware waits exactly untilRetryAtinstead of guessing. Beyond it, a plain rate limit keeps today's backoff behavior, while a quota exhaustion fails fast with the original error naming the renewal moment instead of burning retries that cannot succeed.Defaults are behavior-identical to today: no deadline header and no quota marker means the same backoff and the same error strings.
Evidence
go test ./...— pass (providers/kronkpanics locally on a missing libffi; environmental, it still compiles viago test -c)go vet ./...— passgolangci-lint run— passNew coverage: header parsing (delay-seconds, HTTP-date, ms precedence, case-insensitive names, invalid/zero/negative/past values); quota-body classification through a real round trip against the openai provider; all four retry-middleware deadline paths (no deadline, deadline beyond budget, deadline within budget, non-quota header hint beyond budget)
I have read
CONTRIBUTING.md.I have created a discussion that was approved by a maintainer (for new features).
Happy to move this to a Discussion first if you'd rather shape the API there — filing as a PR because it is test-covered end to end and motivated by a concrete failure mode.
💘 Generated with Crush