You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Proxy: inconsistent 401 handling of gateway credentials #33
In llm_proxy.rs::forward, a freshly fetched controller credential is dropped from the cache on any gateway 401, also when a client token was forwarded in x-llm-token. The retry branch keeps a cached credential in that case, because the 401 may be about the client token. With a bad client token on a pass-through route, every request in a row then fetches a new controller credential.
A request that waited for another request's fetch and got that credential from the cache counts as from_cache. On a 401 it takes the "stale cached credential" retry and fetches again, although the new credential cannot differ. At most one extra controller call per burst.
Expected
Drop the fresh credential only when no client token was sent, and treat a credential fetched during this request's wait as fresh for the retry decision.
Activity