Repository navigation
feat(agent): local LLM proxy for clients without an API-key helper - #134
Conversation
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
URL validation, authorization parsing, OAuth failure handling, credential-fetch concurrency, and the standalone example contain unresolved issues.
Review effort: Balanced
Findings: 1
Open (6)
What changed in this PR
Adds the foundational user-mode loopback LLM proxy for clients without credential-helper support, including authenticated forwarding, GitHub OAuth, diagnostics, and configuration.
Changes:
- Adds paired, route-aware proxy forwarding with credential caching and retry handling.
- Integrates GitHub device-flow credentials and proxy startup/status reporting.
- Adds schemas, desktop status UI, documentation, examples, and tests.
| File | Description |
|---|---|
schema/daemon-config.md |
Documents proxy configuration. |
schema/daemon-config.json |
Adds generated proxy/OAuth schema. |
README.md |
Documents setup, routing, and security. |
frontend/desktop/src/views/StatusView.stories.tsx |
Tests proxy status display. |
frontend/desktop/src/types.ts |
Types proxy daemon information. |
frontend/desktop/src/stories/fixtures.ts |
Adds proxy status fixtures. |
frontend/desktop/src/components/DaemonInformation.tsx |
Displays proxy state. |
examples/standalone/config.yaml |
Adds optional proxy/OAuth configuration. |
examples/standalone/agentgateway.yaml |
Changes standalone gateway routing. |
crates/core/src/model.rs |
Models proxy runtime status. |
crates/core/src/config.rs |
Adds and validates proxy configuration. |
crates/agent/src/remote.rs |
Bounds controller credential calls. |
crates/agent/src/reconcile/mod.rs |
Supplies proxy context to reconcilers. |
crates/agent/src/provider/mod.rs |
Extends reconciliation context. |
crates/agent/src/provider/claude_code/mod.rs |
Updates test context construction. |
crates/agent/src/oidc.rs |
Supports continued GitHub authorization. |
crates/agent/src/llm_proxy.rs |
Implements proxy routing and forwarding. |
crates/agent/src/lib.rs |
Registers new modules. |
crates/agent/src/identity.rs |
Loads device IDs for cache tagging. |
crates/agent/src/github_oauth.rs |
Implements GitHub device OAuth. |
crates/agent/src/gateway_oidc.rs |
Chains gateway and GitHub sign-in. |
crates/agent/src/daemon.rs |
Binds and runs the proxy. |
crates/agent/src/api.rs |
Shares gateway credential acquisition. |
crates/agent/Cargo.toml |
Adds proxy dependencies. |
Cargo.toml |
Configures TLS dependencies. |
Cargo.lock |
Locks dependency updates. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| audiences: [agentdesktop-local] | ||
| jwks: | ||
| url: http://dex:5557/dex/keys | ||
| url: http://localhost:5557/dex/keys |
There was a problem hiding this comment.
Fixed in 5f058fb: examples/standalone/ is back to main's version, so the JWKS URL is http://dex:5557/dex/keys again.
| let code: DeviceCode = client | ||
| .post(format!("{base}/login/device/code")) | ||
| .header("Accept", "application/json") | ||
| .form(&[("client_id", client_id)]) | ||
| .send() |
There was a problem hiding this comment.
Fixed in 5f058fb: if a continued sign-in fails before the GitHub page is served (device-code request, URL check, refresh), the page is now served in its failed state on the callback port, so the tab shows the retry message. Test: continued_flow_shows_failure_when_github_setup_fails; not re-tested on the live path.
| .map(|value| { | ||
| value | ||
| .strip_prefix("Bearer ") | ||
| .unwrap_or(value) | ||
| .trim() | ||
| .to_owned() | ||
| }) |
There was a problem hiding this comment.
Fixed in 5f058fb: the scheme is matched case-insensitively. Test: bearer_scheme_is_case_insensitive.
| let cached = cache_key | ||
| .as_ref() | ||
| .and_then(|(key, device_id)| cache.get(key, device_id)); |
There was a problem hiding this comment.
Fixed in 5f058fb: fetches are single-flight per cache key (different keys stay independent), and a fetched credential is cached as soon as it arrives and dropped if the gateway rejects it. Test: concurrent_misses_share_one_fetch_per_key (7 concurrent misses on 2 keys, 2 fetches). Lab: 8 concurrent requests on one route, one controller issuance.
| if let Some(proxy_url) = &gateway.proxy_url | ||
| && (proxy_url.query().is_some() || proxy_url.fragment().is_some()) | ||
| { | ||
| anyhow::bail!("LLM gateway proxyUrl cannot include a query or fragment"); | ||
| } |
There was a problem hiding this comment.
Fixed in 5f058fb: proxyUrl gets the same scheme, host, credentials, query and fragment checks as url. Test: rejects_an_invalid_llm_gateway_proxy_url. Lab: the controller rejects an ftp:// proxyUrl and keeps the last good configuration.
| # Claude desktop requires an explicit model | ||
| - name: "claude-haiku-4-5" | ||
| provider: anthropic | ||
| - name: "gpt-6-luna" | ||
| provider: copilot |
Part of agentdesktop-dev#86. Adds an optional loopback proxy (`daemon.llmProxy.listen`, user mode only) that forwards a client's model requests to the LLM gateway with the device's gateway credential, for clients that cannot run an API-key helper (GitHub Copilot CLI, VS Code Copilot Chat). - Bound before the first apply; a failed bind keeps the daemon running and is reported in daemon-info (`llmProxy.bound`, `error`), also shown in the desktop app's daemon information panel. - Fixed route prefix per client (`/copilot-cli`, `/vscode-copilot`, `/vscode-copilot-passthrough`), so the client ID the gateway sees comes from the route, not from the request. - A per-device pairing value (`x-agentdesktop-pairing`) is required on every route; loopback Host check; OpenAI-style JSON errors. - Request bodies are buffered (size cap) so a 401 is retried once with a fresh credential; controller credentials are cached briefly. - `llmGateway.proxyUrl` for a proxy target that differs from `url`; `llmGateway.githubOAuth` for the upstream GitHub credential, taken from the client's own request (`source: request`) or from a GitHub App device flow (`deviceFlow`). Co-authored-by: John Howard <john.howard@solo.io> Co-authored-by: James McShane <james.mcshane@solo.io> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- examples/standalone: restore the Claude example from main. The Copilot example comes with the guide later in the series; the JWKS URL stays on the Compose service name. - GitHub OAuth: when a continued sign-in fails before the GitHub page is served, serve it in its failed state, so the browser tab stops polling. - Proxy: parse the Bearer scheme case-insensitively. - Proxy: credential fetches are single-flight per cache key, and a fetched credential is cached as soon as it arrives (dropped if the gateway rejects it), so concurrent requests share one controller call. - Config: llmGateway.proxyUrl gets the scheme, host and credentials checks of llmGateway.url. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
d0912f2 to
5f058fb
Compare
| "Connect GitHub", | ||
| &format!( | ||
| r#" | ||
| <p id="description">Connect your GitHub account to use Copilot through Agentdesktop.</p> |
There was a problem hiding this comment.
could you move this html into github_oauth.html or and then load it instead of having it directly in source like this?
There was a problem hiding this comment.
Done in 0e777cd: the page body is now crates/agent/src/github_oauth.html, loaded with include_str!; the escaped user code fills a {{user_code}} placeholder. The two smaller fragments this PR adds in oidc.rs follow that file's existing inline pages; happy to move those (or all of oidc.rs's pages) into files too if you'd prefer. The stack above (#135–#138) is rebased onto it.
The page body moves out of a format! string into github_oauth.html,
loaded with include_str!; the escaped user code replaces {{user_code}}.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| if continuation.is_none() | ||
| && let Err(error) = open::that_detached(&page_url) | ||
| { | ||
| tracing::warn!(%error, "could not open the Agentdesktop sign-in page; use the URL above"); |
Part of #86 (series overview: #86 overview). Based on main (#133 is merged). The second commit addresses the review.
Why
Copilot CLI and VS Code Copilot Chat accept only a static API key; they have no credential-helper hook like Claude Code's
apiKeyHelper. So the device's gateway identity cannot reach the gateway from these clients.Change
An optional loopback proxy,
daemon.llmProxy.listen, forwards a client's model requests to the gateway and adds the device's gateway credential (the same one the API-key helpers return)./copilot-cli,/vscode-copilot,/vscode-copilot-passthrough). The client ID the gateway sees comes from the route, not the request.llmProxy.bound,error).llmGateway.proxyUrlsets a proxy target other thanurl.llmGateway.githubOAuthsets the upstream GitHub credential: taken from the client's request or from a GitHub App device flow.John Howard and James McShane started the listener and the GitHub credential handling; both are co-authors.
Risk and compatibility
x-agentdesktop-pairing.OPTIONSkeep browser pages out.x-llm-tokenand puts the gateway identity inAuthorization.How to review
llm_proxy.rsholds routing, pairing, checks and forwarding.daemon.rsholds the bind and startup.github_oauth.rsholds the GitHub credential sources.testsmodule inllm_proxy.rs).Tests
CI green. Unit tests cover routing, pairing, the Host check, errors, the body cap, the retry and the credential cache. Lab-tested on a Linux VM with a controller-managed daemon: the bind and a failed bind, daemon-info, and a request through each route to agentgateway. Review fixes (second commit), lab-tested on the same VM: 8 concurrent requests on one route share one controller issuance, a request sent while a long one is still running reuses its credential, two routes fetch independently, and an
ftp://proxyUrlis rejected by the controller. The GitHub sign-in failure page and the Bearer scheme are covered by unit tests only; they were not re-tested on the live path.Screenshot
Desktop app, daemon information with the local proxy (here not bound, with the reason). Storybook: Desktop/Status, StandaloneProxyUnbound.
🤖 Generated with Claude Code