Skip to content

feat(claude-native): enumerate a gateway provider's live models in the launch picker - #7876

Draft
btli wants to merge 3 commits into
omnigent-ai:mainfrom
btli:feat/claude-native-gateway-live-models
Draft

btli wants to merge 3 commits into
omnigent-ai:mainfrom
btli:feat/claude-native-gateway-live-models

Conversation

@btli

@btli btli commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Related issue

Closes #7875

Summary

The claude-native launch picker is built from the claude -p /model probe, then claude_model_catalog() drops every bare claude-* row whenever ANTHROPIC_BASE_URL's host is not anthropic.com. That hostname heuristic protects the namespaced gateways (Bedrock, Databricks) but wrongly empties the picker for a generic Anthropic passthrough (e.g. a LiteLLM Anthropic passthrough) that serves bare canonical ids — the picker reads "Models unavailable" even though turns route fine.

This mirrors the pi pattern from #7007 / #7713 (closed #7019 / #6185):

  • A key/gateway/local anthropic provider records a listing_provider (ResolvedModelProvider) descriptor on its ClaudeNativeUcodeConfig — metadata only, no build-time I/O, compare=False, hash=False (like pi's PiProviderConfig.listing_provider).
  • claude_model_catalog() enumerates that endpoint's <base>/v1/models through the shared model-catalog fetcher (_fetch_openai_compatible_listing for a gateway, _fetch_anthropic_listing for a real anthropic key), off-thread and fail-open.
  • The gateway's own listing is the source of truth: a bare claude-* id — concrete or a claude-* / * wildcard — marks the gateway canonical, so the probe's claude-* rows are kept and any concrete listed ids are surfaced; a namespaced listing (OpenRouter's anthropic/claude-*) leaves them dropped, since Claude Code can't drive those.
  • claude_catalog_fingerprint() keys on the listing endpoint (kind/family/base_url — never the credential) so the persisted catalog re-probes when the configured gateway changes.

This replaces the need for any per-host flag or hostname special-casing: a LiteLLM gateway configured as a providers: entry "just works", and namespaced/unsupported gateways are correctly excluded.

Test Plan

  • uv run pytest tests/test_claude_native.py → 396 pass (9 new): _gateway_claude_listing for wildcard / concrete / namespaced (OpenRouter) / unreachable; the descriptor is recorded by _provider_config_for_native_claude; the catalog keeps canonical rows when the listing serves them, drops them when namespaced, surfaces+dedupes concrete ids; and the fingerprint keys on the endpoint. The keep/drop pair differ only in the listing result and produce opposite outcomes (mutation check).
  • ruff check / ruff format --check / pyrefly check clean.
  • Verified live: the target LiteLLM gateway's /v1/models lists claude-* and POST /v1/messages with claude-sonnet-4-5-… returns 200; OpenRouter's /api/v1/models lists only anthropic/claude-* (no bare claude-*), so it is correctly excluded.

Demo

  • Visual demo attached below
  • Non-visual evidence provided below or in Test Plan
  • Not applicable — no behavioral change

Type of change

  • Bug fix
  • Feature
  • UI / frontend change
  • Refactor / chore
  • Docs
  • Test / CI
  • Breaking change

Test coverage

  • Unit tests added / updated
  • Integration tests added / updated
  • E2E tests added / updated
  • Manual verification completed
  • Existing tests cover this change
  • Not applicable

Coverage notes

Manually verified the live gateway listings (LiteLLM lists claude-* and serves claude-sonnet-4-5 over /v1/messages; OpenRouter lists only anthropic/claude-*). Unit tests cover the listing classifier, the descriptor recording, all three catalog outcomes (keep / drop / surface-concrete), and the fingerprint key.

Changelog

Claude Code sessions routed through a configured Anthropic-passthrough gateway (e.g. LiteLLM) now list the gateway's models in the launch picker instead of showing "Models unavailable".

🤖 Generated with Claude Code

Issues

Resolves OMNI-9136

@github-actions github-actions Bot added P2-medium Priority: bug with workaround, important feature request size/L Pull request size: L labels Sep 20, 2026
btli pushed a commit to btli/omnigent that referenced this pull request Sep 20, 2026
@omni-resolve-agent

Copy link
Copy Markdown
Contributor

Resolve-agent reviewed this contributor PR as the candidate fix. Future actionable maintainer review feedback may be remediated automatically.

@omni-resolve-agent omni-resolve-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolve-agent review outcome: fixed.

Summary
PR #7876 (btli) mirrors pi #7007: ClaudeNativeUcodeConfig gains a metadata-only listing_provider descriptor recorded by _provider_config_for_native_claude; claude_model_catalog() enumerates the endpoint's own /v1/models off-thread and fail-open via the shared model-catalog fetchers — a bare claude-* id (concrete or wildcard) marks the gateway canonical so probe rows are kept and concrete listed ids are surfaced/deduped, a namespaced-only listing (OpenRouter) still drops them, an unreachable/empty listing fails open; claude_catalog_fingerprint() keys on the non-secret listing endpoint. Bedrock/Databricks builders record no descriptor, so the hostname heuristic still protects them

Root cause
claude_model_catalog() classifies the endpoint by hostname alone (_serves_canonical_anthropic_ids / _ambient_env_is_non_anthropic_gateway): any non-anthropic.com ANTHROPIC_BASE_URL host is treated as a namespaced gateway, so every probe row resolving to a bare claude-* id is dropped and the configured default is refused, persisting an empty catalog that the New Chat Claude picker renders as 'Models unavailable' — even for a generic Anthropic-passthrough gateway whose /v1/models lists those exact ids and whose /v1/messages routes them

CI
PR head (fork draft): security/hygiene/DCO/Trivy lanes pass; heavy lanes (Pytest, E2E UI, Pre-commit, Docker) skipping on the draft fork and the Maintainer Approval gate is pending — expected, not diff-caused. Locally on the PR head: repro e2e passes and tests/test_claude_native.py is 399-pass green (ruff/pyrefly unreachable from this sandbox; GitHub CI owns them after publication)

Automated review
no real Polly review has run on this PR (no polly-review-bot comment; fork PRs' automatic check is a credentials-gated no-op). Dispatching one is a GitHub write owned by the workflow publisher — this implementation-only job performed no GitHub writes, so approval is withheld and the review verdict ships as findings for the publisher to post

@github-actions github-actions Bot added size/XL Pull request size: XL and removed size/L Pull request size: L labels Sep 20, 2026
btli pushed a commit to btli/omnigent that referenced this pull request Sep 20, 2026
btli pushed a commit to btli/omnigent that referenced this pull request Sep 20, 2026
btli pushed a commit to btli/omnigent that referenced this pull request Sep 21, 2026
btli pushed a commit to btli/omnigent that referenced this pull request Sep 21, 2026
btli pushed a commit to btli/omnigent that referenced this pull request Sep 21, 2026
btli pushed a commit to btli/omnigent that referenced this pull request Sep 21, 2026
btli pushed a commit to btli/omnigent that referenced this pull request Sep 22, 2026
btli pushed a commit to btli/omnigent that referenced this pull request Sep 22, 2026
@btli
btli force-pushed the feat/claude-native-gateway-live-models branch from 2d7e388 to e36ef2a Compare September 22, 2026 16:47
btli pushed a commit to btli/omnigent that referenced this pull request Sep 22, 2026
btli pushed a commit to btli/omnigent that referenced this pull request Sep 22, 2026
btli pushed a commit to btli/omnigent that referenced this pull request Sep 22, 2026
btli pushed a commit to btli/omnigent that referenced this pull request Sep 23, 2026
btli pushed a commit to btli/omnigent that referenced this pull request Sep 23, 2026
btli pushed a commit to btli/omnigent that referenced this pull request Sep 23, 2026
@btli
btli force-pushed the feat/claude-native-gateway-live-models branch 2 times, most recently from 6aeb534 to 97d42a1 Compare September 23, 2026 14:52
btli pushed a commit to btli/omnigent that referenced this pull request Sep 23, 2026
@btli
btli force-pushed the feat/claude-native-gateway-live-models branch from 97d42a1 to 1078ea8 Compare September 23, 2026 19:24
Bryan Li and others added 3 commits September 23, 2026 14:51
…e launch picker

The claude-native picker is built from the `claude -p /model` probe, then
`claude_model_catalog()` drops every bare `claude-*` row whenever
`ANTHROPIC_BASE_URL` host is not `anthropic.com` — a hostname heuristic that
protects the namespaced gateways (Bedrock, Databricks) but wrongly empties the
picker for a generic Anthropic passthrough (LiteLLM) that serves bare canonical
ids ("Models unavailable").

Mirror the pi pattern (omnigent-ai#7007): a key/gateway/local anthropic provider records a
`listing_provider` descriptor on its `ClaudeNativeUcodeConfig` (metadata only,
no build-time I/O), and `claude_model_catalog` enumerates `<base>/v1/models`
through the shared model-catalog fetcher (off-thread, fail-open). The gateway's
own listing is the source of truth: a bare `claude-*` (concrete or a
`claude-*`/`*` wildcard) keeps the probe's canonical rows and surfaces the
concrete ids listed; a namespaced listing (OpenRouter's `anthropic/claude-*`)
leaves them dropped. The catalog fingerprint keys on the listing endpoint
(never the credential) so a changed gateway re-probes.

Closes omnigent-ai#7875

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Bryan Li <bryan@bryanli.net>
… on gateway listing

Adversarial review (Claude + Codex):
- `_gateway_claude_listing` is now three-valued (True/False/None). A transient
  or empty/unparseable listing returns None so `claude_model_catalog` fails
  OPEN — keeps the probe's canonical rows instead of dropping them and caching
  an empty picker for the store TTL. Only a reachable listing with no bare
  claude-* (OpenRouter) authoritatively drops.
- Servability matching is case-sensitive (Claude Code/Anthropic ids are
  lowercase), so a differently-cased id can't falsely mark the endpoint canonical.
- Concrete-id surfacing dedupes within the listing too (running `known` set), so
  a repeated id can't emit duplicate rows.
- Default-append servability derives from the same `_non_canonical` decision, so
  the fail-open path keeps a canonical default too.
- Tests: the listing provider's base_url now equals ANTHROPIC_BASE_URL (as
  `_provider_config_for_native_claude` builds it), plus undetermined/empty/
  case-sensitive and fail-open catalog cases.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Bryan Li <bryan@bryanli.net>
…models

End-to-end regression for the claude-native launch picker: a generic
Anthropic-passthrough gateway (bare canonical claude-* ids on /v1/models,
routed /v1/messages) configured as the claude provider must yield those
models in the New Chat Claude picker instead of "Models unavailable".
Boots a real server + host with the gateway and a scripted claude CLI
double; fails on the hostname-heuristic drop, passes with the live
/v1/models enumeration.
@btli
btli force-pushed the feat/claude-native-gateway-live-models branch from 1078ea8 to c716b46 Compare September 23, 2026 19:51
btli pushed a commit to btli/omnigent that referenced this pull request Sep 23, 2026
btli pushed a commit to btli/omnigent that referenced this pull request Sep 24, 2026
btli pushed a commit to btli/omnigent that referenced this pull request Sep 24, 2026
btli pushed a commit to btli/omnigent that referenced this pull request Sep 24, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2-medium Priority: bug with workaround, important feature request size/XL Pull request size: XL

Projects

None yet

1 participant