Skip to content

bench: provider / LLM credential layer#1

Open
antfleet-ops wants to merge 1 commit into
mainfrom
pick/provider-llm-credentials
Open

bench: provider / LLM credential layer#1
antfleet-ops wants to merge 1 commit into
mainfrom
pick/provider-llm-credentials

Conversation

@antfleet-ops

Copy link
Copy Markdown
Contributor

Manual file pick from rennf93/roboco — provider routing, LLM service, credential / self-hosted provider surface (8 files, ~57KB).

Two-model security review (Claude Opus 4.7 + GPT-5).

Methodology: https://github.com/AntFleet/bench-roboco

@antfleet

antfleet Bot commented Jun 19, 2026

Copy link
Copy Markdown

AntFleet · 2 findings

Both reviewers flagged the items below on the changed files. AntFleet posts only what two independent frontier models agree on.


Bug · Medium — upsert_assignment auto-enables the LOCAL provider on any assignment, including AGENT_SLUG/ROLE — surprising side effect on mix mode
roboco/services/llm.py:196-207

If a user is in mix mode and assigns a self-hosted model name to a single agent, this silently re-enables the LOCAL provider — even if the operator had explicitly disabled it via PUT /self-hosted. The Settings UI may then report 'self-hosted enabled' immediately after the user thought they disabled it. This contradicts the principle of least surprise. Worse, base_url may still be unconfigured; resolve_for_agent then probes an empty base_url and returns None, but the enabled bit is sticky in the…

Fix: Don't auto-enable inside upsert_assignment; require the user to explicitly enable LOCAL via the /self-hosted endpoint, or raise an error when assigning to a disabled LOCAL provider.


Api-contract · Low — set_ollama_api_key with empty string clears + DISABLES, but set_ollama_key route docstring promises this — risk of ambiguous tri-state
roboco/services/llm.py:248-266

ProviderUpdate documents tri-state semantics on auth_token: None=unchanged, ''=clear, non-empty=set. set_ollama_api_key collapses an empty input to (auth_token=None, clear_auth_token=True). That works, but it does so by abusing the tri-state contract (auth_token=None normally means 'unchanged'); only the additional clear_auth_token=True flag makes the clear happen. This is fragile if anyone later refactors _apply_auth_token_change to early-return on clear_auth_token=False. Tested: apply

Fix: Either pass auth_token='' (matching documented contract) or remove the auth_token arg when clear_auth_token=True for clarity.

Proposed patch (model: claude-opus-4-7)
            auth_token=key if not clear else "",

Review f612ad01 · claude-opus-4-7 + gpt-5.5 (unanimous) · 113s · ~$0.40

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant