Implement model selection for chat with Ollama and external…#233
Open
FilipCivljak wants to merge 1 commit into
Open
Implement model selection for chat with Ollama and external…#233FilipCivljak wants to merge 1 commit into
FilipCivljak wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
… LLM providers
What type of PR is this?
This is a feature because it adds model selection for chat, supporting both local Ollama models and external LLM providers (OpenAI, Anthropic).
What does this do?
Adds a ModelConfig struct to the chat domain and extends the Chat() interface to accept a per-request model override
Implements a ClientFactory pattern in the LLM layer so the embedder can build a per-request client (Ollama or OpenAI-compatible) without circular imports
Adds Temperature and MaxTokens support to both the Ollama and OpenAI LLM clients
Adds a GET /api/v1/models/ollama endpoint that proxies Ollama's model list, filtering out non-chat models (embeddings, code-completion)
Adds ui/src/lib/modelConfig.ts with localStorage persistence for model configuration
Updates the Config page to load/save model settings, fetch live Ollama models, and show an API key field for external providers
Updates the Chat page to read the saved model config and pass it to every chat request
Which issue(s) does this PR fix/relate to?
Have you included tests for your changes?
Did you document any new/modified features?
Notes
The local provider maps to Ollama using the server-configured base URL; openai and anthropic use OpenAI-compatible APIs with the provided API key stored only in the browser's localStorage
If no model is configured by the user, the server falls back to its default LLM configuration