Skip to content

Implement model selection for chat with Ollama and external…#233

Open
FilipCivljak wants to merge 1 commit into
mainfrom
model-selection
Open

Implement model selection for chat with Ollama and external…#233
FilipCivljak wants to merge 1 commit into
mainfrom
model-selection

Conversation

@FilipCivljak
Copy link
Copy Markdown
Contributor

… LLM providers

What type of PR is this?

This is a feature because it adds model selection for chat, supporting both local Ollama models and external LLM providers (OpenAI, Anthropic).

What does this do?

Adds a ModelConfig struct to the chat domain and extends the Chat() interface to accept a per-request model override
Implements a ClientFactory pattern in the LLM layer so the embedder can build a per-request client (Ollama or OpenAI-compatible) without circular imports
Adds Temperature and MaxTokens support to both the Ollama and OpenAI LLM clients
Adds a GET /api/v1/models/ollama endpoint that proxies Ollama's model list, filtering out non-chat models (embeddings, code-completion)
Adds ui/src/lib/modelConfig.ts with localStorage persistence for model configuration
Updates the Config page to load/save model settings, fetch live Ollama models, and show an API key field for external providers
Updates the Chat page to read the saved model config and pass it to every chat request

Which issue(s) does this PR fix/relate to?

Have you included tests for your changes?

Did you document any new/modified features?

Notes

The local provider maps to Ollama using the server-configured base URL; openai and anthropic use OpenAI-compatible APIs with the provided API key stored only in the browser's localStorage
If no model is configured by the user, the server falls back to its default LLM configuration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant