Mistral is set as prompt chat options instead of configured llm in Ollama starter

[AccsoOliverMichalke]

embabel-ollama-mistral-bug.zip

I am having an issue where i can't start up embabel in the 2.0.0-SNAPSHOT with a local Ollama. The agent always tries to run a Mistral LLM instead of the configured default-llm.

I tracked the error down to ChatClientLlmOperations.createMessageSender(...) where val chatOptions = springAiLlm.optionsConverter.convertOptions(options) calls the OllamaChatOptions which loose the information about the set model of the SpringAiLlmService. So the SpringAiLlmMessageSender is initialized with false chatOptions which will be inserted in the Prompt. So embabel falls back to the default OllamaChatOptions which set mistral as the default when no model is set this.model = model != null ? model : OllamaModel.MISTRAL.id();.

I fixed this locally by excluding the AgentOllamaAutoConfiguration by spring.autoconfigure.exclude: com.embabel.agent.autoconfigure.models.ollama.AgentOllamaAutoConfiguration, copying the OllamaModelsConfig and setting a custom OllamaOptionsConverter which gets the model injected in the constructor and setting the model in the convertOptions method.

It's a rather ugly fix, but it shows that it works well and the correct model is chosen instead of mistral. I guess the real idea would be to change the default-behaviour of theOllamaChatOptions to not set mistral as default or having a fix in embabel to set the embabel.models.default-llm in case of a null-model.

I attached a simple project to test this behaviour. Starting it will show the error. On enabling the de.accso.testing.embabel.OllamaModelsConfig and including the autoconfigure-exclusion in the application.properties the error will be gone (But the test will still fail since my agent doesn't work as expected. But you will see that the llm gets called.

[igordayen]

@AccsoOliverMichalke , @alexheifetz - applied label for release 2.0 accordingly, thank you for reporting

[sumin220]

I can reproduce this with a failing unit test in the Ollama starter: a model registered as gemma3:latest produces per-request options with model=mistral (Expected <gemma3:latest>, actual <mistral>).

Root cause: OllamaOptionsConverter.convertOptions in OllamaModelsConfig.kt never sets a model, Spring AI 2.0's OllamaChatOptions coerces a null model to OllamaModel.MISTRAL.id(), and Spring AI 2.0 no longer merges the OllamaChatModel's default options into per-request options — so the configured model is dropped on every call.

PR #1816 fixes this class of bug generically in SpringAiLlmService and includes an Ollama binding test, but it hasn't merged yet.

A small starter-level fix also works and is compatible with #1816: bind the registered model onto the converter output (OllamaOptionsConverter.withModel(uniqueModelName) wired in ollamaLlmOf) — about 15 lines plus regression tests.

I have this implemented with the failing-then-passing test. Happy to open a PR for the starter-level fix, or hold off if maintainers prefer to wait for #1816 — please let me know.

[igordayen]

I have this implemented with the failing-then-passing test. Happy to open a PR for the starter-level fix, or hold off if maintainers prefer to wait for #1816 — please let me know.

@sumin220 , @AccsoOliverMichalke - thank you .
Could you please re-test with #1818?

[sumin220]

Re-tested with #1818 — it fixes the Ollama path.

I took the failing regression test from my earlier comment (real OllamaModelsConfig autoconfig path with a mocked /api/tags, model registered as gemma3:latest), pointed the assertion at the new buildChatOptions(...), and ran it against pr-1818 head 0d4bc04 with embabel-agent-api built locally from the PR. On main it still fails with Expected <gemma3:latest>, actual <mistral>; on #1818 it passes, and the rest of the Ollama autoconfigure module suite passes as well (28/28).

One thing I noticed along the way: for node-prefixed registrations name binds e.g. main-gemma3:latest, which isn't a real Ollama tag. That matches the existing default-options behavior so it's probably fine here, but it might be worth its own issue.

If it's useful I can send the regression test as a small follow-up PR to the Ollama module so this path stays covered.

[igordayen]

@sumin220 @AccsoOliverMichalke Thanks very much for testing!
All contributions to issues are welcome, expected after this massive upgrade. Thank you.
@alexheifetz , @simeshev -FYI

[sumin220]

Thanks for the assign! I'll send the Ollama regression test as a small PR once #1818 lands, and can file the node-prefixed name question as a separate issue if you'd like it tracked.