Skip to content

Fix LiteLLM provider temperature handling for GPT-5.5 #2459

Description

@aronnoordhoek

Bug Description

When using Hindsight with the LiteLLM provider and Azure OpenAI GPT-5.5, internal LLM calls fail because Hindsight forwards explicit temperature values.

GPT-5.5 only supports the default temperature, so calls with values such as 0.0 or 0.1 are rejected.

Steps to Reproduce

  1. Configure Hindsight with:

    HINDSIGHT_API_LLM_PROVIDER=litellm
    HINDSIGHT_API_LLM_MODEL=azure/gpt-5.5

  2. Run an operation that uses the LiteLLM provider, for example retain/fact extraction.

  3. Observe the LiteLLM/Azure OpenAI error about unsupported temperature.

Expected Behavior

Hindsight should work with Azure OpenAI GPT-5.5 through the LiteLLM provider.

For models that do not support configurable temperature, the LiteLLM provider should omit temperature rather than forwarding explicit internal values.

Actual Behavior

The LiteLLM provider forwards explicit temperature values from internal calls.

Examples include:

  • connection verification uses temperature=0.0
  • retain/fact extraction uses temperature=0.1
  • think uses temperature=0.9
  • mental model delta operations use temperature=0.0

Azure OpenAI GPT-5.5 rejects these calls with an error like:

Unsupported value: 'temperature' does not support 0.1 with this model. Only the default (1) value is supported.

Version

0.8.3

LLM Provider

Other

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions