With NATIVE Thinking - requires attention:

[arnabnandy7]

With NATIVE Thinking - requires attention:

16:29:16.399 [main] DEBUG CacheEligibilityResolver - Caching not enabled for tool definition, cacheStrategy=NONE
16:29:17.767 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: 'The user is asking about'
16:29:18.166 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: ' the two hottest months in Florida and their highest temperatures. They want me to convert the temperatures to Celsius using the available tools'
16:29:18.673 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: '.\n\nBased on general knowledge, the hottest months in Florida are typically July and August. The average high temperatures during'
16:29:18.673 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: 'The user is asking about the two hottest months in Florida and their highest temperatures. They want me to convert the temperatures to Celsius using the available tools.'
16:29:18.679 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: The user is asking about the two hottest months in Florida and their highest temperatures. They want me to convert the temperatures to Celsius using the available tools.
16:29:19.111 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: ' these months are around 90-92°F.\n\nHowever, I need to be precise about what "highest'
16:29:19.111 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: 'Based on general knowledge, the hottest months in Florida are typically July and August. The average high temperatures during these months are around 90-92°F.'
16:29:19.111 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: Based on general knowledge, the hottest months in Florida are typically July and August. The average high temperatures during these months are around 90-92°F.
16:29:19.624 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: ' temperatures" means - this could mean:\n1. Average high temperatures for those months\n2. Record high temperatures ever recorded in those'
16:29:19.624 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: 'However, I need to be precise about what "highest temperatures" means - this could mean:'
16:29:19.624 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: However, I need to be precise about what "highest temperatures" means - this could mean:
16:29:19.625 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: '1. Average high temperatures for those months'
16:29:19.625 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: 1. Average high temperatures for those months
16:29:19.959 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: ' months\n\nSince the user asks for "their respective highest temperatures" and wants exactly two months'
16:29:19.959 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: '2. Record high temperatures ever recorded in those months'
16:29:19.959 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: 2. Record high temperatures ever recorded in those months
16:29:20.399 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: ', I'll interpret this as the typical peak temperatures during the'
16:29:21.131 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: ' hottest months. In Florida:\n- July: typically around 91-92°F average high'
16:29:21.131 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: 'Since the user asks for "their respective highest temperatures" and wants exactly two months, I'll interpret this as the typical peak temperatures during the hottest months. In Florida:'
16:29:21.131 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: Since the user asks for "their respective highest temperatures" and wants exactly two months, I'll interpret this as the typical peak temperatures during the hottest months. In Florida:
16:29:21.415 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: '\n- August: typically around 91-92°F average high\n\nFor conversion purposes, I'll use reasonable values. Let me use:'
16:29:21.415 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: '- July: typically around 91-92°F average high'
16:29:21.416 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: - July: typically around 91-92°F average high
16:29:21.416 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: '- August: typically around 91-92°F average high'
16:29:21.416 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: - August: typically around 91-92°F average high
16:29:21.847 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: '\n- July: 92°F\n- August: 91°F\n\nI need to convert these from Fahrenheit to Celsius using the convertFromFahrenheitToCelsius function.\n\nLet'
16:29:21.847 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: 'For conversion purposes, I'll use reasonable values. Let me use:'
16:29:21.847 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: For conversion purposes, I'll use reasonable values. Let me use:
16:29:21.848 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: '- July: 92°F'
16:29:21.848 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: - July: 92°F
16:29:21.848 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: '- August: 91°F'
16:29:21.848 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: - August: 91°F
16:29:21.849 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: 'I need to convert these from Fahrenheit to Celsius using the convertFromFahrenheitToCelsius function.'
16:29:21.849 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: I need to convert these from Fahrenheit to Celsius using the convertFromFahrenheitToCelsius function.
16:29:22.359 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: ' me make the function calls to convert both temperatures.'
16:29:22.382 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: '<think>\nThe user is asking for the two hottest months in Florida and their highest temperatures,'
16:29:22.383 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: 'Let me make the function calls to convert both temperatures.<think>'
16:29:22.383 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: Let me make the function calls to convert both temperatures.<think>
16:29:22.973 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: ' converted to Celsius. Based on climatological data, the two hottest months in Florida are typically July and August, with average high'
16:29:23.286 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: ' temperatures around 91-92°F. I'll convert these temperatures from Fahrenheit to Celsius using the available tool.\n</think>'
16:29:23.286 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: 'The user is asking for the two hottest months in Florida and their highest temperatures, converted to Celsius. Based on climatological data, the two hottest months in Florida are typically July and August, with average high temperatures around 91-92°F. I'll convert these temperatures from Fahrenheit to Celsius using the available tool.'
16:29:23.286 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: The user is asking for the two hottest months in Florida and their highest temperatures, converted to Celsius. Based on climatological data, the two hottest months in Florida are typically July and August, with average high temperatures around 91-92°F. I'll convert these temperatures from Fahrenheit to Celsius using the available tool.
16:29:24.137 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - beforeToolCall: tool=convertFromFahrenheitToCelsius, argsLength=17
16:29:24.138 [boundedElastic-1] WARN  ObservabilityTool - No parent observation for tool call convertFromFahrenheitToCelsius with input: {"inputTemp": 92}, observation registry: ObservationRegistry.NOOP
16:29:24.140 [boundedElastic-1] INFO  Embabel - [exciting_burnell] calling tool convertFromFahrenheitToCelsius({"inputTemp": 92})
16:29:24.147 [boundedElastic-1] INFO  Embabel - [exciting_burnell] tool convertFromFahrenheitToCelsius returned 30 in 6ms with payload {"inputTemp": 92}
16:29:24.150 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - afterToolCall: tool=convertFromFahrenheitToCelsius, status=Text, resultLength=2, durationMs=11
16:29:24.151 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - beforeToolCall: tool=convertFromFahrenheitToCelsius, argsLength=17
16:29:24.151 [boundedElastic-1] WARN  ObservabilityTool - No parent observation for tool call convertFromFahrenheitToCelsius with input: {"inputTemp": 91}, observation registry: ObservationRegistry.NOOP
16:29:24.152 [boundedElastic-1] INFO  Embabel - [exciting_burnell] calling tool convertFromFahrenheitToCelsius({"inputTemp": 91})
16:29:24.152 [boundedElastic-1] INFO  Embabel - [exciting_burnell] tool convertFromFahrenheitToCelsius returned 29 in 0ms with payload {"inputTemp": 91}
16:29:24.152 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - afterToolCall: tool=convertFromFahrenheitToCelsius, status=Text, resultLength=2, durationMs=1
16:29:24.164 [boundedElastic-1] DEBUG CacheEligibilityResolver - Caching not enabled for tool definition, cacheStrategy=NONE
16:29:25.331 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: '<think'
16:29:25.841 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: '>\nThe two hottest months in Florida are typically July and August. Based on climate data:'
16:29:25.841 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: '</think><think>'
16:29:25.841 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: </think><think>
16:29:26.250 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: '\n- July: Average high around 91-92°F\n- August: Average high around 91-92°F\n\nI'
16:29:26.250 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: 'The two hottest months in Florida are typically July and August. Based on climate data:'
16:29:26.250 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: The two hottest months in Florida are typically July and August. Based on climate data:
16:29:26.251 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: '- July: Average high around 91-92°F'
16:29:26.251 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: - July: Average high around 91-92°F
16:29:26.251 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: '- August: Average high around 91-92°F'
16:29:26.251 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: - August: Average high around 91-92°F
16:29:26.762 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: ''ve converted the temperatures from Fahrenheit to Celsius using the tool:\n- 92°F = 30°C\n- 91°F = '
16:29:26.762 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: 'I've converted the temperatures from Fahrenheit to Celsius using the tool:'
16:29:26.762 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: I've converted the temperatures from Fahrenheit to Celsius using the tool:
16:29:26.763 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: '- 92°F = 30°C'
16:29:26.763 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: - 92°F = 30°C
16:29:27.274 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: '29°C\n\nSince both months have very similar temperatures, I'll use July at'
16:29:27.274 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: '- 91°F = 29°C'
16:29:27.275 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: - 91°F = 29°C
16:29:27.684 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: ' 92°F (30°C) and August at 91°F (29°C) as the two hottest months, though they're nearly'
16:29:28.201 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: ' tied.\n\nNow I need to output this in JSON Lines format according to the schema provided,'
16:29:28.201 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: 'Since both months have very similar temperatures, I'll use July at 92°F (30°C) and August at 91°F (29°C) as the two hottest months, though they're nearly tied.'
16:29:28.202 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: Since both months have very similar temperatures, I'll use July at 92°F (30°C) and August at 91°F (29°C) as the two hottest months, though they're nearly tied.
16:29:28.582 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: ' which requires "name" (string) and "temperature" (integer) fields.\n</think>\n\n{"name": "July",'
16:29:28.583 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: 'Now I need to output this in JSON Lines format according to the schema provided, which requires "name" (string) and "temperature" (integer) fields.'
16:29:28.583 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: Now I need to output this in JSON Lines format according to the schema provided, which requires "name" (string) and "temperature" (integer) fields.
16:29:28.583 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: '</think>'
16:29:28.583 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received thinking: </think>
16:29:28.810 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: ' "temperature": 30}\n{"name": "August", "temperature": 29}'
16:29:28.810 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: '{"name": "July", "temperature": 30}'
16:29:28.813 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received object: July
16:29:28.816 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - FINAL LINE: '{"name": "August", "temperature": 29}'
16:29:28.817 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: '{"name": "August", "temperature": 29}'
16:29:28.817 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test received object: August
16:29:28.817 [boundedElastic-1] INFO  LLMAnthropicStreamingBuilderIT - Integration test stream completed successfully
16:29:28.818 [main] INFO  LLMAnthropicStreamingBuilderIT - Integration streaming test completed successfully with 26 total events
16:29:28.832 [SpringApplicationShutdownHook] INFO  JvmType - Clearing JvmType children cache (0 entries)

Note:

[16:29:25.331 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: '<think'
16:29:25.841 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - RAW CHUNK: '>\nThe two hottest months in Florida are typically July and August. Based on climate data:'
16:29:25.841 [boundedElastic-1] TRACE StreamingLlmOperationsImpl - COMPLETE LINE: '</think>

it was discussed separately on generate stream

How do you know in advance how long the sequence of raw chunks ....... is? If it's too long, it defeats the purpose of streaming.

But putting this aside, could you please carefully verify every line in the log to ensure the reasoning before calling the tool execution and in between is preserved?

thats how to enable:

  LlmOptions thinkingOptions = new LlmOptions().withThinking(Thinking.withTokenBudget(8000));

        // Given: Use the existing streaming test LLM (configured as "best")
        PromptRunner runner = ai.withDefaultLlm().withLlm(thinkingOptions)```

_Originally posted by @igordayen in https://github.com/embabel/embabel-agent/issues/1826#issuecomment-5153338438_
            

[arnabnandy7]

working on this.

[igordayen]

@arnabnandy7 - tagged as dup of #1716 provisionally