Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions articles/flow/ai-support/ai-powered-chart.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,13 @@
[classname]`ChartAIController` does not support OpenAI's strict tool-calling mode. Strict mode is off by default in both LangChain4j and Spring AI; only users who explicitly opt in are affected.


== Query and Configuration Validation

Before queueing an update, the controller validates the LLM's work within the turn: series queries are executed against the database, and configuration JSON is parsed eagerly. Invalid updates are rejected and never reach the chart.

Check failure on line 58 in articles/flow/ai-support/ai-powered-chart.adoc

View workflow job for this annotation

GitHub Actions / lint

[vale] reported by reviewdog 🐶 [Vale.Spelling] Did you really mean 'LLM's'? Raw Output: {"message":"[Vale.Spelling] Did you really mean 'LLM's'?","location":{"path":"articles/flow/ai-support/ai-powered-chart.adoc","range":{"start":{"line":58,"column":57},"end":{"line":58,"column":62}}},"severity":"ERROR","code":{"value":"Vale.Spelling"}}

The LLM can [since:com.vaadin:vaadin@V25.3]#learn why a rejected update failed#: configuration parse errors are relayed as-is, since the LLM authored the configuration. Query errors are relayed only if your [classname]`DatabaseProvider` implementation opts in by throwing a [classname]`ToolException` from [methodname]`executeQuery()`; any other exception is logged and replaced with a generic error message. See <<controllers#tool-error-handling,Tool Error Handling>>.


== Persisting Chart State

[classname]`ChartState` captures both the SQL queries and the Highcharts configuration. Register a state change listener to persist the state automatically after each successful AI request:
Expand Down
4 changes: 3 additions & 1 deletion articles/flow/ai-support/ai-powered-grid.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,9 @@

== Query Validation

Before queueing an update, the controller runs a lightweight probe against the database to validate the LLM's query. If the probe fails, the error is returned to the LLM so that it can correct the query on the next turn. Invalid queries never reach the grid.
Before queueing an update, the controller runs a lightweight probe against the database to validate the LLM's query. If the probe fails, the query is rejected and never reaches the grid.

Check failure on line 54 in articles/flow/ai-support/ai-powered-grid.adoc

View workflow job for this annotation

GitHub Actions / lint

[vale] reported by reviewdog 🐶 [Vale.Spelling] Did you really mean 'LLM's'? Raw Output: {"message":"[Vale.Spelling] Did you really mean 'LLM's'?","location":{"path":"articles/flow/ai-support/ai-powered-grid.adoc","range":{"start":{"line":54,"column":105},"end":{"line":54,"column":110}}},"severity":"ERROR","code":{"value":"Vale.Spelling"}}

The LLM can [since:com.vaadin:vaadin@V25.3]#learn why the query failed#, but only if your [classname]`DatabaseProvider` implementation opts in by throwing a [classname]`ToolException` from [methodname]`executeQuery()` -- its message is relayed to the LLM so it can correct the query on the next turn. Any other exception is logged and replaced with a generic error message. See <<controllers#tool-error-handling,Tool Error Handling>>.


== Persisting Grid State
Expand Down
37 changes: 35 additions & 2 deletions articles/flow/ai-support/controllers.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -91,14 +91,19 @@ public class JdbcDatabaseProvider implements DatabaseProvider {
}
return rows;
} catch (SQLException e) {
throw new IllegalArgumentException("Query failed: " + e.getMessage(), e);
// ToolException relays the message to the LLM so it can
// correct its query. Check what your database includes in
// its error messages before relaying them like this.
throw new ToolException("Query failed: " + e.getMessage(), e);
}
}
}
----

The example returns a hardcoded string for clarity, but [methodname]`getSchema()` can return whatever helps the LLM. Build the description at runtime from [classname]`java.sql.DatabaseMetaData` if your schema changes often, and add free-form context -- column meanings, business rules, common joins, units, sample values -- in plain English. The LLM treats the entire string as guidance, so anything that improves its queries is fair game.

When a query fails -- for example, because it references an unknown column -- the exception type determines what the LLM learns about the failure. The example above [since:com.vaadin:vaadin@V25.3]#throws a `ToolException`#, whose message is relayed to the LLM so it can correct the query on its next attempt; any other exception is replaced with a generic error message. See <<#tool-error-handling,Tool Error Handling>>.

.Read-Only Database Access
[IMPORTANT]
The LLM writes the SQL that gets executed. Always back a [classname]`DatabaseProvider` implementation with a database account that has read-only access to the tables and views you intend to expose. This prevents the LLM from modifying or deleting data and limits the impact of a prompt-injection attempt that tries to trick the LLM into running destructive statements.
Expand Down Expand Up @@ -128,7 +133,7 @@ Each tool is an implementation of [classname]`LLMProvider.ToolSpec`, which has f
* [methodname]`getName()` -- the unique name the LLM uses to invoke the tool.
* [methodname]`getDescription()` -- a human-readable description shown to the LLM.
* [methodname]`getParametersSchema()` -- a JSON Schema string describing the tool's parameters, or `null` for tools that take no parameters.
* [methodname]`execute(JsonNode arguments)` -- receives the arguments passed by the LLM as a [classname]`JsonNode` (from `tools.jackson.databind`) and returns the tool's result as a string.
* [methodname]`execute(JsonNode arguments)` -- receives the arguments passed by the LLM as a [classname]`JsonNode` (from `tools.jackson.databind`) and returns the tool's result as a string. To report a failure in terms the LLM is allowed to see, throw a [classname]`ToolException` -- see <<#tool-error-handling,Tool Error Handling>>.

A minimal custom controller looks like this:

Expand Down Expand Up @@ -189,3 +194,31 @@ Hand-writing JSON schemas is fine for small tools but becomes error-prone as the
[NOTE]
Controllers are not serialized with the orchestrator. After session restore, pass the controller to [methodname]`reconnect(provider).withController(controller).apply()` -- see <<conversation-history#,Conversation History & Session Persistence>>.


[[tool-error-handling]]
[role="since:com.vaadin:vaadin@V25.3"]
== Tool Error Handling

When a tool call fails, the message the LLM receives determines whether it can recover. Given the actual reason, it can correct its next attempt; given a generic error, it tends to retry the same call unchanged.

By default, any exception thrown from tool code is caught, logged, and replaced with a generic error message before reaching the LLM, so internal details such as SQL fragments, schema names, or file paths don't leak into the conversation. To let the LLM see why a call failed, throw a [classname]`ToolException` (from the `com.vaadin.flow.component.ai.provider` package) -- its message is forwarded to the LLM verbatim as the tool's error output:

[source,java]
----
@Override
public String execute(JsonNode arguments) {
String city = arguments.get("city").asString();
if (!weatherService.isKnownCity(city)) {
throw new ToolException("Unknown city: '" + city
+ "'. Pass the city name in English, e.g. 'Munich'.");
}
return weatherService.lookup(city);
}
----

[classname]`ToolException` works anywhere the framework executes tool code: a [methodname]`ToolSpec.execute()` implementation or a <<#database-provider,[classname]`DatabaseProvider`>> alike. Because the message is passed to the LLM as-is, make sure it's safe to expose: no personal data, no internal identifiers beyond what the LLM already sent, and no third-party error text you haven't vetted. A database error that points out a mistake in an LLM-written query is usually reasonable to relay, but check first what your database actually puts in its error messages. A stack trace from your infrastructure is never safe to relay.

.Vendor-Annotated Tools Are Out of Scope
[NOTE]
This contract applies only to framework-agnostic [classname]`ToolSpec` tools -- those contributed by controllers and by [classname]`DatabaseProvider`-backed features. Tool objects registered via <<tool-calling#,[methodname]`withTools()`>> are executed by the vendor framework itself, whose own error handling decides what reaches the LLM: by default, both LangChain4j and Spring AI relay the raw message of any exception. Route error-sensitive tools through a controller to keep control over what the model sees.

4 changes: 4 additions & 0 deletions articles/flow/ai-support/tool-calling.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ For Spring AI, use [annotationname]`@org.springframework.ai.tool.annotation.Tool
[TIP]
For a reusable set of tools that does not depend on a specific LLM framework's annotations, or when a lifecycle hook is needed after each LLM request cycle, implement <<controllers#,[classname]`AIController`>> instead. [classname]`GridAIController` and [classname]`ChartAIController` are built-in examples. Controllers and tool objects can be combined on the same orchestrator.

.Tool Errors
[NOTE]
Tool objects registered via [methodname]`withTools()` are executed by the vendor framework, whose own error handling decides what the LLM sees when a tool throws -- by default, both LangChain4j and Spring AI relay the raw message of any exception. To control what the LLM learns about failures, define the tool through a controller instead and throw a [classname]`ToolException` for messages the LLM is meant to see; see <<controllers#tool-error-handling,Tool Error Handling>>.


== Programmatic Prompts

Expand Down
Loading