Problem
The built-in create_confirmation_hook() from ragbits.agents.hooks.confirmation does not work reliably across conversation turns.
When the hook sets decision="ask", the HookManager.execute_pre_tool() method generates a confirmation_id by hashing hook_name:tool_name:json.dumps(arguments, sort_keys=True). Within a single agent turn this
works fine the hash is computed once and matched against itself. But in a human-in-the-loop scenario, where the agent pauses, yields a ConfirmationRequest, and resumes on the next turn after the user confirms, the
LLM regenerates the tool call from scratch. The new arguments are semantically identical but textually different (reordered nested keys, whitespace changes, float 1.0 vs 1, rephrased free-text fields), so the
freshly computed hash no longer matches the confirmation_id the user approved.
This causes either:
- Infinite confirmation loop - the agent keeps asking for confirmation on the same action
- Abrupt termination - the loop detection breaks the agent run, silently dropping the user's intent
Root Cause
confirmation_id generation in HookManager.execute_pre_tool() is too sensitive to argument text:
confirmation_id_str = (
f"{hook_name}:{tool_name}:{json.dumps(current_tool_call.arguments, sort_keys=True)}"
)
confirmation_id = hashlib.sha256(confirmation_id_str.encode()).hexdigest()[:16]
sort_keys=True stabilizes top-level key order but does not account for:
- Nested object key ordering differences
- Whitespace or formatting differences in string argument values
- The LLM rephrasing free-text fields while preserving semantic intent
- Numeric representation drift (
1.0 vs 1, 100 vs 1e2)
Context
The internal-rag project hit this issue with Slack message confirmations and worked around it by replacing the built-in confirmation hook entirely with a custom PRE_TOOL hook that checks whether the user has
already accepted any confirmation in the current turn (has_accepted flag), bypassing the confirmation_id matching in HookManager altogether. The confirmation_id generation and matching logic in ragbits was
left unchanged because the workaround just avoids it.
Expected Behavior
Once a user confirms a tool execution, the agent should execute that tool without re-requesting confirmation, even if the LLM regenerates the call with cosmetically different arguments.
Proposed Approach
- Persist the original arguments from the
ConfirmationRequest into ChatContext.state["pending_confirmation"] so they survive the turn boundary.
- On the continuation turn, when
tool_confirmations contains confirmed=True and a pending confirmation exists, normalize the user message into an explicit continuation prompt that tells the LLM to call the
tool with the exact original arguments avoiding argument regeneration entirely.
- Scope confirmation acceptance to tool name + turn as a fallback, so that if the LLM still produces different arguments, the prior approval for the same tool is honored within the same turn.
- Keep the existing
confirmation_id mechanism for cases where multiple calls to the same tool in one turn need independent confirmation.
Problem
The built-in
create_confirmation_hook()fromragbits.agents.hooks.confirmationdoes not work reliably across conversation turns.When the hook sets
decision="ask", theHookManager.execute_pre_tool()method generates aconfirmation_idby hashinghook_name:tool_name:json.dumps(arguments, sort_keys=True). Within a single agent turn thisworks fine the hash is computed once and matched against itself. But in a human-in-the-loop scenario, where the agent pauses, yields a
ConfirmationRequest, and resumes on the next turn after the user confirms, theLLM regenerates the tool call from scratch. The new arguments are semantically identical but textually different (reordered nested keys, whitespace changes, float
1.0vs1, rephrased free-text fields), so thefreshly computed hash no longer matches the
confirmation_idthe user approved.This causes either:
Root Cause
confirmation_idgeneration inHookManager.execute_pre_tool()is too sensitive to argument text:sort_keys=Truestabilizes top-level key order but does not account for:1.0vs1,100vs1e2)Context
The
internal-ragproject hit this issue with Slack message confirmations and worked around it by replacing the built-in confirmation hook entirely with a customPRE_TOOLhook that checks whether the user hasalready accepted any confirmation in the current turn (
has_acceptedflag), bypassing theconfirmation_idmatching inHookManageraltogether. Theconfirmation_idgeneration and matching logic in ragbits wasleft unchanged because the workaround just avoids it.
Expected Behavior
Once a user confirms a tool execution, the agent should execute that tool without re-requesting confirmation, even if the LLM regenerates the call with cosmetically different arguments.
Proposed Approach
ConfirmationRequestintoChatContext.state["pending_confirmation"]so they survive the turn boundary.tool_confirmationscontainsconfirmed=Trueand a pending confirmation exists, normalize the user message into an explicit continuation prompt that tells the LLM to call thetool with the exact original arguments avoiding argument regeneration entirely.
confirmation_idmechanism for cases where multiple calls to the same tool in one turn need independent confirmation.