Skip to content

fix: MCP tool confirmation lifecycle, vault token resolution, and dialect-portable SQL - #9

Open
pandacooming wants to merge 3 commits into
rogeriochaves:mainfrom
pandacooming:fix/mcp-tool-confirmation-vault-credentials
Open

fix: MCP tool confirmation lifecycle, vault token resolution, and dialect-portable SQL#9
pandacooming wants to merge 3 commits into
rogeriochaves:mainfrom
pandacooming:fix/mcp-tool-confirmation-vault-credentials

Conversation

@pandacooming

@pandacooming pandacooming commented Apr 11, 2026

Copy link
Copy Markdown

Summary

Follow-up to PR #8, addressing the review feedback: the MCP tool confirmation flow and vault token resolution are now fully wired into the engine, with dialect-portable SQL and matching test coverage.

What was left on the table in #8, and what this PR does about it

Feedback What this PR does
AgentConfig doesn't declare vault_ids; resolveTools() never reads it Added vault_ids?: string[] to AgentConfig; resolveTools() passes it to credential resolution
json_replace is SQLite-only; Postgres CI job would break Replaced with read → JSON.parse → modify → JSON.stringify — works on both SQLite and Postgres
Engine doesn't consume evaluated_permission Engine skips MCP tool calls where evaluated_permission === "pending"; re-execution loop also skips any tool that already has a tool_result with matching tool_use_id
deny branch needs matching engine logic deny injects agent.tool_result; engine's re-execution loop detects it via existingResultRow full-scan and skips
mcp/ module is unwired — don't merge parallel implementations mcp/executor (resolveMCPTools + executeMCPTool) is now wired into the engine loop; mcp_toolset agent entries use mcp_<server>_<tool> naming

Changes

1. mcp/ module wired into engine loop (engine/index.ts, new files)

The mcp/ module (SSE transport + per-session registry + vault-based credential resolution + executor) from the previous PR is now called by the engine:

  • resolveMCPTools resolves tools for mcp_toolset agent config entries
  • executeMCPTool executes mcp_<server>_<tool> calls with session-scoped MCP connections
  • Coexists with legacy __mcp__<connector>__<tool> routing (StreamableHTTP via lib/mcp-client.ts)

New files: mcp/client.ts, mcp/executor.ts, mcp/registry.ts, mcp/types.ts, mcp/index.ts (1087 lines)

2. Vault token resolution (engine/index.ts)

  • AgentConfig now includes vault_ids?: string[]
  • resolveTools() passes vaultIds to credential resolution; MCP tokens are fetched from vault via resolveMCPCredential before falling back to the mcp_connections table

3. Tool confirmation lifecycle (engine/index.ts, routes/events.ts)

  • All MCP tools (__mcp__ and mcp_ prefixed) set evaluated_permission=pending in agent.tool_use events and go idle
  • buildMessagesFromEvents() skips pending tool_use entries so the LLM doesn't see them in history
  • allow: patches evaluated_permission from pendingallow using dialect-portable SQL; query finds the event by tool_use_id in the JSON data field (not by event id)
  • deny: injects agent.tool_result {is_error: true} with deny_message; engine skips re-execution when it detects an existing result for that tool_use_id
  • storeEvent and buildMessagesFromEvents exported from engine/index.ts for test usage

4. Confirmation integration tests (sessions.test.ts) — updated

Replaced 2 stop-session tests with 5 new confirmation lifecycle tests:

  • Pending agent.tool_use event is stored correctly
  • allow confirmation updates evaluated_permission to allow
  • deny with custom message injects the correct error content
  • deny with no message uses the default denial text
  • Unknown tool_use_id is a silent no-op (200, not 404)

5. mcp/executor.ts unit tests (mcp-executor.test.ts, new)

20 tests covering:

  • isMCPTool / getMCPServerName / mcpToolRequiresConfirmation (pure functions)
  • resolveMCPTools with mocked registry: server connection, tool prefixing, default/toolset perms, error fallback, uninitialized skip
  • executeMCPTool with mocked registry: correct routing, invalid format, disconnected server, uninitialized server, error propagation

6. SSE MCP round-trip integration test (mcp-sse-roundtrip.test.ts, new)

7 tests proving the mcp/client.ts SSE MCP client works against a real in-process HTTP server:

  • createMCPServerConnection + discoverMCPTools returns mcp_<server>_... prefixed tools
  • callMCPTool executes echo / add / uppercase via SSE and returns correct results
  • Unknown tool returns is_error: true
  • Sequential calls on the same SSE connection all succeed
  • Invalid tool name format is rejected

Includes a bugfix: startSSEListener was await'd inside createMCPServerConnection, but SSE responses are long-lived and never "complete", causing the call to deadlock indefinitely. Changed to fire-and-forget with a .catch() error handler.

7. buildMessagesFromEvents skip-pending unit tests (engine-build-messages.test.ts, new)

4 tests proving the skip logic:

  • evaluated_permission=pending entries are excluded from LLM history
  • evaluated_permission=allow entries are included
  • Mixed pending + allowed: only allowed tools appear
  • agent.tool_result (denial) is always included regardless of permission state

Test results

sessions.test.ts (16 tests)           ← +5 confirmation tests, −2 stop-session tests
mcp-executor.test.ts (20 tests)       ← new
mcp-sse-roundtrip.test.ts (7 tests)    ← new
engine-build-messages.test.ts (4 tests)← new
engine-mcp-tools.test.ts (7 tests)      ← existing, updated for new MCP module
mcp-org-isolation.test.ts (5 tests)    ← existing
provider-access.test.ts (9 tests)      ← existing
environments-mcp.test.ts (13 tests)    ← existing

Test Files: 24 passed (24)
Tests: 194 passed (194)

maxyangcn added 3 commits April 11, 2026 03:31
Adds a higher-level MCP module (mcp/) that provides:
- SSE transport client for persistent MCP server connections
- Per-session registry for connection management
- Vault-based credential resolution
- Executor with tool resolution and confirmation support

Also includes:
- events.ts: user.tool_confirmation handler (approve/deny)

Note: The engine currently uses lib/mcp-client.ts (StreamableHTTP)
for MCP integration. This module is available as an alternative
implementation and can be integrated into the engine loop in a
follow-up PR.

Co-authored-by: Claude
…lect-portable SQL

- Add vault_ids to AgentConfig; resolveTools() looks up MCP tokens
  from vault (via resolveMCPCredential) before falling back to
  mcp_connections table

- MCP tools (both __mcp__ and mcp_ prefixed) now go idle with
  evaluated_permission=pending until user sends
  user.tool_confirmation {result: allow|deny}

- buildMessagesFromEvents() skips pending tool_use entries so
  the LLM won't see them in the message history

- allow confirmation: patch evaluated_permission from pending→allow
  using dialect-portable SQL (read → JSON.parse → modify → write;
  avoids json_replace which is SQLite-only). Query by tool_use_id
  in the JSON data field, not by event id.

- deny confirmation: inject agent.tool_result {is_error: true}
  with user's deny_message (or default). Engine skips re-execution
  when it sees an existing tool_result for the same tool_use_id

- Export storeEvent from engine/index.ts for test usage

- Add integration tests covering: pending event storage,
  allow confirmation, deny with custom message, deny with default
  message, and graceful no-op for unknown tool_use_id

- Wire resolveMCPTools/executeMCPTool from mcp/executor into
  engine loop: mcp_toolset agent tools use mcp_<server>_<tool>
  prefix, coexists with legacy __mcp__<connector>__<tool>
  approach
…ages skip-pending tests

- mcp/executor.test.ts: 20 tests covering pure functions (isMCPTool,
  getMCPServerName, mcpToolRequiresConfirmation), resolveMCPTools (mocked
  registry), and executeMCPTool (mocked registry + callMCPTool)

- mcp-sse-roundtrip.test.ts: 7 integration tests proving the SSE MCP
  client (createMCPServerConnection + discoverMCPTools + callMCPTool)
  works end-to-end against a real in-process HTTP server that speaks
  the SSE+JSON-RPC MCP protocol

- engine-build-messages.test.ts: 4 tests for buildMessagesFromEvents
  skip-pending logic — pending tool_use entries are excluded from the
  LLM message history while allowed/denied ones are included

Bugfix (client.ts): startSSEListener was await'ed inside
createMCPServerConnection, but SSE responses are long-lived and never
"complete", causing the connection to deadlock indefinitely.
Changed to fire-and-forget with a .catch() error handler.

Also exports buildMessagesFromEvents from engine/index.ts so it can be
unit-tested directly.
@pandacooming
pandacooming force-pushed the fix/mcp-tool-confirmation-vault-credentials branch from 16bfc8c to 672f7d1 Compare April 11, 2026 03:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant