RFC: Session-Scoped Architecture for vMCP by yrobla · Pull Request #38 · stacklok/toolhive-rfcs

yrobla · 2026-02-04T13:31:46Z

RFC: Session-Scoped Architecture for vMCP

This RFC refactors vMCP session management to fix per-request client creation and scattered session concerns.

Problem

Per-request overhead: Backend clients created/closed on every tool call, causing connection overhead and state loss for stateful
backends (Playwright, databases)
Scattered architecture: Session creation tangled across middleware, adapters, and SDK hooks with no clear ownership
Hard to extend: Current design makes it difficult to add features (optimizer-in-vmcp) or write integration tests without spinning up
the full server

Solution

Introduce two core interfaces:

Session - Domain object that owns backend clients, encapsulates routing logic, manages lifecycle
SessionFactory - Creates fully-formed sessions (capability discovery + client initialization)
SessionManager - Bridges domain logic to SDK protocol concerns

Key improvement: Clients created once during session init, reused for all requests, closed on expiration.

Copilot

Pull request overview

Adds an RFC proposing a shift from per-request/lazy MCP backend client creation to session-scoped client lifecycle management to simplify the current pooling approach while preserving backend state across tool calls.

Changes:

Introduces a new RFC describing a session-scoped backend client map owned by VMCPSession.
Specifies eager client initialization during session setup (AfterInitialize) and cleanup during session close.
Outlines phased implementation steps, testing strategy, and alternatives considered.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

rfcs/THV-0034-session-scoped-client-lifecycle.md

The RFC captures the complete architectural vision from the discussion, providing a roadmap for simplifying the client pooling implementation while maintaining all current functionality.

Copilot

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 5 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

rfcs/THV-0038-session-scoped-client-lifecycle.md

rfcs/THV-0034-session-scoped-client-lifecycle.md

rfcs/THV-0038-session-scoped-client-lifecycle.md

rfcs/THV-0034-session-scoped-client-lifecycle.md

Copilot

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

rfcs/THV-0038-session-scoped-client-lifecycle.md

Copilot

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

rfcs/THV-0038-session-scoped-client-lifecycle.md

jerm-dro

I left two bigger comments elaborating on my thoughts. To summarize, I think there are larger problems with how the session concept is handled within vMCP. Namely, we have session concerns scattered throughout the codebase and all of these concerns are tightly coupled. As a solution, I think we should create an interface that encapsulates these concerns:

// Session represents an active MCP session with all its capabilities and resources.
// This is a pure domain object - no protocol concerns and can be run without being spun up in a server.
type Session interface {
    ID() string

    // Capabilities - returns discovered tools/resources for this session
    // Perhaps these could be combined into one "GetCapabilities" method.
    Tools() []Tool
    Resources() []Resource

    // MCP operations - routing logic is encapsulated here
    CallTool(ctx context.Context, name string, arguments map[string]any) (*ToolResult, error)
    ReadResource(ctx context.Context, uri string) (*ResourceResult, error)
    GetPrompt(ctx context.Context, name string, arguments map[string]any) (*PromptResult, error)

    // Lifecycle
    Close() error
}

That would make solving this client-lifetime problem trivial.

rfcs/THV-0038-session-scoped-client-lifecycle.md

Address architectural feedback from PR #38 review, specifically jerm-dro's comments about the root cause being scattered session concerns rather than just client lifecycle management. Major changes: 1. **Expanded Problem Statement**: - Identified root cause: session concerns scattered across middleware, adapters, and hooks - Current VMCPSession is a passive data container, not a domain object - Session construction tangled with SDK lifecycle callbacks - Hard to create session-scoped resources (clients, optimizer features) 2. **Introduced Core Interfaces**: - Session interface: Domain object that owns clients, encapsulates routing, manages lifecycle (not just a data container) - SessionFactory interface: Creates fully-formed sessions with all dependencies (discovery, client init, routing setup) - SessionManager: Bridges between domain (Session) and protocol (SDK) 3. **Updated Goals**: - Primary goal: Encapsulate session behavior in clean interfaces - Decouple session creation from SDK wiring - Enable testing without full server (addresses toolhive#2852) - Enable future features through interface decoration 4. **Revised Implementation Plan**: - Phase 1: Introduce Session/SessionFactory interfaces - Phase 2: Wire SessionManager to SDK - Phase 3: Migrate existing code to use Session interface - Phase 4: Cleanup and observability 5. **Added Benefits Documentation**: - Concrete examples of decorator pattern (optimizer-in-vmcp) - Simplified capability refresh approach - Clear testing strategy without server dependency 6. **Fixed Formatting Issues** (from Copilot review): - Split Goals & Non-Goals into separate sections - Changed "Security" to "Security Considerations" - Restructured Compatibility section - Fixed References to use full repo-qualified format - Added credential refresh considerations The RFC now addresses the architectural root cause (scattered session concerns) rather than just the symptom (per-request client creation). Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Refactor RFC to address scattered session architecture concerns Address architectural feedback from PR #38 review, specifically jerm-dro's comments about the root cause being scattered session concerns rather than just client lifecycle management. Major changes: 1. **Expanded Problem Statement**: - Identified root cause: session concerns scattered across middleware, adapters, and hooks - Current VMCPSession is a passive data container, not a domain object - Session construction tangled with SDK lifecycle callbacks - Hard to create session-scoped resources (clients, optimizer features) 2. **Introduced Core Interfaces**: - Session interface: Domain object that owns clients, encapsulates routing, manages lifecycle (not just a data container) - SessionFactory interface: Creates fully-formed sessions with all dependencies (discovery, client init, routing setup) - SessionManager: Bridges between domain (Session) and protocol (SDK) 3. **Updated Goals**: - Primary goal: Encapsulate session behavior in clean interfaces - Decouple session creation from SDK wiring - Enable testing without full server (addresses toolhive#2852) - Enable future features through interface decoration 4. **Revised Implementation Plan**: - Phase 1: Introduce Session/SessionFactory interfaces - Phase 2: Wire SessionManager to SDK - Phase 3: Migrate existing code to use Session interface - Phase 4: Cleanup and observability 5. **Added Benefits Documentation**: - Concrete examples of decorator pattern (optimizer-in-vmcp) - Simplified capability refresh approach - Clear testing strategy without server dependency 6. **Fixed Formatting Issues** (from Copilot review): - Split Goals & Non-Goals into separate sections - Changed "Security" to "Security Considerations" - Restructured Compatibility section - Fixed References to use full repo-qualified format - Added credential refresh considerations The RFC now addresses the architectural root cause (scattered session concerns) rather than just the symptom (per-request client creation). Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Refactor RFC to address scattered session architecture concerns Address architectural feedback from PR #38 review, specifically jerm-dro's comments about the root cause being scattered session concerns rather than just client lifecycle management. Major changes: 1. **Expanded Problem Statement**: - Identified root cause: session concerns scattered across middleware, adapters, and hooks - Current VMCPSession is a passive data container, not a domain object - Session construction tangled with SDK lifecycle callbacks - Hard to create session-scoped resources (clients, optimizer features) 2. **Introduced Core Interfaces**: - Session interface: Domain object that owns clients, encapsulates routing, manages lifecycle (not just a data container) - SessionFactory interface: Creates fully-formed sessions with all dependencies (discovery, client init, routing setup) - SessionManager: Bridges between domain (Session) and protocol (SDK) 3. **Updated Goals**: - Primary goal: Encapsulate session behavior in clean interfaces - Decouple session creation from SDK wiring - Enable testing without full server (addresses toolhive#2852) - Enable future features through interface decoration 4. **Revised Implementation Plan**: - Phase 1: Introduce Session/SessionFactory interfaces - Phase 2: Wire SessionManager to SDK - Phase 3: Migrate existing code to use Session interface - Phase 4: Cleanup and observability 5. **Added Benefits Documentation**: - Concrete examples of decorator pattern (optimizer-in-vmcp) - Simplified capability refresh approach - Clear testing strategy without server dependency 6. **Fixed Formatting Issues** (from Copilot review): - Split Goals & Non-Goals into separate sections - Changed "Security" to "Security Considerations" - Restructured Compatibility section - Fixed References to use full repo-qualified format - Added credential refresh considerations The RFC now addresses the architectural root cause (scattered session concerns) rather than just the symptom (per-request client creation). Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> Update Last Updated date to 2026-02-09

jerm-dro

The changes motivated by others' feedback LGTM.

yrobla · 2026-02-16T14:46:30Z

Different advices collected:

one different session per vmcp per client, how it will affect the current RFC - that's a problem with the transport and single endpoint
sessions in one mcp can terminate before others
investigate if we could replace this with Stateful Streamable HTTP, and will be the implementation that is needed. Can we assume that all MCP servers support it?
we can create a test suite to validate that on multiple clients - but this should also be better done at MCP level, and we could reuse for vMCP

Copilot

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

rfcs/THV-0038-session-scoped-client-lifecycle.md

Copilot

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 5 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

rfcs/THV-0038-session-scoped-client-lifecycle.md

Copilot

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 13 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

rfcs/THV-0038-session-scoped-client-lifecycle.md

yrobla requested a review from Copilot February 4, 2026 13:31

Copilot started reviewing on behalf of yrobla February 4, 2026 13:32 View session

Copilot AI reviewed Feb 4, 2026

View reviewed changes

yrobla force-pushed the session-scoped-mcp-client branch from f935166 to 6e503d4 Compare February 4, 2026 13:40

Add RFC: Session-scoped MCP client lifecycle management

4f970e6

The RFC captures the complete architectural vision from the discussion, providing a roadmap for simplifying the client pooling implementation while maintaining all current functionality.

yrobla force-pushed the session-scoped-mcp-client branch from 6e503d4 to 4f970e6 Compare February 4, 2026 13:46

yrobla requested a review from Copilot February 4, 2026 13:49

Copilot started reviewing on behalf of yrobla February 4, 2026 13:49 View session

Copilot AI reviewed Feb 4, 2026

View reviewed changes

yrobla force-pushed the session-scoped-mcp-client branch 3 times, most recently from 840b512 to ddf4273 Compare February 4, 2026 15:26

yrobla requested a review from Copilot February 4, 2026 15:26

Copilot started reviewing on behalf of yrobla February 4, 2026 15:26 View session

Copilot AI reviewed Feb 4, 2026

View reviewed changes

yrobla force-pushed the session-scoped-mcp-client branch 2 times, most recently from 50f4742 to ec70d08 Compare February 4, 2026 15:51

yrobla requested a review from Copilot February 4, 2026 15:52

Copilot started reviewing on behalf of yrobla February 4, 2026 15:52 View session

Copilot AI reviewed Feb 4, 2026

View reviewed changes

rfcs/THV-0038-session-scoped-client-lifecycle.md Outdated Show resolved Hide resolved

rfcs/THV-0038-session-scoped-client-lifecycle.md Outdated Show resolved Hide resolved

rfcs/THV-0038-session-scoped-client-lifecycle.md Outdated Show resolved Hide resolved

yrobla force-pushed the session-scoped-mcp-client branch 2 times, most recently from 54dd1b4 to d714819 Compare February 4, 2026 16:28

jerm-dro reviewed Feb 4, 2026

View reviewed changes

rfcs/THV-0038-session-scoped-client-lifecycle.md Show resolved Hide resolved

This was referenced Feb 5, 2026

vMCP: Implement backend session persistence with client pooling stacklok/toolhive#3062

Closed

Create an RFC for vMCP session management architecture improvements #39

Closed

jerm-dro reviewed Feb 6, 2026

View reviewed changes

rfcs/THV-0038-session-scoped-client-lifecycle.md Show resolved Hide resolved

yrobla force-pushed the session-scoped-mcp-client branch from d698c3e to 9654f19 Compare February 9, 2026 15:16

yrobla force-pushed the session-scoped-mcp-client branch from 6bcdb92 to 1f73b30 Compare February 13, 2026 12:51

yrobla mentioned this pull request Feb 13, 2026

Per-session prompt tracking in vMCP stacklok/toolhive#2841

Open

jerm-dro previously approved these changes Feb 13, 2026

View reviewed changes

JAORMX previously approved these changes Feb 16, 2026

View reviewed changes

yrobla dismissed stale reviews from JAORMX and jerm-dro via 3914771 February 16, 2026 15:22

yrobla requested review from JAORMX, Copilot and jerm-dro February 16, 2026 15:22

Copilot started reviewing on behalf of yrobla February 16, 2026 15:23 View session

Copilot AI reviewed Feb 16, 2026

View reviewed changes

yrobla force-pushed the session-scoped-mcp-client branch from 3914771 to a415e4f Compare February 16, 2026 15:47

yrobla requested review from blkt and Copilot February 16, 2026 15:47

Copilot started reviewing on behalf of yrobla February 16, 2026 15:51 View session

Copilot AI reviewed Feb 16, 2026

View reviewed changes

yrobla force-pushed the session-scoped-mcp-client branch from a415e4f to 0682b34 Compare February 16, 2026 16:06

amirejaz reviewed Feb 17, 2026

View reviewed changes

rfcs/THV-0038-session-scoped-client-lifecycle.md Show resolved Hide resolved

changes from review meeting

0ca7b5a

yrobla force-pushed the session-scoped-mcp-client branch from 0682b34 to 0ca7b5a Compare February 17, 2026 15:56

yrobla requested review from amirejaz and Copilot and removed request for Copilot February 17, 2026 16:13

Copilot started reviewing on behalf of yrobla February 17, 2026 16:14 View session

Copilot AI reviewed Feb 17, 2026

View reviewed changes

jerm-dro approved these changes Feb 17, 2026

View reviewed changes

rfcs/THV-0038-session-scoped-client-lifecycle.md Show resolved Hide resolved

rfcs/THV-0038-session-scoped-client-lifecycle.md Show resolved Hide resolved

JAORMX approved these changes Feb 18, 2026

View reviewed changes

yrobla merged commit 127fe09 into main Feb 18, 2026
7 checks passed

Conversation

yrobla commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jerm-dro left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jerm-dro left a comment

Choose a reason for hiding this comment

Uh oh!

yrobla commented Feb 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yrobla commented Feb 4, 2026 •

edited

Loading

yrobla commented Feb 16, 2026 •

edited

Loading