Add Virtual MCP Server proposal

[JAORMX]

Summary

This PR introduces a proposal for the Virtual MCP Server, a new ToolHive feature that aggregates multiple MCP servers from a group into a single unified interface.

What does this PR do?

Adds a technical proposal document that outlines the design and implementation approach for the Virtual MCP Server (thv virtual command).

Key Design Decisions

• Group-based architecture: Leverages existing ToolHive groups as the fundamental abstraction for managing backend MCP servers
• Reuses existing constructs: Uses ToolsFilter for including specific tools and ToolOverride for renaming/redescribing tools
• Per-backend authentication: Supports different authentication strategies per backend (pass-through, token exchange, service accounts, header injection)
• Phased implementation: MVP focuses on basic aggregation, with composite tools and advanced features in later phases

Benefits

• Simplifies client integration (single connection instead of multiple)
• Centralizes authentication complexity handling
• Enables cross-service workflows via composite tools
• Maintains full backward compatibility with existing MCP clients and servers

Testing

This is a proposal document - implementation and testing will follow in subsequent PRs.

Related Issues

This proposal addresses the need for consolidating multiple MCP servers as discussed in the Virtual MCP design document.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 48.64%. Comparing base (2d6a56f) to head (13e4f9d).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2106      +/-   ##
==========================================
- Coverage   48.71%   48.64%   -0.07%     
==========================================
  Files         241      241              
  Lines       30592    30592              
==========================================
- Hits        14902    14883      -19     
- Misses      14567    14589      +22     
+ Partials     1123     1120       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[JAORMX]

@claude please review this

[claude]

Claude finished @JAORMX's task —— View job

---

Code Review: Virtual MCP Server Proposal

I've reviewed the Virtual MCP Server proposal. This is a well-thought-out design that appropriately leverages existing ToolHive constructs. Here's my detailed feedback:

---

Strengths

1. Excellent reuse of existing patterns ✓

• Leverages groups.Manager and workloads.Manager (docs/proposals/virtual-mcp-server.md:121-124)
• Reuses ToolsFilter and ToolOverride middleware (docs/proposals/virtual-mcp-server.md:136-138, 218-219)
• These are battle-tested components already in production

2. Clear phased implementation ✓

• MVP focuses on core aggregation (docs/proposals/virtual-mcp-server.md:186-196)
• Composite tools deferred to Phase 2 is pragmatic
• Allows incremental delivery and validation

3. Good alignment with MCP specification ✓

• Maintains protocol compatibility (docs/proposals/virtual-mcp-server.md:13)
• Proper handling of tools, resources, and prompts (docs/proposals/virtual-mcp-server.md:128-132)

---

Issues & Recommendations

1. Authentication design needs clarification ⚠️

Issue: The proposal mentions token exchange (docs/proposals/virtual-mcp-server.md:66-68, 144) but there's already a separate token exchange proposal. The relationship between these two features isn't clear.

Questions:

• Will Virtual MCP reuse the token exchange middleware from PR Design proposal: Token exchange to acquire tokens for external auth #2063?
• How does per-backend auth interact with the existing OIDC/OAuth infrastructure?
• Should backend_auth configuration reference the token exchange proposal?

Recommendation: Add a section explicitly stating:

# References token exchange from docs/proposals/token-exchange-middleware.md
backend_auth:
  github:
    type: "token_exchange"
    # Uses existing token exchange middleware implementation

2. Group reference validation ⚠️

Issue: The proposal doesn't specify what happens if the referenced group doesn't exist or is empty (docs/proposals/virtual-mcp-server.md:38).

Recommendation: Add validation behavior:

• Return clear error if group doesn't exist
• Handle case where group has no running workloads
• Specify whether virtual MCP should start automatically when group workloads start

3. Tool name conflict resolution needs more detail ⚠️

Issue: The conflict_resolution field mentions "prefix | priority | manual" (docs/proposals/virtual-mcp-server.md:42) but doesn't explain:

• What prefix format is used (workload name? custom prefix?)
• How priority order is determined
• What "manual" means (presumably the override section?)

Recommendation: Expand with examples:

aggregation:
  conflict_resolution: "prefix"  # Adds workload name as prefix: github_create_pr
  # OR
  conflict_resolution: "priority"  # First workload wins, others ignored
  priority_order: ["github", "jira", "slack"]

4. Missing error handling specifications ⚠️

Open questions from docs/proposals/virtual-mcp-server.md:240-244:

1. ✓ Streaming responses - acknowledged as open question
2. ✓ Backend capabilities caching - acknowledged as open question
3. ✓ Rate limits - acknowledged as open question
4. Missing: What happens when a backend workload fails/stops?
5. Missing: Partial failure handling (some backends succeed, others fail)
6. Missing: Circuit breaker patterns for unhealthy backends

Recommendation: Add section on failure modes and degradation strategies.

5. CLI command naming 💭

Minor consideration: The command thv virtual (docs/proposals/virtual-mcp-server.md:169) could be more descriptive. Consider:

• thv virtual-server - clearer that it's running a server
• thv aggregate - describes the function
• thv virtual is fine if brevity is preferred

6. Composite tools implementation complexity ⚠️

Concern: The composite tools feature (docs/proposals/virtual-mcp-server.md:91-104) is Phase 2 but may be architecturally complex:

• Requires DAG execution for depends_on
• Needs variable substitution ({{.params.pr_number}})
• Error handling across multiple tool calls
• Transaction semantics (what if step 2 fails after step 1 succeeds?)

Recommendation: Consider whether composite tools belong in a separate proposal, as they're almost a workflow engine feature rather than an aggregation feature.

7. Security considerations 🔒

Missing: Security implications of aggregation:

• Can workloads in the same group belong to different security contexts?
• Should there be authorization checks per backend in addition to virtual MCP auth?
• How are secrets scoped across aggregated backends?

Recommendation: Add explicit security boundary documentation.

8. Performance metrics and observability 📊

Success criteria (docs/proposals/virtual-mcp-server.md:246-252) includes "< 10ms routing overhead" but:

• No mention of how this will be measured
• No observability/telemetry strategy for debugging multi-backend issues
• No mention of distributed tracing across backends

Recommendation: Add observability requirements to implementation notes.

---

Minor Issues

9. Typo/Grammar

Line 238: "LLM-generated backend" - this alternative seems like a different feature entirely and may confuse readers. Consider removing or clarifying.

10. Implementation notes location

The "Reusing Existing Components" section (docs/proposals/virtual-mcp-server.md:214-223) is excellent but buried at the end. Consider moving it closer to "High-Level Design" since it explains how the pieces fit together.

---

Questions for Discussion

1. Workload lifecycle: Should the virtual MCP automatically start/stop underlying workloads, or only work with already-running workloads?

2. Dynamic membership: If workloads are added/removed from the group while virtual MCP is running, should it automatically update? (Mentioned as Phase 3, line 199)

3. Client registration: Should the virtual MCP be registered with MCP clients separately from its backend workloads?

4. Backwards compatibility: Can existing workloads be added to groups without modification, or do they need reconfiguration?

---

Overall Assessment

Verdict: ✅ Approve with minor revisions

This is a solid proposal that appropriately reuses existing ToolHive infrastructure. The phased approach is sensible, and the core aggregation feature is well-scoped. The main gaps are around error handling, security boundaries, and clarifying the relationship with the token exchange proposal.

Priority fixes before implementation:

1. Clarify authentication strategy and token exchange integration (Issue fix(typo): corrects readme #1)
2. Specify failure handling and degradation behavior (Issue Implement secret store #4)
3. Expand conflict resolution details (Issue Bump golangci/golangci-lint-action from 2f856675483cb8b9378ee77ee0beb67955aca9d7 to 4696ba8babb6127d732c3c6dde519db15edab9ea #3)

Nice-to-haves:

• Security considerations section (Issue Create proxy subcommand #7)
• Observability strategy (Issue Start simple hardcoded registry #8)
• Consider splitting composite tools into separate proposal (Issue Figure out ergonomics for exposing directories #6)

---

Job Run | Branch: proposal/virtual-mcp-server

[Copilot]

Pull Request Overview

This PR introduces a technical proposal for a Virtual MCP Server feature that consolidates multiple MCP servers from a ToolHive group into a single unified interface. The proposal aims to simplify client integration and centralize authentication complexity while maintaining backward compatibility.

• Adds comprehensive design documentation for the Virtual MCP Server (thv virtual command)
• Defines group-based architecture leveraging existing ToolHive constructs
• Outlines phased implementation approach from basic aggregation to enterprise features

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[ChrisJBurns]

Should there be authorization checks per backend in addition to virtual MCP auth?

@JAORMX This part of Claudes review is what I was getting at in the vMCP GDoc. I'll explain more here just so it's visible to others.

If the benefit of vMCP itself is that it can aggregate 10 backing MCP servers, that's great, but let's think of the security implications / risks of this. In this case, a single vMCP server will have access to different 10 MCP servers. In the event that vMCP is ever exploited, would it be able to make calls to the backing MCP servers directly? Or will they themselves be protected by authn/z? If so, then this isn't as much of a concern from me.

[JAORMX]

If the benefit of vMCP itself is that it can aggregate 10 backing MCP servers, that's great, but let's think of the security implications / risks of this. In this case, a single vMCP server will have access to different 10 MCP servers. In the event that vMCP is ever exploited, would it be able to make calls to the backing MCP servers directly? Or will they themselves be protected by authn/z? If so, then this isn't as much of a concern from me.

I see. So... we should clarify the architecture and deployment recommendations then. backed MCP servers should still not be deployed as they are without appropriate protections. That is, they should still enforce authentication and perhaps even authorization. If the vMCP is ever exploited, the backed MCP servers will still authenticate and authorize the request. I'm more keen on a zero-trust policy in general.

[ChrisJBurns]

That is, they should still enforce authentication and perhaps even authorization. If the vMCP is ever exploited, the backed MCP servers will still authenticate and authorize the request. I'm more keen on a zero-trust policy in general.

@JAORMX Yep, that would be my understanding (and hope) too! I had a feeling you thought the same way but I just wanted to triple check 😄

[JAORMX]

@ChrisJBurns picture me with a tinfoil hat on at any given moment 😄

[jhrozek]

Should there be authorization checks per backend in addition to virtual MCP auth?

@JAORMX This part of Claudes review is what I was getting at in the vMCP GDoc. I'll explain more here just so it's visible to others.

If the benefit of vMCP itself is that it can aggregate 10 backing MCP servers, that's great, but let's think of the security implications / risks of this. In this case, a single vMCP server will have access to different 10 MCP servers. In the event that vMCP is ever exploited, would it be able to make calls to the backing MCP servers directly? Or will they themselves be protected by authn/z? If so, then this isn't as much of a concern from me.

Hmm, but the vMCP server would be an OAuth app so the user would connect with a token that says aud=vmcp right? And then the vMCP server would have to authenticate to each of the back end servers which means sending a token with aud=server1, aud=server2 if we are to reuse the usual authnz for the back end MCP servers. We could do that provided that the token exchange is set up to allow that. However, we'd have to send the external API token along with the request -or- let the back end token servers exchange the tokens they receive for external API tokens.

That would work if we can use token exchange all the way through, but not if we need to drive a consent and do the full flow (we would want to drive the consent chain to the consent on the vMCP server correct)? So in that case we'd have to send the back end server through a back channel?