feat(ai-gateway): upstream mcp servers#5245
Open
tomek-labuk wants to merge 2 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds documentation for an AI MCP Proxy upstream-server mode and introduces a new MCP how-to describing how to aggregate tools using upstream-server + listener mode, extending the existing aggregation guidance for AI Gateway users.
Changes:
- Updates the AI MCP Proxy plugin “Configuration modes” section and ACL inheritance note to reference
upstream-server. - Adds a new how-to page for aggregating MCP tools using
upstream-servermode (min Gateway 3.14).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| app/_kong_plugins/ai-mcp-proxy/index.md | Extends the plugin mode matrix and ACL guidance to include upstream-server. |
| app/_how-tos/mcp/aggregate-mcp-tools-upstream-server.md | New end-to-end tutorial for tool aggregation using upstream-server + listener mode. |
Comments suppressed due to low confidence (2)
app/_kong_plugins/ai-mcp-proxy/index.md:260
- In the
upstream-servermode row, the description says it does not accept incoming MCP requests, but the use case says it provides a “single MCP endpoint”; please reword the use case to clarify that the endpoint is provided bylistenermode andupstream-serveronly supplies/aggregates tools for listeners.
description: |
Aggregates multiple upstream MCP servers but does **not** accept incoming MCP requests.
`tags` can be defined at the plugin level and are used by `listener` plugins to expose the tools.<br/><br/>
This mode must be used together with other AI MCP Proxy plugins configured with the `listener` mode.
usecase: |
Use when you need a single MCP endpoint that aggregates multiple backend tools.
app/_how-tos/mcp/aggregate-mcp-tools-upstream-server.md:222
- This section says the listener aggregates tools registered by “conversion-only” plugins, but this how-to configures the upstream instances in
upstream-servermode; update the text to avoid mode confusion.
## Configure the listener AI MCP Proxy plugin
Now, let's configure another AI MCP Proxy plugin instance in listener mode to aggregate and expose the tools registered by the conversion-only plugins. The listener plugin discovers tools based on their shared tag value—in this case, `mcp-tools`—and serves them through an MCP server that AI clients can connect to.
| Use when you need a single MCP endpoint that aggregates tools from multiple `conversion-only` plugins. | ||
| Typical in multi-service or multi-team environments that expose a unified MCP interface. | ||
| - mode: | | ||
| [`upstream-server`](./examples/pupstream-server/) {% new_in 3.14 %} |
| This mode must be used together with other AI MCP Proxy plugins configured with the `conversion-only` mode. | ||
| This mode must be used together with other AI MCP Proxy plugins configured with the `conversion-only` or `upstream-server` mode. | ||
| usecase: | | ||
| Use when you need a single MCP endpoint that aggregates tools from multiple `conversion-only` plugins. |
Comment on lines
+129
to
+131
| We configure the plugin in `upstream-server` mode because this instance only converts RESTful API paths into MCP tool definitions. It doesn’t handle incoming MCP requests directly. Later, we’ll aggregate these tools from multiple conversion-only instances using listener-mode plugins. | ||
|
|
||
| In this configuration we define `tags[]` at the plugin level because listener AI MCP Proxy plugin will use them to discover, aggregate, and expose the registered tools. |
Comment on lines
+57
to
+58
| curl -s -o api.js "https://gist.githubusercontent.com/subnetmarco/5ddb23876f9ce7165df17f9216f75cce/raw/a44a947d69e6f597465050cc595b6abf4db2fbea/api.js" | ||
| npm install express |
| You should see: | ||
|
|
||
| ```text | ||
| {"name":"Sample Users API"}% |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #issue
Preview Links
Checklist
descriptionentry in frontmatter.