[Frontend] Responses API MCP tools for built in tools and to pass through headers #24628

alecsolder · 2025-09-11T04:16:10Z

Purpose

This PR is to add the ability to use MCP tools to enable built in tools for gpt-oss. Either the OpenAI tool type (i.e browser) or an MCP tool with server_label set to the name of the OpenAI tool type.

This enables header passthrough from the request to the MCP server in addition to the session_id

Test Plan

Tested with responses API server with curl. Added one unit test which worked locally but can't run in CI which tests code_interpreter tool functionality.

curl -X POST "http://localhost:8000/v1/responses" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer dummy-api-key" \
  -d '{
        "model": "gpt-oss-20b",
        "input": "Multiply 64548*15151 using builtin python.",
        "tools": [
          {
            "type": "mcp",
            "server_label": "container", 
            "headers": {"test": "test"},
            "server_url": "IGNORED"
          }
        ]
      }'

Test Result

Interacting with the server worked locally and local unit test worked as well.

…h headers Signed-off-by: Alec Solder <[email protected]>

…tion issues Signed-off-by: Alec Solder <[email protected]>

Signed-off-by: Alec Solder <[email protected]>

heheda12345

Like this direction. And I think it is possible to setup this test in CI by using the python MCP server and PYTHON_EXECUTION_BACKEND=dangerously_use_uv. You can have a try if you don't want this feature be broken by future PRs.

heheda12345 · 2025-09-15T23:33:46Z

tests/entrypoints/openai/test_response_api_with_harmony.py

+async def test_mcp_tool(client: OpenAI, model_name: str):
+    response = await client.responses.create(
+        model=model_name,
+        input="Please multiply 123 and 456 using the available python tool.",


Can you use a complex test like the one in #24315 :

input=("What's the first 4 digits after the decimal point of " "cube root of `19910212 * 20250910`? " "Show only the digits."),

model can be very clever and skip using python tools.

I gave this a try and it did work, however because we don't respect limiting the output tokens when tools are enabled and max_tool_calls is not implemented yet, we can't stop the generation after just one tool call.

Because the dangerously_use_uv implementation is not jupyter, the model gets confused and keeps trying to make tool calls until it seems some output (it does eventually start printing), which causes the tests to time out in CI. Otherwise I think it would work just fine though.

@alecsolder let's open another issues to improve the tests and possibly bring in swe-rex / docker client in oss as we get time on this.

Signed-off-by: Alec Solder <[email protected]>

lacora · 2025-09-17T20:45:31Z

cc @yeqcharlotte @houseroad

yeqcharlotte · 2025-09-18T07:27:04Z

vllm/entrypoints/openai/serving_responses.py

+            # If there is a MCP tool with a matching server_label, then
+            # the tool is enabled. Native types like code_interpreter work
+            # as well
+            tool_types = [
+                tool.server_label if tool.type == "mcp" else tool.type
+                for tool in request.tools
+            ]


use mcp as built-in tool is a bit weird for most developers. do you think we should gate that with similar env like VLLM_GPT_OSS_USE_CONTAINER_TOOL. possibly we can modify the naming and use one env to control

yeqcharlotte · 2025-09-18T07:28:06Z

tests/entrypoints/openai/test_response_api_with_harmony.py

+@pytest.mark.asyncio
+@pytest.mark.parametrize("model_name", [MODEL_NAME])
+@pytest.mark.skip(reason="Code interpreter tool is not available in CI yet.")
+async def test_mcp_tool(client: OpenAI, model_name: str):


let's also add some unit tests coverage tool server init logic for mcp tool

yeqcharlotte

thanks for the change! let's also remember to update the recipe and make this easy to follow. main concern is to make this opt-in and more coverage in unit tests

…cp tools to enable built in tools Signed-off-by: Alec Solder <[email protected]>

vllm/envs.py

yeqcharlotte · 2025-09-19T21:38:07Z

mind also taking this vllm-project/recipes#43 and update it? feel free to open a new one explaining how to use it.

vllm/entrypoints/context.py

yeqcharlotte · 2025-09-19T21:47:03Z

thanks for pushing this, allow mcp tools to override built-in should be generic enough to enable more dev and research usage.

Signed-off-by: Alec Solder <[email protected]>

yeqcharlotte · 2025-09-20T01:04:51Z

vllm/envs.py

+        for val in values:
+            if not case_sensitive:
+                check_value = val.lower()
+                check_choices = [choice.lower() for choice in actual_choices]


this list doesn't have to be recreated every time

This is implemented in a lazy way in order to support valid values being a callable, which has to happen after other python importing logic is done. This means there isn't a great way to "cache" these values without writing a bigger class for this purpose, which I think it out of the scope for this PR

yeqcharlotte · 2025-09-20T01:07:45Z

vllm/envs.py

    return _get_validated_env


+def env_list_with_choices(


can we add some unit tests here. also @youkaichao do you have concerns to init a list from environment variables?

I did something similar to how it is done for plugins right now:

vllm/vllm/envs.py

Line 740 in 1d7f95b

lambda: None if "VLLM_PLUGINS" not in os.environ else os.environ[

mergify · 2025-09-22T19:43:16Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @alecsolder.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: Alec Solder <[email protected]>

Signed-off-by: Alec S <[email protected]>

yeqcharlotte

LGTM! Thanks for addressing all the feedback. cc: @heheda12345

…ough headers (vllm-project#24628) Signed-off-by: Alec Solder <[email protected]> Signed-off-by: Alec S <[email protected]> Co-authored-by: Alec Solder <[email protected]> Co-authored-by: Ye (Charlotte) Qi <[email protected]>

…ough headers (vllm-project#24628) Signed-off-by: Alec Solder <[email protected]> Signed-off-by: Alec S <[email protected]> Co-authored-by: Alec Solder <[email protected]> Co-authored-by: Ye (Charlotte) Qi <[email protected]> Signed-off-by: charlifu <[email protected]>

…ough headers (#24628) Signed-off-by: Alec Solder <[email protected]> Signed-off-by: Alec S <[email protected]> Co-authored-by: Alec Solder <[email protected]> Co-authored-by: Ye (Charlotte) Qi <[email protected]> Signed-off-by: yewentao256 <[email protected]>

…ough headers (vllm-project#24628) Signed-off-by: Alec Solder <[email protected]> Signed-off-by: Alec S <[email protected]> Co-authored-by: Alec Solder <[email protected]> Co-authored-by: Ye (Charlotte) Qi <[email protected]> Signed-off-by: gaojc <[email protected]>

…ough headers (vllm-project#24628) Signed-off-by: Alec Solder <[email protected]> Signed-off-by: Alec S <[email protected]> Co-authored-by: Alec Solder <[email protected]> Co-authored-by: Ye (Charlotte) Qi <[email protected]> Signed-off-by: xuebwang-amd <[email protected]>

…ough headers (vllm-project#24628) Signed-off-by: Alec Solder <[email protected]> Signed-off-by: Alec S <[email protected]> Co-authored-by: Alec Solder <[email protected]> Co-authored-by: Ye (Charlotte) Qi <[email protected]>

mergify bot added frontend gpt-oss Related to GPT-OSS models labels Sep 11, 2025

yeqcharlotte added this to gpt-oss Issues & Enhancements Sep 14, 2025

github-project-automation bot moved this to To Triage in gpt-oss Issues & Enhancements Sep 14, 2025

alecsolder force-pushed the builtin_mcp_tools_with_headers branch from 4f24d18 to fecbeaf Compare September 15, 2025 19:09

Alec Solder added 3 commits September 15, 2025 15:13

Adding the ability to use MCP tools for built in tools to pass throug…

e6ad3ae

…h headers Signed-off-by: Alec Solder <[email protected]>

Adding one unit test which works locally, fixing type to name transla…

69b7e04

…tion issues Signed-off-by: Alec Solder <[email protected]>

Removing log line

f0553be

Signed-off-by: Alec Solder <[email protected]>

alecsolder force-pushed the builtin_mcp_tools_with_headers branch from cd2c876 to f0553be Compare September 15, 2025 19:13

alecsolder marked this pull request as ready for review September 15, 2025 19:14

alecsolder requested review from DarkLight1337, NickLucche, aarnphm, chaunceyjiang, robertgshaw2-redhat and simon-mo as code owners September 15, 2025 19:14

heheda12345 reviewed Sep 15, 2025

View reviewed changes

Alec Solder and others added 3 commits September 16, 2025 12:52

Enabling oss coding tool for tests

94162b0

Signed-off-by: Alec Solder <[email protected]>

Merge branch 'main' into builtin_mcp_tools_with_headers

103f6d9

adding the skip back

8bc171d

Signed-off-by: Alec Solder <[email protected]>

yeqcharlotte reviewed Sep 18, 2025

View reviewed changes

yeqcharlotte requested changes Sep 18, 2025

View reviewed changes

github-project-automation bot moved this from To Triage to In progress in gpt-oss Issues & Enhancements Sep 18, 2025

Alec Solder and others added 2 commits September 19, 2025 14:09

Moving MCP tests to separate file, adding env variable for allowing m…

aa572f5

…cp tools to enable built in tools Signed-off-by: Alec Solder <[email protected]>

Merge branch 'main' into builtin_mcp_tools_with_headers

12f67da

yeqcharlotte reviewed Sep 19, 2025

View reviewed changes

vllm/envs.py Outdated Show resolved Hide resolved

yeqcharlotte reviewed Sep 19, 2025

View reviewed changes

vllm/entrypoints/context.py Show resolved Hide resolved

yeqcharlotte reviewed Sep 19, 2025

View reviewed changes

vllm/entrypoints/context.py Outdated Show resolved Hide resolved

yeqcharlotte and others added 3 commits September 19, 2025 14:47

Merge branch 'main' into builtin_mcp_tools_with_headers

dd50bda

Fixing nits and adding env variable validation

0bd5e53

Signed-off-by: Alec Solder <[email protected]>

Merge branch 'main' into builtin_mcp_tools_with_headers

da6d8a5

yeqcharlotte reviewed Sep 20, 2025

View reviewed changes

mergify bot added the needs-rebase label Sep 22, 2025

Alec Solder and others added 2 commits September 22, 2025 13:03

Adding tests for env variable parsing

823414a

Signed-off-by: Alec Solder <[email protected]>

Merge branch 'main' into builtin_mcp_tools_with_headers

d247304

Signed-off-by: Alec S <[email protected]>

mergify bot removed the needs-rebase label Sep 22, 2025

Merge branch 'main' into builtin_mcp_tools_with_headers

175a77c

yeqcharlotte approved these changes Sep 22, 2025

View reviewed changes

github-project-automation bot moved this from In progress to Ready in gpt-oss Issues & Enhancements Sep 22, 2025

yeqcharlotte enabled auto-merge (squash) September 22, 2025 21:53

github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 22, 2025

yeqcharlotte merged commit 45d7d85 into vllm-project:main Sep 22, 2025
51 of 53 checks passed

github-project-automation bot moved this from Ready to Done in gpt-oss Issues & Enhancements Sep 22, 2025

alecsolder deleted the builtin_mcp_tools_with_headers branch September 23, 2025 15:45

Uh oh!

[Frontend] Responses API MCP tools for built in tools and to pass through headers #24628

[Frontend] Responses API MCP tools for built in tools and to pass through headers #24628

Uh oh!

Conversation

alecsolder commented Sep 11, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Test Plan

Test Result

Uh oh!

heheda12345 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lacora commented Sep 17, 2025

Uh oh!

yeqcharlotte Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yeqcharlotte left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

yeqcharlotte commented Sep 19, 2025

Uh oh!

Uh oh!

Uh oh!

yeqcharlotte commented Sep 19, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mergify bot commented Sep 22, 2025

Uh oh!

yeqcharlotte left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

alecsolder commented Sep 11, 2025 •

edited by github-actions bot

Loading

yeqcharlotte Sep 18, 2025 •

edited

Loading