Skip to content

Conversation

eyurtsev
Copy link
Collaborator

@eyurtsev eyurtsev commented Sep 12, 2025

Description:

We encountered a bug where we used a pydantic validator to patch broken llm inputs to our tools. The validator would fix the input dict, but the BaseTool would return an empty dict to the tool because our validator changed the dictionary keys. This PR changes one line of code to fix the issue and adds a test to improve the test coverage.

  • Issue:

Concretely, the following code would fail before the PR:

from pydantic import BaseModel, model_validator
from langchain_core.tools import tool


class B(BaseModel):
    b: int


class A(BaseModel):
    a: B

    @model_validator(mode="before")
    def a_wrapper(cls, data):
        if "a" not in data:
            return {"a": data}
        return data


if __name__ == "__main__":
    # Valid input passes
    print(A.model_validate({"a": {"b": 1}}))
    # Invalid input also passes because validator patches it
    print(A.model_validate({"b": 1}))


@tool(args_schema=A, infer_schema=False)
def answer(
    **kwargs,
):
    """Use this function to provide your final answer to the request."""
    return kwargs


if __name__ == "__main__":
    # Valid input passes
    assert answer.invoke({"a": {"b": 1}}) == {"a": B(b=1)}
    # Invalid input should pass, but throws assertion error
    res = answer.invoke({"b": 1})
    assert res == {"a": B(b=1)}, f"Failure: {res} of type {type(res)}"
    # AssertionError: Failure: {} of type <class 'dict'>

Add tests and docs:
I added a test for the above use case.

Comments:

Please note that I had to remove *args: Any from one of the tests. The previous code would filter out the args keyword. I don't see any realistic use case where one would have *args in their tool as this gives no meaningful instruction to the LLM on how to use the *args. Please let me know if you see any issues here. I don't see a clean and robust method that would have the new test passing while still filtering out the *args argument. Let me know if you see a better solution.

This PR is a port of #30004 courtesy of @VMinB12

Copy link

vercel bot commented Sep 12, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
langchain Ready Ready Preview Comment Sep 12, 2025 7:55pm

@@ -643,7 +643,7 @@ def test_named_tool_decorator_return_direct() -> None:
"""Test functionality when arguments and return direct are provided as input."""

@tool("search", return_direct=True)
def search_api(query: str, *args: Any) -> str:
def search_api(query: str) -> str:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the breaking part

Copy link

codspeed-hq bot commented Sep 12, 2025

CodSpeed WallTime Performance Report

Merging #32925 will not alter performance

Comparing eugene/basetool_2_breaking (23c2b92) with wip-v1.0 (67aa37b)1

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

Summary

✅ 13 untouched

Footnotes

  1. No successful run was found on wip-v1.0 (b88115f) during the generation of this report, so 4cc242e was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@eyurtsev eyurtsev changed the title fix(core)!: respect extra inputs that may come from the validator fix(core)!: fix BaseTool when args_schema has pydantic validators that change the input keys Sep 12, 2025
return {
k: getattr(result, k) for k, v in result_dict.items() if k in tool_input
}
return {k: getattr(result, k) for k, v in result_dict.items()}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The filter here is pretty odd. I don't understand why the original code had it

@mdrxy mdrxy added the core Related to the package `langchain-core` label Sep 12, 2025
@eyurtsev eyurtsev self-assigned this Sep 12, 2025
@eyurtsev eyurtsev requested a review from mdrxy September 12, 2025 19:52
@eyurtsev eyurtsev removed their assignment Sep 12, 2025
Copy link

codspeed-hq bot commented Sep 12, 2025

CodSpeed Instrumentation Performance Report

Merging #32925 will not alter performance

Comparing eugene/basetool_2_breaking (23c2b92) with wip-v1.0 (67aa37b)1

Summary

✅ 14 untouched

Footnotes

  1. No successful run was found on wip-v1.0 (b88115f) during the generation of this report, so 4cc242e was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Related to the package `langchain-core`
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants