ValidationError.from_exception_data doesn't work with error type: 'ip_v4_address'"

[mikedavidson-evertz]

Describe the bug

When constructing validation errors using ValidationError.from_exception_data, this fails when the error type is ip_v4_address.It will raise:

KeyError: "Invalid error type: 'ip_v4_address'

To Reproduce
Run the code below. It uses a custom model validator that manually constructs the validation error using ValidationError.from_exception_data

from pydantic import BaseModel, Field, model_validator, ValidationError
from ipaddress import IPv4Address
from pydantic_core import InitErrorDetails


class OutputStream(BaseModel):
    destination_ip: IPv4Address = Field(
        ...,
    )


class StreamingDevice(BaseModel):
    output_stream: OutputStream = Field(
        ...,
    )

    @model_validator(mode="before")
    @classmethod
    def validate_model(cls, data: dict) -> dict:
        validation_errors: list[InitErrorDetails] = []

        if data.get("output_stream") is not None:
            try:
                OutputStream.model_validate(data["output_stream"])
            except ValidationError as e:
                validation_errors.extend(e.errors())

        if validation_errors:
            raise ValidationError.from_exception_data(title=cls.__name__, line_errors=validation_errors)

        return data



streaming_device_payload = {"output_stream": {"destination_ip": "123"}}
streaming_device = StreamingDevice.model_validate(streaming_device_payload)

Expected behavior
Pydantic is capable of generating 'ip_v4_address' errors when using model validation without the custom model validator. Therefore, we should be able to manually construct the validation error using the error dictionary that contains the ip_v4_address error type as input into ValidationError.from_exception_data.

If you were to remove the custom model validator in the example above, Pydantic would successfully throw that error type:

pydantic_core._pydantic_core.ValidationError: 1 validation error for StreamingDevice
output_stream.destination_ip
  Input is not a valid IPv4 address [type=ip_v4_address, input_value='123', input_type=str

Haven't tested it, but should ip_v4_address be in this error list here?
https://github.com/pydantic/pydantic-core/blob/main/python/pydantic_core/core_schema.py#L3900

Version:

• OS: Ubuntu 20.04.4 LTS
• Python version: 3.9.16
• pydantic version: 2.3.0

[westrachel]

I'm running into a similar issue where ValidationError.from_exception_data doesn't work with other error types, specifically value_error and enum.

Reproducing the Errors:
(1) enum

from pydantic_core import InitErrorDetails
from pydantic import ValidationError

validation_errors = []

validation_errors.append(InitErrorDetails(
    type="enum",
    loc=("direction",),
    input="Z",
))

raise ValidationError.from_exception_data(
    title="example",
    line_errors=validation_errors
)

# Result:
Traceback (most recent call last):
  File ".../example.py", line 12, in <module>
    raise ValidationError.from_exception_data(
TypeError: Enum: 'expected' required in context

(2) value_error

from pydantic_core import InitErrorDetails
from pydantic import ValidationError

validation_errors = []

validation_errors.append(InitErrorDetails(
    type="value_error",
    loc=("direction",),
    input="Z",
))

raise ValidationError.from_exception_data(
    title="example",
    line_errors=validation_errors
)

# Result:
Traceback (most recent call last):
  File ".../example.py", line 12, in <module>
    raise ValidationError.from_exception_data(
TypeError: ValueError: 'error' required in context

Expected Behavior:

Instead of the TypeErrors raised above, I expect a pydantic ValidationError to be raised that shows relevant info for the respective enum and value_error types. For example, If I assign type in InitErrorDetails in the above code to an error that does not fully fit my use case (string_type), it does work and does produce the validation error I want to raise (but with the wrong message; I don't want the invalid string message for my use case):

from pydantic_core import InitErrorDetails
from pydantic import ValidationError

validation_errors = []

validation_errors.append(InitErrorDetails(
    type="string_type",
    loc=("direction",),
    input="Z",
))

raise ValidationError.from_exception_data(
    title="example",
    line_errors=validation_errors
)

# Result:
Traceback (most recent call last):
  File ".../example.py", line 12, in <module>
    raise ValidationError.from_exception_data(
pydantic_core._pydantic_core.ValidationError: 1 validation error for example
direction
  Input should be a valid string [type=string_type, input_value='Z', input_type=str]
    For further information visit https://errors.pydantic.dev/2.5/v/string_type

Versions:
OS: macOS Sonoma 14.1.1
Python version: 3.10.12
pydantic version: 2.5.2

[KeynesYouDigIt]

I am having the same issues. The new things like @model_validator are an exciting direction, but we really need support on how to properly aggregate custom errors in this case.

When trying to aggregate errors, I am really struggling. Is this a sign I am using pydantic not as intended? if not, are there more complex cases in the docs?

If I am using this as intended, here were debug steps
specific error I am getting - TypeError: ValueError: 'error' required in context.

I looked at the Init error typed dict..... nothing jumps out as to what was wrong in my case.

pydantic-core/python/pydantic_core/__init__.py

Line 94 in 47aff70

class InitErrorDetails(_TypedDict):

[dsayling]

@KeynesYouDigIt

It's complaining about there being no error in the ctx dict

e.g.

...
            InitErrorDetails(
                {
                    "type": "value_error",
                    "loc": (info.field_name,),
                    "input": v,
                    "ctx": {
                        "error": f"your_message {v}",
                    },
                }
            )

---

edit

Also, depending on the "type" here, the ctx might need different key values. See the linked doc page and scroll down to ctx

As far as I know, you can inspect what needs to be in the ctx to render the error by doing

from pydantic_core._pydantic_core import list_all_errors
list_all_errors()

[KeynesYouDigIt]

@dsayling awesome, thanks. Is Pydantic actually tring to move away from error aggregation? I notice _pydantic_core is semi private.

LMK if anyone knows the "right" way to aggregate errors for full feedback, or if theres an intentional break away from this.

edit- reading that error handling example in the doc kinda helps! whats the difference between list_all_errors and e.errors() ? I am guessing list_all_errors is multiple model instances or something? (looking now)

[samuelcolvin]

What do you mean by "error aggregation"?

I didn't think we're trying to move anywhere with errors specifically, just allow people do to what want.

If you can give some context on what you're trying to do, I'll try to explain how I think you should proceed.

[dsayling]

@samuelcolvin - I can't speak for OP, but for myself, I'd assume that I could define multiple field_validators to an attribute to verify it meets a few different criteria (and report ALL missed criteria at once). However, I've noticed that once one field_validator raises a ValueError, the rest of the field_validators for that field do not execute. It would be nice to not have to aggregate errors in field_validator by being able to define multiple and allow them to just get aggregated.

Also, in the case of a model_validator, I collect errors into an array of InitErrorDetails with the attribute as the loc, and then call ValidationError.from_exception_data with the array of InitErrorDetails.

It’s been working for me for the most part, but explaining to others on my team has been a small challenge.

[samuelcolvin]

Yeah, that's fundamentally against how pydantic works, and has always worked.

I think your work around is the best solution.

[interifter]

I think in general, the documentation should be better. It is a challenge to understand how to properly use ValidationError.from_exception_data.

https://docs.pydantic.dev/latest/api/pydantic_core/#pydantic_core.ValidationError.from_exception_data does not go into any detail about what InitErrorDetails actually wants us to provide.

Maybe we can get a pointer to better documentation, here, so Google eventually gets people to something useful.

[sydney-runkle]

Following up here re the reason this doesn't work with ip_v4_address - that's associated with a PydanticCustomError, not an endorsed ErrorType from pydantic-core.

Definitely confusing from the user's perspective, though.