pydantic dist.version is None causes TypeError: expected string or bytes-like object, got 'NoneType'

[Torxed]

This might be an issue in my environment, and it might belong in packaging but throwing out a question/bug report here just in case as I drew a judgement call that it's build supplying packaging with information in a potentially wrong way. And just to note if it matters, I'm managing packages via my operating system.

When I manually build pymilter I get the following error:

Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/build/__main__.py", line 375, in main
    built = build_call(
            ^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/build/__main__.py", line 208, in build_package
    out = _build(isolation, builder, outdir, distribution, config_settings, skip_dependency_check)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/build/__main__.py", line 147, in _build
    return _build_in_current_env(builder, outdir, distribution, config_settings, skip_dependency_check)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/build/__main__.py", line 127, in _build_in_current_env
    missing = builder.check_dependencies(distribution)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/build/__init__.py", line 370, in check_dependencies
    return {u for d in dependencies for u in check_dependency(d)}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/build/__init__.py", line 370, in <setcomp>
    return {u for d in dependencies for u in check_dependency(d)}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/build/__init__.py", line 186, in check_dependency
    yield from check_dependency(other_req_string, ancestral_req_strings + (normalised_req_string,), req.extras)
  File "/usr/lib/python3.11/site-packages/build/__init__.py", line 186, in check_dependency
    yield from check_dependency(other_req_string, ancestral_req_strings + (normalised_req_string,), req.extras)
  File "/usr/lib/python3.11/site-packages/build/__init__.py", line 186, in check_dependency
    yield from check_dependency(other_req_string, ancestral_req_strings + (normalised_req_string,), req.extras)
  File "/usr/lib/python3.11/site-packages/build/__init__.py", line 180, in check_dependency
    if req.specifier and not req.specifier.contains(dist.version, prereleases=True):
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/packaging/specifiers.py", line 905, in contains
    item = Version(item)
           ^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/packaging/version.py", line 196, in __init__
    match = self._regex.search(version)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: expected string or bytes-like object, got 'NoneType'

ERROR expected string or bytes-like object, got 'NoneType'

(The source of build is modified by adding one print just to see what package and/or version was causing the hiccup)
Here, dist.version becomes None for pydantic specifically in this case:

build/src/build/__init__.py

Lines 173 to 179 in cd06da2

	try:
	dist = importlib_metadata.distribution(req.name) # type: ignore[no-untyped-call]
	except importlib_metadata.PackageNotFoundError:
	# dependency is not installed in the environment.
	yield ancestral_req_strings + (normalised_req_string,)
	else:
	if req.specifier and not req.specifier.contains(dist.version, prereleases=True):

It feels either like a bug in pydantic not presenting the version in the right way, or build for passing in unexpected data to packaging or packaging not handling the case. But what if anything is wrong with my environment? And how to debug/correct this, as I most likely have more packages down the line that will produce the same issue.

Steps to potentially reproduce:

anton@bigrigv2 ~/github $ git clone git@github.com:sdgathman/pymilter.git
anton@bigrigv2 ~/github $ cd pymilter/
anton@bigrigv2 ~/github/pymilter (master)$ python -m build --wheel --no-isolation
* Getting build dependencies for wheel...
running egg_info
creating pymilter.egg-info
writing pymilter.egg-info/PKG-INFO
writing dependency_links to pymilter.egg-info/dependency_links.txt
writing top-level names to pymilter.egg-info/top_level.txt
writing manifest file 'pymilter.egg-info/SOURCES.txt'
reading manifest file 'pymilter.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'milter-template.py'
warning: no files found matching 'start.sh'
adding license file 'COPYING'
writing manifest file 'pymilter.egg-info/SOURCES.txt'
wheel version: 0.40.0
setuptools version: 68.0.0
platformdirs version: 3.5.1
jaraco.text version: 3.11.1
jaraco.functools version: 3.7.0
more-itertools version: 9.1.0
jaraco.context version: 4.3.0
autocommand version: 2.2.2
inflect version: 6.0.4
pydantic version: None

My environment:

• Python --version = 3.11.3
• pydantic.__version__ = '1.10.9'
• python-build = 0.10.0-4
• python-packaging = 23.1-1
• python-installer = 0.7.0-3
• Arch Linux = 2023-06-25

[layday]

How did you install Pydantic and where is it located? I'm suspecting you have an old Pydantic dist-info folder lying around that's been emptied but not deleted. Unfortunately importlib.metadata won't raise for IO errors.

[Torxed]

How did you install Pydantic

Both with pip and the OS package manager pacman.
I cleaned the environment between attempts just to see if one worked better than the other.

where is it located

>>> import pydantic
>>> pydantic
<module 'pydantic' from '/usr/lib/python3.11/site-packages/pydantic/__init__.cpython-311-x86_64-linux-gnu.so'>

I'm suspecting you have an old Pydantic dist-info folder lying around

I attempted a sudo find / -iname '*pydantic*' but couldn't find anything that stood out.
Only thing that I could find was /usr/lib/python3.11/site-packages/pydantic-1.10.7.dist-info as well as remanence of an old Python installation that got left in the great upgrade from 3.10 to 3.11 a while back: /home/anton/.local/lib/python3.10/site-packages/pydantic as well as /usr/lib/python3.10/site-packages/pydantic-1.9.0.dist-info.

Removed all those but still the same issue.

[layday]

You should look at locations on your sys.path. Ensure that there's only one pydantic-*.dist-info folder and that it contains a METADATA file with a valid Version.

[Torxed]

You should look at locations on your sys.path. Ensure that there's only one pydantic-*.dist-info folder and that it contains a METADATA file with a valid Version.

Only thing containing the name pydantic would be:

find: ‘/usr/lib/python311.zip’: No such file or directory
/usr/lib/python3.11/site-packages/archinstall/lib/models/pydantic.py
/usr/lib/python3.11/site-packages/archinstall/lib/models/__pycache__/pydantic.cpython-311.pyc
/usr/lib/python3.11/site-packages/archinstall/lib/models/__pycache__/pydantic.cpython-311.opt-1.pyc
/usr/lib/python3.11/site-packages/archinstall/lib/models/pydantic.py
/usr/lib/python3.11/site-packages/archinstall/lib/models/__pycache__/pydantic.cpython-311.pyc
/usr/lib/python3.11/site-packages/archinstall/lib/models/__pycache__/pydantic.cpython-311.opt-1.pyc

I didn't think those findings were worth mentioning, but could they be relevant?

In case my bash sucks (list taken from sys.path):

#!/bin/bash

find /usr/lib/python311.zip -iname '*pydantic*'
find /usr/lib/python3.11 -iname '*pydantic*'
find /usr/lib/python3.11/lib-dynload -iname '*pydantic*'
find /home/anton/.local/lib/python3.11/site-packages -iname '*pydantic*'
find /usr/lib/python3.11/site-packages -iname '*pydantic*'

[layday]

Are these your sys.path entries? And are you still able to load the Pydantic distribution? Try running python -c "import importlib.metadata; print(importlib.metadata.distribution('pydantic')._path)".

[Torxed]

Those were my sys.path yes.
The result from your command when pydantic is not installed is:

anton@bigrigv2 ~ $ python -c "import importlib.metadata; print(importlib.metadata.distribution('pydantic')._path)"
Traceback (most recent call last):
  File "/usr/lib/python3.11/importlib/metadata/__init__.py", line 563, in from_name
    return next(cls.discover(name=name))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.11/importlib/metadata/__init__.py", line 981, in distribution
    return Distribution.from_name(distribution_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/metadata/__init__.py", line 565, in from_name
    raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for pydantic

While installed it's:

anton@bigrigv2 ~ $ python -c "import importlib.metadata; print(importlib.metadata.distribution('pydantic')._path)"
/usr/lib/python3.11/site-packages/pydantic-1.10.9.dist-info

[layday]

And python -c "import importlib.metadata; print(importlib.metadata.distribution('pydantic').version)" returns the correct version number and build still errors?

[Torxed]

and build still errors?

Yepp sadly :/

[layday]

Are you certain this is still Pydantic having a None version?

[Torxed]

Glad you asked, now it's typing_extensions that's breaking.
Cleared it out as previously instructed and now builds work.

Thanks for the assistance!

Is there anything that could be done to improve packaging or build here? Or to avoid this in the future?
For instance check for empty paths as you mentioned in importlib.metadata or even printing the package that's breaking things to give the user a hint?

Not sure how I even ended up in this state, probably mixing pip and pacman at some point. But it would feel more stable if the user did not have to alter installed libs source to figure out which package is causing the issue. Maybe it's a real one-off, but on the off chance that it something to do with PEP 668 or user error, it would be nice to enhance this area.

[layday]

I think it'd be best to address this in importlib.metadata. I opened python/importlib_metadata#457 some time ago, but I'm not sure it'd help in your case. I don't know why importlib.metadata suppresses IO errors; ideally it would error when encountering a dist-info folder wihout METADATA.

In the meantime, build should handle None versions and raise something informative instead of letting the value bubble through to packaging.

[Torxed]

Both routes sounds ideal to me.
I'll close this one as it will at best serve a reference for the issue at hand, not really helpful for future development tasks.

I'll also subscribe to your ticket, it seems quite related.