Skip to content

Sync typeshed #12766

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 13, 2022
Merged

Sync typeshed #12766

merged 4 commits into from
May 13, 2022

Conversation

hauntsaninja
Copy link
Collaborator

@hauntsaninja hauntsaninja commented May 11, 2022

Source commit:
python/typeshed@a27f15e

@github-actions

This comment has been minimized.

@hauntsaninja hauntsaninja marked this pull request as draft May 11, 2022 05:35
@hauntsaninja
Copy link
Collaborator Author

Note the paroxython change is a true positive. But a tonne of test failures, let's see what's gone wrong.

@AlexWaygood
Copy link
Member

AlexWaygood commented May 11, 2022

For reference, we discussed the new homeassistant hit in depth over in python/typeshed#7737. It's a... weird one.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@JelleZijlstra
Copy link
Member

The new paroxython error is a true positive, I think it came from one of my regex changes.

@JelleZijlstra
Copy link
Member

The errors are on code that looks like this:

        return re.sub('[^\\x20-\\x7e]',
                      lambda m: r'\u%.4x' % ord(m.group(0)), s)

return re.sub('[^\\x20-\\x7e]',

I suspect mypyc miscompiles this into something involving bytes.

@JukkaL
Copy link
Collaborator

JukkaL commented May 11, 2022

I suspect mypyc miscompiles this into something involving bytes.

Mypyc probably somehow believes that the call returns bytes and raises an exception when it actually returns a str. I can investigate this tomorrow.

JukkaL added a commit that referenced this pull request May 13, 2022
Previously we could export an invalid type for a lambda passed to an
overloaded function. We find the matching overload variant by type checking
the arguments against all overload items. We exported types for lambdas
always from the final potential overload item, even if that wasn't the matching
one. This could result in mypyc adding invalid type coercions that could result in
bogus TypeErrors.

The fix is to store types inferred when looking for matching overload items
into temporary type maps, and only the type map from the matching item
gets merged into the exported types.

This doesn't fully solve the problem -- if there are Any types in the
arguments, we can still export invalid types. This should be enough to
unblock syncing typeshed (#12766). Typeshed started triggering the issue
in compiled mypy when re.sub was changed to an overloaded function.

Work on #12773.
@JukkaL
Copy link
Collaborator

JukkaL commented May 13, 2022

Hopefully #12780 fixes the test failures if you merge master.

@github-actions
Copy link
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

vision (https://github.com/pytorch/vision)
- torchvision/datasets/utils.py:69: error: Argument 1 to "md5" has incompatible type "**Dict[str, bool]"; expected "Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData]]"  [arg-type]
+ torchvision/datasets/utils.py:69: error: Argument 1 to "md5" has incompatible type "**Dict[str, bool]"; expected "Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData, PickleBuffer]]"  [arg-type]

paroxython (https://github.com/laowantong/paroxython)
+ paroxython/map_taxonomy.py:223: error: Argument 1 to "append" of "list" has incompatible type "str"; expected "TaxonName"

core (https://github.com/home-assistant/core)
+ homeassistant/util/color.py:300: error: Redundant cast to "float"  [redundant-cast]

@hauntsaninja hauntsaninja marked this pull request as ready for review May 13, 2022 21:18
@hauntsaninja hauntsaninja merged commit 7fbf4de into python:master May 13, 2022
@hauntsaninja hauntsaninja deleted the sync branch May 13, 2022 21:18
@JelleZijlstra
Copy link
Member

Thanks @JukkaL for the fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants