**kwargs should be of type Dict[Union[str, unicode], Any] in Python 2

[rowillia]

If a python 2 file is using from __future__ import unicode_literals breaks reading values out of kwargs using string literals.

Tested using e7ddab1acf44f02d63e8f5d70b1b07c70eeff73b

$ cat > test_kwargs.py << EOL
from __future__ import unicode_literals

def kwargs_func(**kwargs):
  # type: (...) -> str
  return kwargs.get('some_value', '')

kwargs_func(some_value='foo')
EOL

$ mypy --fast-parser --py2 test_kwargs.py
test_kwargs.py: note: In function "kwargs_func":
test_kwargs.py:5: error: Argument 1 to "get" of "dict" has incompatible type "unicode"; expected "str"

[gvanrossum]

Oh, I think the type of **kwargs may have to be made Dict[unicode, ...] instead of Dict[str, ...]. There's probably somewhere in mypy where it does that.

[rowillia]

Thanks @gvanrossum! Out of curiosity, why unicode and not Union[str, unicode]? Shouldn't users be able to access **kwargs via string literals as well as unicode literals?

[gvanrossum]

In mypy, anything that accepts unicode is also deemed to accept str (even though that's a bit of a lie). So the union would be unneeded.

[rowillia]

@gvanrossum Thanks for the explanation! Does that suggest typeshed should use unicode everywhere instead of str? For example:

https://github.com/python/typeshed/blob/39325bf159e9af94e7aad5086b713ec03a23e6a3/stdlib/2and3/argparse.pyi#L49
https://github.com/python/typeshed/blob/746f4db9899dfccb2e236033f6736e1665e1286d/stdlib/2.7/os/__init__.pyi#L12-L13

(We tend to use unicode_literals in our Python Codebase everywhere so this is tripping us up when adopting mypy)

[gvanrossum]

I think the pytype folks would prefer the status quo. Also, we still haven't come up with the final solution for the str/unicode/bytes issue, and it would be better to wait with massive changes until the design is settled. FWIW I think unicode_literals was a bad idea and I now count it as an anti-pattern. Though you may be stuck with it. Still, for `**kwds` mypy should be able to fix this (but it's just me ATM while Jukka is on vacation so don't expect instant fixes).

[rowillia]

@gvanrossum Unfortunately a284c48 didn't fix this issue, it fixed passing a Dict[unicode, Any] in as **kwargs but didn't fix reading values out of **kwargs.

[gvanrossum]

Reopening, since I rolled back PR #1971.

[rowillia]

@gvanrossum Any more info on what broke? Happy to fix it up.

[gvanrossum]

This is the prime example: #1971 (comment)

[rowillia]

@gvanrossum OK digging into this a bit it seems that in Python 2 type of kwargs should be Dict[basetring, Any] with a contravariant key. Does that seem right? Trying to figure out how to accomplish that in the mypy codebase itself now.

[gvanrossum]

I don't think you can do that without creating a custom subclass of Dict --
co/contravariance is a property of a type, not of a type alias (nor of a
type variable). Maybe we will just have to do that. The value type should
be taken from the type specified in **kwds: <type> though.

On Thu, Aug 4, 2016 at 11:16 AM, Roy Williams [email protected]
wrote:

@gvanrossum https://github.com/gvanrossum OK digging into this a bit it
seems that in Python 2 type of kwargs should be Dict[basetring, Any] with
a contravariant key. Does that seem right? Trying to figure out how to
accomplish that in the mypy codebase itself now.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1954 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACwrMlqnUOm6LGs3-E54A5Jqkh-qt6k4ks5qcixrgaJpZM4JXvR1
.

--Guido van Rossum (python.org/~guido)

[gnprice]

This may be something best resolved with the more precise types for Python 2's interconversion bytes <-> unicode when things are all ASCII -- @ddfisher will be writing up more about this soon. (There was a discussion back at PyCon that unfortunately wasn't written up electronically, but we just now reconstructed it and will post one or more issues with the specifics.)

[JukkaL]

I think that it's too late to change this, and since this problem is specific to Python 2, it will gradually get less important.