Type refinements are not applied to index expressions

[tomalexander]

Ran into this issue today, I believe mypy is incorrectly flagging this as an error:

from typing import Any, Dict, Optional


def rename_keys(
    original_dict: Dict[str, Any], key_mapping: Dict[str, Optional[str]]
) -> Dict[str, Any]:
    """Renames keys in the dictionary using the key_mapping. If the value in key_mapping is None then it will remove the key entirely"""
    return {
        key_mapping.get(k, k): v
        for k, v in original_dict.items()
        if k not in key_mapping or key_mapping[k] is not None
    }

output from mypy:

/tmp/mypytest.py:8: error: Key expression in dictionary comprehension has incompatible type "Optional[str]"; expected type "str"

if key_mapping.get(k) is None then it would fail the if-statement at the end of the dictionary comprehension and therefore get filtered out

if k is not in key_mapping then k is the result of the expression key_mapping.get(k,k). k can't be None because k comes from the keys of original_dict which has the type Dict[str, Any]

Versions: mypy 0.701, Python 3.7.4

no flags

[ilevkivskyi]

Your example doesn't type check because mypy never applies any type restrictions to function calls (even for builtin ones where the semantics is known to be pure), and this is unlikely to change.

But when I tried to propose refactoring it to use only index expressions instead of function call, it looks like there is indeed an issue, for example this fails as well:

x: List[Optional[str]]
y: Dict[str, int] = {x[i]: 0 for i in range(5) if x[i]}

It looks like the cause is that binder only considers things like x[0] a bindable expression, but not x[i]. Many things are not very principled there, so I am not sure whether this is deliberate or not.

cc @JukkaL

[JukkaL]

Narrowing down expressions like x[i] would probably be okay, as long as assigning to i would widen it back. I don't remember what the reasoning for not doing this was.

[ilevkivskyi]

Narrowing down expressions like x[i] would probably be okay, as long as assigning to i would widen it back.

OK, this makes sense.

[ilevkivskyi]

#2458 has an interesting example of this with isinstance()

[rnestler]

I think I'm hitting a similar issue with the following code:

this_is_fine = {
    "bar": True,
    "value_0": 1.0,
    "value_1": 2.0,
}

this_is_not = {
    "bar": True,
    **{f"value_{n}": float(n) for n in range(2)},
}

While the first one compiles fine I get error: Value expression in dictionary comprehension has incompatible type "float"; expected type "bool" for the second.

I created https://github.com/rnestler/minimal-example-for-mypy-dict-issue to reproduce the issue.

[lafrech]

In case it helps anyone brought here by their search engine, I stumbled upon the same case as OP today and solved it with a cast.

def rename_keys(
    original_dict: Dict[str, Any], key_mapping: Dict[str, Optional[str]]
) -> Dict[str, Any]:
    """Renames keys in the dictionary using the key_mapping. If the value in key_mapping is None then it will remove the key entirely"""
    return {
        typing.cast(key_mapping.get(k, k)): v
        for k, v in original_dict.items()
        if k not in key_mapping or key_mapping[k] is not None
    }