Add basic support for enum literals #6668
Conversation
This pull request adds in basic support for enum literals. Basically, you can construct them and use them as types, but not much else. (I originally said I was going to wait until the new semantic analyzer work was finished, but I'm bored.) Some notes: 1. I plan on submitting incremental and fine-grained-mode tests in a future PR. 2. I added two copies of each test -- one using the old semantic analyzer and one using the new one. Let me know if we prefer not doing this and if I should pick one over the other. 3. I wanted to add support for aliases to enums, but ran into some difficulty doing so. See python#6667 for some analysis on the root cause.
mypy/newsemanal/semanal.py
Outdated
| for i in range(1, len(parts) - 1): | ||
| next_sym = n.names[parts[i]] | ||
| assert isinstance(next_sym.node, MypyFile) | ||
| assert isinstance(next_sym.node, (MypyFile, TypeInfo)) |
There was a problem hiding this comment.
I think there is some misunderstanding/bug somewhere. This function should never be used for things like local lookup. It should be only used to construct a known type like builtins.str. For local lookup one needs to use lookup_qualified(). Why do you need this modification?
There was a problem hiding this comment.
I made this change in order to get this example working:
from enum import Enum
from typing_extensions import Literal
class Outer:
class Inner(Enum):
FOO = 1
BAR = 2
x: Literal[Outer.Inner.FOO]
reveal_type(x)
Basically, the literal-handling code in typeanal receives back the string "module.Outer.Inner". We then call typeanal.named_type_with_normalized_str which in turn calls typeanal.named_type which in turn calls semanal.lookup_fully_qualified. That function then ends up crashing because "Outer" is a TypeInfo, not a MypyFile.
FWIW I was under the impression that:
lookup_qualifiedis for looking up fully-qualified types, and has a bunch of error handling for when the thing we're trying to look up doesn't exist.lookup_fully_qualifiedis a simpler version oflookup_qualified, except without error handling.lookupis for looking up some unqualified name across all namespaces (local and global).
I also noticed that lookup_qualified handles both MypyFile and TypeInfo, and so figured that the fact that lookup_fully_qualified doesn't was an oversight.
There was a problem hiding this comment.
FWIW I was under the impression that
This is a wrong impression. Which is normal, we have more than dozen lookup functions in the code-base (likely caused by the fact that many people who are not familiar with mypy contributed some reinvented bicycles, which is again normal). Largely, they can be separated in two classes:
- Local lookup: what given name or member expression points to using Python name resolution
- Global lookup: find a symbol using its "full address" used to construct some standard known types.
The named_type() and friends are from the second group. While the first group is roughly two methods lookup() and lookup_qualified().
Essentially what is going on here looks like a hack (and TBH I don't like it). You already have the TypeInfo needed to construct the literal type, but instead you get its full name and pass it to RawExpressionType that later passes the name to find the same TypeInfo again.
Initially, string name was passed to RawLiteralExpression because it is was constructed at very early stage. Here however, it looks unnecessary. Why can't we just immediately return LiteralType there with a fallback instance constructed from a known TypeInfo?
There was a problem hiding this comment.
The
named_type()and friends are from the second group. While the first group is roughly two methodslookup()andlookup_qualified().
Hmm, ok. In that case, what's the correct way of looking up a nested type via a global lookup?
It seems all of the "global lookup" functions assume that all types are defined within modules only, which I'm not so sure is correct.
Why can't we just immediately return
LiteralTypethere with a fallback instance constructed from a knownTypeInfo?
Sure, we can do that. Is it safe to just construct new whenever we want Instances though? E.g. there isn't a need to register them in some global registry?
There was a problem hiding this comment.
It seems all of the "global lookup" functions assume that all types are defined within modules only, which I'm not so sure is correct.
It is not correct, but this is kind of optimization. After the refactoring we will have two methods, one fast for builtin things like builtins.str, and another one that can be used in plugins etc., with full semantics.
E.g. there isn't a need to register them in some global registry?
What kind of registry do you mean? I think one can just call Instance(info, args, line, column) or am I missing something?.
There was a problem hiding this comment.
What kind of registry do you mean? I think one can just call Instance(info, args, line, column) or am I missing something?.
Honestly, I have no idea. Instances and TypeInfos are stuffed with so much functionality that I always get a little worried that touching either of them or creating new ones will break something or interfere with something related to incremental or fine-grained mode in some weird way.
test-data/unit/check-literal.test
Outdated
| c: Literal[mod_b.Test.FOO] | ||
|
|
||
| f(a) | ||
| f(b) # E: Argument 1 to "f" has incompatible type "Literal[mod_a.Test2.FOO]"; expected "Literal[mod_a.Test.FOO]" |
There was a problem hiding this comment.
Even here I would skip the module name. Maybe you can somehow hack this with MessageBuilder.format_distinctly()?
There was a problem hiding this comment.
Ooh, I forgot about MessageBuilder.format_distinctly! It turned out tweaking that function was sufficient to simplify most of the error messages above in the way you suggested.
mypy/newsemanal/semanal.py
Outdated
| for i in range(1, len(parts) - 1): | ||
| next_sym = n.names[parts[i]] | ||
| assert isinstance(next_sym.node, MypyFile) | ||
| assert isinstance(next_sym.node, (MypyFile, TypeInfo)) |
There was a problem hiding this comment.
FWIW I was under the impression that
This is a wrong impression. Which is normal, we have more than dozen lookup functions in the code-base (likely caused by the fact that many people who are not familiar with mypy contributed some reinvented bicycles, which is again normal). Largely, they can be separated in two classes:
- Local lookup: what given name or member expression points to using Python name resolution
- Global lookup: find a symbol using its "full address" used to construct some standard known types.
The named_type() and friends are from the second group. While the first group is roughly two methods lookup() and lookup_qualified().
Essentially what is going on here looks like a hack (and TBH I don't like it). You already have the TypeInfo needed to construct the literal type, but instead you get its full name and pass it to RawExpressionType that later passes the name to find the same TypeInfo again.
Initially, string name was passed to RawLiteralExpression because it is was constructed at very early stage. Here however, it looks unnecessary. Why can't we just immediately return LiteralType there with a fallback instance constructed from a known TypeInfo?
I think probably not? I'll make sure to double-check this and see if we can delete the comment while preparing the incremental mode and fine-grained mode tests PR. |
This pull request adds in basic support for enum literals. Basically, you can construct them and use them as types, but not much else.
(I originally said I was going to wait until the new semantic analyzer work was finished, but I'm bored.)
Some notes:
I plan on submitting incremental and fine-grained-mode tests in a future PR.
I added two copies of each test -- one using the old semantic analyzer and one using the new one. Let me know if we prefer not doing this and if I should pick one over the other.
I wanted to add support for aliases to enums, but ran into some difficulty doing so. See Semantic analyzers do not correctly handle qualified type aliases #6667 for some analysis on the root cause.