-
-
Notifications
You must be signed in to change notification settings - Fork 32k
gh-115528: Update language reference for PEP 646 #121181
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
To recap: the objective is to make starred expressions valid in `subscription`, which is used for generics: `Generic[...]`, `list[...]`, etc. What _is_ gramatically valid in such contexts? Seemingly any of the following. (At least, none of the following throw `SyntaxError` in a 3.12.3 REPL.) Generic[x] Generic[*x] Generic[*x, y] Generic[y, *x] Generic[x := 1] Generic[x := 1, y := 2] So introducting flexible_expression: expression | assignment_expression | starred_item end then switching `subscription` to use `flexible_expression` sorts that. But then we need to field `yield` - for which any of the following are apparently valid: yield x yield x, yield x, y yield *x, yield *x, *y Introducing a separate `yield_list` is the simplest way I've been figure out to do this - separating out the special case of `starred_item ,`.
@JelleZijlstra Sorry this revision took a while - had a very busy few weeks at work, and it took a while to figure this out 😅 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! It's quite tricky to get all these expression variants right.
starred_list: `starred_item` ("," `starred_item`)* [","] | ||
starred_expression: `expression` | (`starred_item` ",")* [`starred_item`] | ||
starred_item: `assignment_expression` | "*" `or_expr` | ||
starred_expression: ["*"] `or_expr` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
or_expr
is valid for what comes after *
, but starred_expression now also gets used (via expression_list) in the rule for list_display, and there other expressions are valid, e.g. comparisons. I think this should be:
starred_expression: ["*"] `or_expr` | |
starred_expression: "*" `or_expr` | `expression` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think expression
is already included in expression_list
through assignment_expression
- though maybe it would be worth adding the extra | expression
here to make it clear?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah you're right, I missed that assignment_expression
includes expression
.
But I think there's still something wrong here: yield a == b, c == d
is valid, but a == b
is a comparison
, and the grammar for yield_list
indicates that after the comma we can only have a starred_expression
.
Co-authored-by: Jelle Zijlstra <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, this looks right to me now but I'd like someone else to look over the grammar too, since it's been quite tricky to get right.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM (though I'm also not an expert here)
Thanks @mrahtz for the PR, and @JelleZijlstra for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13. |
To recap: the objective is to make starred expressions valid in `subscription`, which is used for generics: `Generic[...]`, `list[...]`, etc. What _is_ gramatically valid in such contexts? Seemingly any of the following. (At least, none of the following throw `SyntaxError` in a 3.12.3 REPL.) Generic[x] Generic[*x] Generic[*x, y] Generic[y, *x] Generic[x := 1] Generic[x := 1, y := 2] So introducting flexible_expression: expression | assignment_expression | starred_item end then switching `subscription` to use `flexible_expression` sorts that. But then we need to field `yield` - for which any of the following are apparently valid: yield x yield x, yield x, y yield *x, yield *x, *y Introducing a separate `yield_list` is the simplest way I've been figure out to do this - separating out the special case of `starred_item ,`. (cherry picked from commit 7d3497f) Co-authored-by: Matthew Rahtz <[email protected]> Co-authored-by: Jelle Zijlstra <[email protected]>
To recap: the objective is to make starred expressions valid in `subscription`, which is used for generics: `Generic[...]`, `list[...]`, etc. What _is_ gramatically valid in such contexts? Seemingly any of the following. (At least, none of the following throw `SyntaxError` in a 3.12.3 REPL.) Generic[x] Generic[*x] Generic[*x, y] Generic[y, *x] Generic[x := 1] Generic[x := 1, y := 2] So introducting flexible_expression: expression | assignment_expression | starred_item end then switching `subscription` to use `flexible_expression` sorts that. But then we need to field `yield` - for which any of the following are apparently valid: yield x yield x, yield x, y yield *x, yield *x, *y Introducing a separate `yield_list` is the simplest way I've been figure out to do this - separating out the special case of `starred_item ,`. (cherry picked from commit 7d3497f) Co-authored-by: Matthew Rahtz <[email protected]> Co-authored-by: Jelle Zijlstra <[email protected]>
GH-124632 is a backport of this pull request to the 3.13 branch. |
GH-124633 is a backport of this pull request to the 3.12 branch. |
…124633) gh-115528: Update language reference for PEP 646 (GH-121181) To recap: the objective is to make starred expressions valid in `subscription`, which is used for generics: `Generic[...]`, `list[...]`, etc. What _is_ gramatically valid in such contexts? Seemingly any of the following. (At least, none of the following throw `SyntaxError` in a 3.12.3 REPL.) Generic[x] Generic[*x] Generic[*x, y] Generic[y, *x] Generic[x := 1] Generic[x := 1, y := 2] So introducting flexible_expression: expression | assignment_expression | starred_item end then switching `subscription` to use `flexible_expression` sorts that. But then we need to field `yield` - for which any of the following are apparently valid: yield x yield x, yield x, y yield *x, yield *x, *y Introducing a separate `yield_list` is the simplest way I've been figure out to do this - separating out the special case of `starred_item ,`. (cherry picked from commit 7d3497f) Co-authored-by: Matthew Rahtz <[email protected]> Co-authored-by: Jelle Zijlstra <[email protected]>
…124632) gh-115528: Update language reference for PEP 646 (GH-121181) To recap: the objective is to make starred expressions valid in `subscription`, which is used for generics: `Generic[...]`, `list[...]`, etc. What _is_ gramatically valid in such contexts? Seemingly any of the following. (At least, none of the following throw `SyntaxError` in a 3.12.3 REPL.) Generic[x] Generic[*x] Generic[*x, y] Generic[y, *x] Generic[x := 1] Generic[x := 1, y := 2] So introducting flexible_expression: expression | assignment_expression | starred_item end then switching `subscription` to use `flexible_expression` sorts that. But then we need to field `yield` - for which any of the following are apparently valid: yield x yield x, yield x, y yield *x, yield *x, *y Introducing a separate `yield_list` is the simplest way I've been figure out to do this - separating out the special case of `starred_item ,`. (cherry picked from commit 7d3497f) Co-authored-by: Matthew Rahtz <[email protected]> Co-authored-by: Jelle Zijlstra <[email protected]>
There are two aims here:
*args: *Ts
valid.subscription
, such thatGeneric[*a]
andlist[*a]
are valid.1 is straightforward, but 2 requires a bit more work:
The
subscription
rule relies onexpression_list
. But it's not as simple as changingexpression_list
, because it's also used byyield
. Foryield
, althoughyield *a, b
etc is valid,yield *a
is not - the closest isyield *a,
.I also noticed that rules elsewhere that use this group of expression rules aren't reflective of the current grammar. Subscriptions, lists and sets all allow assignment expressions:
Generic[x := 1]
,[x := 1]
and{x := 1}
all seem to be valid, at least in a REPL loop on Python 3.12.4.Combining all this, this PR introduces a new rule
flexible expression: assignment_expression | starred_expression
, and changesexpression_list
,list_display
andset_display
to use this new rule. So all that fixes both starred expressions and assignment expressions valid in the contexts mentioned above.(I've also made this whole group of expression rules less hopefully confusing by a) changing
starred_expression
to literally just be a single expression rather than potentially a list of expressions, and b) changedstar_item
to only be a starred expression.)But for
yield
, things are different. Assignment expressions are not valid in the context ofyield
, and we need to express the stipulation that if the yield expression begins with a starred expression, there needs to be a comma. So I've introduced another new ruleyield_list
, that expresses this requirement.📚 Documentation preview 📚: https://cpython-previews--121181.org.readthedocs.build/