parse yada-yada only as a statement

Commit f5727a1 tried to make yada-yada
be parsed consistently as a term expression, but actually things are
more complicated than that.  The tokeniser didn't accept yada-yada in
the right contexts to make it usable as an expression, and changing
that would require decisions on resolving ambiguities between yada-yada
and flip-flop.  It's also documented as being a statement rather than
an expression, though with some incorrect information about ambiguities.

Overall it looks more like the intent was for yada-yada to be a statement.
This commit makes it grammatically treated as such, and also fixes up
the dubious parts of the documentation.  [perl #132150]

Author: Zefram

perly.act

@@ -181,6 +181,6 @@ int yyparse (void);
 
 
 /* Generated from:
- * 68a62998f7c607d7a1fa0fdf2d16ec19074c256dde18dfc475daff70d378e025 perly.y
+ * 4667736d3c31a5169bab73c89d70a27dbce5ea4fe7e3c332a236f8a210aafdc2 perly.y
  * 153cba5d215c1a083a0459c43f4d55c45fd0a7093c197d7247a456dcde21ea53 regen_perly.pl
  * ex: set ro: */

perly.h

@@ -90,7 +90,7 @@
 %left <ival> ','
 %right <ival> ASSIGNOP
 %right <ival> '?' ':'
-%nonassoc DOTDOT YADAYADA
+%nonassoc DOTDOT
 %left <ival> OROR DORDOR
 %left <ival> ANDAND
 %left <ival> BITOROP
@@ -481,6 +481,11 @@ barestmt:	PLUGSTMT
 			{
 			  $$ = $1;
 			}
+	|	YADAYADA ';'
+			{
+			  $$ = newLISTOP(OP_DIE, 0, newOP(OP_PUSHMARK, 0),
+				newSVOP(OP_CONST, 0, newSVpvs("Unimplemented")));
+			}
 	|	';'
 			{
 			  $$ = NULL;
@@ -1210,11 +1215,6 @@ term	:	termbinop
 			{ $$ = pmruntime($1, $4, $5, 1, $<ival>2); }
 	|	BAREWORD
 	|	listop
-	|	YADAYADA
-			{
-			  $$ = newLISTOP(OP_DIE, 0, newOP(OP_PUSHMARK, 0),
-				newSVOP(OP_CONST, 0, newSVpvs("Unimplemented")));
-			}
 	|	PLUGEXPR
 	;
 

perly.tab

@@ -51,6 +51,22 @@ XXX For a release on a stable branch, this section aspires to be:
 
 [ List each incompatible change as a =head2 entry ]
 
+=head2 Yada-yada is now strictly a statement
+
+By the time of its initial stable release in Perl 5.12, the C<...>
+(yada-yada) operator was explicitly intended to serve as a statement,
+not an expression.  However, the original implementation was confused
+on this point, leading to inconsistent parsing.  The operator was
+accidentally accepted in a few situations where it did not serve as a
+complete statement, such as
+
+    ... . "foo";
+    ... if $a < $b;
+
+The parsing has now been made consistent, permitting yada-yada only as
+a statement.  Affected code can use C<do{...}> to put a yada-yada into
+an arbitrary expression context.
+
 =head1 Deprecations
 
 XXX Any deprecated features, syntax, modules etc. should be listed here.

perly.y

@@ -754,12 +754,7 @@ X<whatever operator>
 X<triple-dot operator>
 
 Beginning in Perl 5.12, Perl accepts an ellipsis, "C<...>", as a
-placeholder for code that you haven't implemented yet.  This form of
-ellipsis, the unimplemented statement, should not be confused with the
-binary flip-flop C<...> operator.  One is a statement and the other an
-operator.  (Perl doesn't usually confuse them because usually Perl can tell
-whether it wants an operator or a statement, but see below for exceptions.)
-
+placeholder for code that you haven't implemented yet.
 When Perl 5.12 or later encounters an ellipsis statement, it parses this
 without error, but if and when you should actually try to execute it, Perl
 throws an exception with the text C<Unimplemented>:
@@ -771,8 +766,11 @@ throws an exception with the text C<Unimplemented>:
         say "I found an ellipsis!";
     }
 
-You can only use the elliptical statement to stand in for a
-complete statement.  These examples of how the ellipsis works:
+You can only use the elliptical statement to stand in for a complete
+statement.  Syntactically, "C<...;>" is a complete statement, but,
+as with other kinds of semicolon-terminated statement, the semicolon
+may be omitted if "C<...>" appears immediately before a closing brace.
+These examples show how the ellipsis works:
 
     use v5.12;
     { ... }
@@ -791,32 +789,25 @@ complete statement.  These examples of how the ellipsis works:
     };
 
 The elliptical statement cannot stand in for an expression that
-is part of a larger statement, since the C<...> is also the three-dot
-version of the flip-flop operator (see L<perlop/"Range Operators">).
-
+is part of a larger statement.
 These examples of attempts to use an ellipsis are syntax errors:
 
     use v5.12;
 
     print ...;
     open(my $fh, ">", "/dev/passwd") or ...;
     if ($condition && ... ) { say "Howdy" };
+    ... if $a > $b;
+    say "Cromulent" if ...;
+    $flub = 5 + ...;
 
 There are some cases where Perl can't immediately tell the difference
 between an expression and a statement.  For instance, the syntax for a
 block and an anonymous hash reference constructor look the same unless
 there's something in the braces to give Perl a hint.  The ellipsis is a
-syntax error if Perl doesn't guess that the C<{ ... }> is a block.  In that
-case, it doesn't think the C<...> is an ellipsis because it's expecting an
-expression instead of a statement:
-
-    @transformed = map { ... } @input;    # syntax error
-
+syntax error if Perl doesn't guess that the C<{ ... }> is a block.
 Inside your block, you can use a C<;> before the ellipsis to denote that the
-C<{ ... }> is a block and not a hash reference constructor.  Now the ellipsis
-works:
-
-    @transformed = map {; ... } @input;   # ';' disambiguates
+C<{ ... }> is a block and not a hash reference constructor.
 
 Note: Some folks colloquially refer to this bit of punctuation as a
 "yada-yada" or "triple-dot", but its true name

pod/perldelta.pod

@@ -8,7 +8,7 @@ BEGIN {
 
 use strict;
 
-plan 12;
+plan 34;
 
 my $err;
 my $err1 = "Unimplemented at $0 line ";
@@ -19,6 +19,11 @@ eval { ... };
 is $@, $err, "Execution of ellipsis statement reported 'Unimplemented' code";
 $@ = '';
 
+my $i = 0;
+is eval { $i++; ...; $i+=10; 123 }, undef;
+like $@, qr/\AUnimplemented /;
+is $i, 1;
+
 note("RT #122661: Semicolon before ellipsis statement disambiguates to indicate block rather than hash reference");
 my @input = (3..5);
 my @transformed;
@@ -42,21 +47,19 @@ eval { @transformed = map {;... } @input; };
 is $@, $err, "Disambiguation case 4";
 $@ = '';
 
-note("RT #132150: ... is a term, not an operator");
-$err = $err1 . ( __LINE__ + 1 ) . $err2;
-eval { ... + 0 };
-is $@, $err, "... + 0 parses";
-$@ = '';
-
-$err = $err1 . ( __LINE__ + 1 ) . $err2;
-eval { ... % 1 };
-is $@, $err, "... % 1 parses";
-$@ = '';
-
-$err = $err1 . ( __LINE__ + 1 ) . $err2;
-eval { ... / 1 };
-is $@, $err, "... / 1 parses";
-$@ = '';
+note("RT #132150: ... in other contexts is a syntax error");
+foreach(
+	"... + 0", "0 + ...",
+	"... . 0", "0 . ...",
+	"... or 1", "1 or ...",
+	"... if 1", "1 if ...",
+	'[...]',
+	'my $a = ...',
+	'... sub quux {}',
+) {
+	is eval($_), undef;
+	like $@, qr/\Asyntax error /;
+}
 
 #
 # Regression tests, making sure ... is still parsable as an operator.

pod/perlsyn.pod

@@ -6887,7 +6887,7 @@ Perl_yylex(pTHX)
 	}
 	if (PL_expect == XSTATE && s[1] == '.' && s[2] == '.') {
 	    s += 3;
-	    TERM(YADAYADA);
+	    OPERATOR(YADAYADA);
 	}
 	if (PL_expect == XOPERATOR || !isDIGIT(s[1])) {
 	    char tmp = *s++;