Assertion `!(o->op_private & ~PL_op_private_valid[type])' failed

[p5pRT]

Migrated from rt.perl.org#129320 (status was 'resolved')

Searchable as RT129320$

[p5pRT]

From @geeknik

Triggered in Perl v5.25.5 (v5.25.4-110-g95c0a76).

./perl -e 'sub{map{0}0..~0'
Range iterator outside integer range at -e line 1.
perl​: op.c​:791​: void Perl_op_free(OP *)​: Assertion `!(o->op_private & ~PL_op_private_valid[type])' failed.
Aborted

[p5pRT]

From @hvds

On Tue, 20 Sep 2016 13​:39​:11 -0700, brian.carpenter@​gmail.com wrote​:

Triggered in Perl v5.25.5 (v5.25.4-110-g95c0a76).

./perl -e 'sub{map{0}0..~0'
Range iterator outside integer range at -e line 1.
perl​: op.c​:791​: void Perl_op_free(OP *)​: Assertion `!(o->op_private &
~PL_op_private_valid[type])' failed.
Aborted

op_private is "by default, set to the number of children until the operation is privatized by a check routine".

In this case we're in ck_grep() for the MAPSTART op, and op_private has been set to 2 for the number of children; however we see the range error and die (and thus try to free the ops) before that gets far enough to reset it at op.c​:10304​:
  o->op_private = gwop->op_private = 0;

Here are the parts of the stack trace I think are relevant​:

#3 0x00007ffff72e7ca2 in __GI___assert_fail (
  assertion=0x720f60 "!(o->op_private & ~PL_op_private_valid[type])",
  file=0x7204ee "op.c", line=791,
  function=0x72e413 <__PRETTY_FUNCTION__.15100> "Perl_op_free")
  at assert.c​:101
#4 0x00000000004073fa in Perl_op_free (o=0xa6bad8) at op.c​:791
[...]
#17 0x00000000005408d5 in Perl_die (
  pat=0x7d8ed0 "Range iterator outside integer range") at util.c​:1736
#18 0x000000000063f182 in Perl_pp_flop () at pp_ctl.c​:1187
#19 0x000000000053a7ba in Perl_runops_debug () at dump.c​:2260
#20 0x0000000000413494 in S_gen_constant_list (o=0xa6bb60) at op.c​:4581
#21 0x000000000040aa3c in Perl_list (o=0xa6bb60) at op.c​:2295
#22 0x000000000040a6a4 in S_listkids (o=0xa6bad8) at op.c​:2234
#23 0x000000000042f487 in Perl_ck_fun (o=0xa6bad8) at op.c​:10205
#24 0x000000000042f855 in Perl_ck_grep (o=0xa6bad8) at op.c​:10294
[...]

I don't know why setting op_private to numargs doesn't cause problems like this more regularly, or how this sort of thing should be fixed.

Hugo

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @iabyn

On Tue, Dec 06, 2016 at 05​:40​:36AM -0800, Hugo van der Sanden via RT wrote​:

On Tue, 20 Sep 2016 13​:39​:11 -0700, brian.carpenter@​gmail.com wrote​:

Triggered in Perl v5.25.5 (v5.25.4-110-g95c0a76).

./perl -e 'sub{map{0}0..~0'
Range iterator outside integer range at -e line 1.
perl​: op.c​:791​: void Perl_op_free(OP *)​: Assertion `!(o->op_private &
~PL_op_private_valid[type])' failed.
Aborted

op_private is "by default, set to the number of children until the operation is privatized by a check routine".

In this case we're in ck_grep() for the MAPSTART op, and op_private has been set to 2 for the number of children; however we see the range error and die (and thus try to free the ops) before that gets far enough to reset it at op.c​:10304​:
o->op_private = gwop->op_private = 0;

Here are the parts of the stack trace I think are relevant​:

#3 0x00007ffff72e7ca2 in __GI___assert_fail (
assertion=0x720f60 "!(o->op_private & ~PL_op_private_valid[type])",
file=0x7204ee "op.c", line=791,
function=0x72e413 <__PRETTY_FUNCTION__.15100> "Perl_op_free")
at assert.c​:101
#4 0x00000000004073fa in Perl_op_free (o=0xa6bad8) at op.c​:791
[...]
#17 0x00000000005408d5 in Perl_die (
pat=0x7d8ed0 "Range iterator outside integer range") at util.c​:1736
#18 0x000000000063f182 in Perl_pp_flop () at pp_ctl.c​:1187
#19 0x000000000053a7ba in Perl_runops_debug () at dump.c​:2260
#20 0x0000000000413494 in S_gen_constant_list (o=0xa6bb60) at op.c​:4581
#21 0x000000000040aa3c in Perl_list (o=0xa6bb60) at op.c​:2295
#22 0x000000000040a6a4 in S_listkids (o=0xa6bad8) at op.c​:2234
#23 0x000000000042f487 in Perl_ck_fun (o=0xa6bad8) at op.c​:10205
#24 0x000000000042f855 in Perl_ck_grep (o=0xa6bad8) at op.c​:10294
[...]

I don't know why setting op_private to numargs doesn't cause problems like this more regularly, or how this sort of thing should be fixed.

I haven't looked closely at this particular issue, but that assert isn't
supposed to be checked if we have errored, i.e.

  Perl_op_free(pTHX_ OP *o)
  {
  ...
  #ifdef DEBUGGING
  if ( o->op_ppaddr == PL_ppaddr[o->op_type]
  && PL_parser
  && !PL_parser->error_count)
  {
  assert(!(o->op_private & ~PL_op_private_valid[type]));
  }
  #endif

so perhaps the issue here is why PL_parser->error_count isn't true at that
point.

--
"I do not resent criticism, even when, for the sake of emphasis,
it parts for the time with reality".
  -- Winston Churchill, House of Commons, 22nd Jan 1941.

[p5pRT]

From @hvds

On Tue, 06 Dec 2016 06​:15​:08 -0800, davem wrote​:

I haven't looked closely at this particular issue, but that assert
isn't supposed to be checked if we have errored, i.e.

Perl_op_free(pTHX_ OP *o)
{
...
#ifdef DEBUGGING
if ( o->op_ppaddr == PL_ppaddr[o->op_type]
&& PL_parser
&& !PL_parser->error_count)
{
assert(!(o->op_private & ~PL_op_private_valid[type]));
}
#endif

so perhaps the issue here is why PL_parser->error_count isn't true at
that point.

Ah, good point, I missed that. I'll look more this evening.

Hugo

[p5pRT]

From @hvds

On Tue, 06 Dec 2016 06​:15​:08 -0800, davem wrote​:

so perhaps the issue here is why PL_parser->error_count isn't true at that
point.

It looks like error_count is only touched by Perl_qerror(), which is called in only a handful of places - and I guess it's intended only for compile errors. This particular error is a runtime error, triggered at this point by constant folding.

Given that a runtime error can kill us mid-compile, we could make the op_free check require IN_PERL_RUNTIME - but my gut says that's a lot too broad, where the current check is only slightly too narrow.

I think the right answer is not to let it happen in the first place. Back in 2006 (commit b7f7fd0), constant folding was (effectively) wrapped in an eval{} so we could defer till runtime if it raised an error. I'm not sure why the same wasn't applied to gen_constant_list() at the time, but it feels like it's needed here for the same reason, and it'd also handily fix this problem.

The test-case (and primary justification) for that is that neither of these should die​:

% perl -le 'if (0) { $a = 1 / 0 } print "fold_constants ok"'
fold_constants ok
% perl -le 'if (0) { @​a = 0..~0 } print "gen_constant_list ok"'
Range iterator outside integer range at -e line 1.
%

Hugo

[p5pRT]

From zefram@fysh.org

Fixed by 3a0fe09.

-zefram

[p5pRT]

@iabyn - Status changed from 'open' to 'pending release'

[p5pRT]

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.26.0, this and 210 other issues have been
resolved.

Perl 5.26.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.26.0

If you find that the problem persists, feel free to reopen this ticket.

[p5pRT]

@khwilliamson - Status changed from 'pending release' to 'resolved'