@_ gets corrupted when F(@X) shortens @X

[p5pRT]

Migrated from rt.perl.org#8358 (status was 'open')

Searchable as RT8358$

[p5pRT]

From vallon@bear.com

Created by vallon@bear.com

(Also reproduced in 5.005)

SAMPLE CODE​: Here is a sample program​:
 
  local @​X;
  local @​Y;

  sub F {
  print join(',', @​_), "\n";
  @​X = ();
  print join(',', @​_), "\n";
  @​Y = qw/ magic /;
  print join(',', @​_), "\n";
  }
 
  @​X = qw/ A B C D E /;
  F('start', @​X, 'end');

With the curious output​:

  start,A,B,C,D,E,end
  start,,,,,,end
  start,magic,,,,,end

BUG​: The bug is that in F, after @​X is emptied, @​_ seems to contain dangling
references. The allocation of 'magic' seems to reuse some recently released
memory, and @​_ (still) references that memory.

EXPECTED BEHAVIOR​: How is this supposed to work? What should @​_ reference if @​X
gets shorter? My guess would be that it evaluates to undef for @​_ elements that
reference truncated values of @​X, and assignment to non-existent values
automatically extends @​X (ala lvalue array entries).

BIGGER EXAMPLE​: In the following, @​_ ends up referencing something that causes a
croak from within Perl_sv_setsv ("Bizarre copy of..."). As with all memory
corruption, actual behavior is undefined.

#!/usr/local/bin/perl5.6.1 -w

local @​X;

sub Main {
  print 'orig​: ', join(',', @​_), "\n"; # 'start', 'A'..'M', 'end'

  @​X = ();

  print 'clear​: ', join(',', @​_), "\n"; # @​_ seems ok, but isn't

  my @​z = qw/ MAGIC STRINGS /;

  print 'magic​: ', join(',', @​_), "\n"; # oops, dangling pointer???

  require Carp; # massage heap

  print 'trash​: ', join(',', @​_), "\n"; # oops, more trash from load

  my @​a = @​_; # crash (maybe?) : Bizarre copy of CODE in aassign at ./test2 line 9.
}

@​X = qw/ A B C D E F G H I J K L M /;
Main('start', @​X, 'end');

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl v5.6.1:

Configured by software at Wed Dec 12 09:52:28 EST 2001.

Summary of my perl5 (revision 5.0 version 6 subversion 1) configuration:
  Platform:
    osname=hpux, osvers=10.20, archname=PA-RISC2.0-multi
    uname='hp-ux devsrv1 b.10.20 a 9000785 2004553983 two-user license '
    config_args='-d -e -Dperl5=/usr/local/bin/perl5 -Dprefix=/usr/local/public/perl-5.6.1 -Uusemymalloc -Uinstallusrbinperl -Dusemultiplicity -Dccflags=-Ae -O +DAportable +DS2.0a +ESlit +Oentrysched +Ofastaccess +Olibcalls +Onolimit '
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=define
    useperlio=undef d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
  Compiler:
    cc='cc', ccflags =' -Ae -O +DAportable +DS2.0a +ESlit +Oentrysched +Ofastaccess +Olibcalls +Onolimit -D_HPUX_SOURCE -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 ',
    optimize='-O',
    cppflags='-Ae -O +DAportable +DS2.0a +ESlit +Oentrysched +Ofastaccess +Olibcalls +Onolimit -D_HPUX_SOURCE -Aa -I/usr/local/include'
    ccversion='A.10.32.30', gccversion='', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=4321
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, usemymalloc=n, prototype=define
  Linker and Libraries:
    ld='ld', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib /usr/ccs/lib
    libs=-lnsl_s -lndbm -lgdbm -lmalloc -ldld -lm -lc -lndir -lcrypt -lsec
    perllibs=-lnsl_s -lmalloc -ldld -lm -lc -lndir -lcrypt -lsec
    libc=/lib/libc.sl, so=sl, useshrplib=false, libperl=libperl.a
  Dynamic Linking:
    dlsrc=dl_hpux.xs, dlext=sl, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-B,deferred '
    cccdlflags='+z', lddlflags='-b +vnocompatwarnings -L/usr/local/lib'

Locally applied patches:
    


@INC for perl v5.6.1:
    /usr/derivs/public/lib/perl
    /a/vallon/lib/perl
    /usr/local/public/perl-5.6.1/lib/5.6.1/PA-RISC2.0-multi
    /usr/local/public/perl-5.6.1/lib/5.6.1
    /usr/local/public/perl-5.6.1/lib/site_perl/5.6.1/PA-RISC2.0-multi
    /usr/local/public/perl-5.6.1/lib/site_perl/5.6.1
    /usr/local/public/perl-5.6.1/lib/site_perl
    .


Environment for perl v5.6.1:
    HOME=/a/vallon
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/derivs/public/bin:/a/vallon/jv/bin:/usr/local/licensed/autosys/HP-UX/autosys/bin:/a/vallon/bin:/a/vallon/import/bin:/a/vallon/public/bin:/usr/local/bin:/usr/bin/X11:/usr/local/bin/X11:/usr/ucb:/bin:/usr/bin:/opt/softbench/bin:/usr/bin:/usr/ccs/bin:/usr/contrib/bin:/opt/ansic/bin:/opt/nettladm/bin:/opt/pd/bin:/opt/upgrade/bin:/usr/bin/X11:/usr/contrib/bin/X11:/opt/hparray/bin:/opt/resmon/bin:/opt/perf/bin:/opt/langtools/bin:/opt/imake/bin:/opt/aCC/bin:/opt/fortran/bin:/usr/local/X11R5/bin:/usr/atria/bin:/usr/derivs/prod/bin:/usr/derivs/apps/bin:/usr/local/sybase-11.1.1/bin:/a/derivs/bin
    PERL5LIB=/usr/derivs/public/lib/perl:/a/vallon/lib/perl
    PERL_BADLANG (unset)
    SHELL=/bin/ksh
    SHLIB_PATH (unset)


�****************************************************************
Bear Stearns is not responsible for any recommendation, solicitation, 
offer or agreement or any information about any transaction, customer 
account or account activity contained in this communication.
***********************************************************************

[p5pRT]

From @schwern

Confirmed from bleadperl back to 5.003_07 at least.

On Mon, Jan 28, 2002 at 01​:30​:13PM -0500, Justin Vallon wrote​:

This is a bug report for perl from vallon@​bear.com,
generated with the help of perlbug 1.33 running under perl v5.6.1.

-----------------------------------------------------------------
[Please enter your report here]

(Also reproduced in 5.005)

SAMPLE CODE​: Here is a sample program​:

local @​X;
local @​Y;

The local doesn't appear to be necessary for the bug to manifest.

sub F \{
                      print join\('\,'\, @​\_\)\, "\\n";
    @​X = \(\);
                      print join\('\,'\, @​\_\)\, "\\n";
    @​Y = qw/ magic /;
                      print join\('\,'\, @​\_\)\, "\\n";
\}

@​X = qw/ A B C D E /;
F\('start'\, @​X\, 'end'\);

With the curious output​:

start\,A\,B\,C\,D\,E\,end
start\,\,\,\,\,\,end
start\,magic\,\,\,\,\,end

BUG​: The bug is that in F, after @​X is emptied, @​_ seems to contain dangling
references. The allocation of 'magic' seems to reuse some recently released
memory, and @​_ (still) references that memory.

EXPECTED BEHAVIOR​: How is this supposed to work? What should @​_ reference if @​X
gets shorter? My guess would be that it evaluates to undef for @​_ elements that
reference truncated values of @​X, and assignment to non-existent values
automatically extends @​X (ala lvalue array entries).

BIGGER EXAMPLE​: In the following, @​_ ends up referencing something that causes a
croak from within Perl_sv_setsv ("Bizarre copy of..."). As with all memory
corruption, actual behavior is undefined.

#!/usr/local/bin/perl5.6.1 -w

local @​X;

sub Main {
print 'orig​: ', join(',', @​_), "\n"; # 'start', 'A'..'M', 'end'

@​X = \(\);

           print 'clear​: '\, join\('\,'\, @​\_\)\, "\\n"; \# @​\_ seems ok\, but isn't

my @​z = qw/ MAGIC STRINGS /;

           print 'magic​: '\, join\('\,'\, @​\_\)\, "\\n"; \# oops\, dangling pointer???

require Carp; \# massage heap

           print 'trash​: '\, join\('\,'\, @​\_\)\, "\\n"; \# oops\, more trash from load

my @​a = @​\_; \# crash \(maybe?\) : Bizarre copy of CODE in aassign at \./test2 line 9\.

}

@​X = qw/ A B C D E F G H I J K L M /;
Main('start', @​X, 'end');

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
category=core
severity=medium
---
Site configuration information for perl v5.6.1​:

Configured by software at Wed Dec 12 09​:52​:28 EST 2001.

Summary of my perl5 (revision 5.0 version 6 subversion 1) configuration​:
Platform​:
osname=hpux, osvers=10.20, archname=PA-RISC2.0-multi
uname='hp-ux devsrv1 b.10.20 a 9000785 2004553983 two-user license '
config_args='-d -e -Dperl5=/usr/local/bin/perl5 -Dprefix=/usr/local/public/perl-5.6.1 -Uusemymalloc -Uinstallusrbinperl -Dusemultiplicity -Dccflags=-Ae -O +DAportable +DS2.0a +ESlit +Oentrysched +Ofastaccess +Olibcalls +Onolimit '
hint=recommended, useposix=true, d_sigaction=define
usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=define
useperlio=undef d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=undef use64bitall=undef uselongdouble=undef
Compiler​:
cc='cc', ccflags =' -Ae -O +DAportable +DS2.0a +ESlit +Oentrysched +Ofastaccess +Olibcalls +Onolimit -D_HPUX_SOURCE -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 ',
optimize='-O',
cppflags='-Ae -O +DAportable +DS2.0a +ESlit +Oentrysched +Ofastaccess +Olibcalls +Onolimit -D_HPUX_SOURCE -Aa -I/usr/local/include'
ccversion='A.10.32.30', gccversion='', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=4321
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, usemymalloc=n, prototype=define
Linker and Libraries​:
ld='ld', ldflags =' -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib /usr/ccs/lib
libs=-lnsl_s -lndbm -lgdbm -lmalloc -ldld -lm -lc -lndir -lcrypt -lsec
perllibs=-lnsl_s -lmalloc -ldld -lm -lc -lndir -lcrypt -lsec
libc=/lib/libc.sl, so=sl, useshrplib=false, libperl=libperl.a
Dynamic Linking​:
dlsrc=dl_hpux.xs, dlext=sl, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-B,deferred '
cccdlflags='+z', lddlflags='-b +vnocompatwarnings -L/usr/local/lib'

Locally applied patches​:

---
@​INC for perl v5.6.1​:
/usr/derivs/public/lib/perl
/a/vallon/lib/perl
/usr/local/public/perl-5.6.1/lib/5.6.1/PA-RISC2.0-multi
/usr/local/public/perl-5.6.1/lib/5.6.1
/usr/local/public/perl-5.6.1/lib/site_perl/5.6.1/PA-RISC2.0-multi
/usr/local/public/perl-5.6.1/lib/site_perl/5.6.1
/usr/local/public/perl-5.6.1/lib/site_perl
.

---
Environment for perl v5.6.1​:
HOME=/a/vallon
LANG (unset)
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/usr/derivs/public/bin​:/a/vallon/jv/bin​:/usr/local/licensed/autosys/HP-UX/autosys/bin​:/a/vallon/bin​:/a/vallon/import/bin​:/a/vallon/public/bin​:/usr/local/bin​:/usr/bin/X11​:/usr/local/bin/X11​:/usr/ucb​:/bin​:/usr/bin​:/opt/softbench/bin​:/usr/bin​:/usr/ccs/bin​:/usr/contrib/bin​:/opt/ansic/bin​:/opt/nettladm/bin​:/opt/pd/bin​:/opt/upgrade/bin​:/usr/bin/X11​:/usr/contrib/bin/X11​:/opt/hparray/bin​:/opt/resmon/bin​:/opt/perf/bin​:/opt/langtools/bin​:/opt/imake/bin​:/opt/aCC/bin​:/opt/fortran/bin​:/usr/local/X11R5/bin​:/usr/atria/bin​:/usr/derivs/prod/bin​:/usr/derivs/apps/bin​:/usr/local/sybase-11.1.1/bin​:/a/derivs/bin
PERL5LIB=/usr/derivs/public/lib/perl​:/a/vallon/lib/perl
PERL_BADLANG (unset)
SHELL=/bin/ksh
SHLIB_PATH (unset)

�****************************************************************
Bear Stearns is not responsible for any recommendation, solicitation,
offer or agreement or any information about any transaction, customer
account or account activity contained in this communication.
***********************************************************************

--

Michael G. Schwern <schwern@​pobox.com> http​://www.pobox.com/~schwern/
Perl Quality Assurance <perl-qa@​perl.org> Kwalitee Is Job One
The eye opening delightful morning taste of expired cheese bits in sour milk!

[p5pRT]

From [Unknown Contact. See original ticket]

On Jan 28, Michael G Schwern said​:

Confirmed from bleadperl back to 5.003_07 at least.

I'm not sure this is a bug.

The elements of @​_ are aliases to the elements passed to the function.

  ($x, $y, $z) = (1,2,3);
  foo($x, $y, $z);

  sub foo {
  print "[", join("][", @​_), "]\n"; # [1][2][3]
  ($x, $y, $z) = ($y, $z, $x);
  print "[", join("][", @​_), "]\n"; # [2][3][1]
  }

@​_ is its separate array, it holds its own elements, albeit aliases to
other scalars. Its size remains the same until you change its size, but
the elements can end up being different when they are changed from their
sources, just as the sources can change when the elements of @​_ are
changed​:

  $x = "japhy";
  $c = prechop($x);

  sub prechop { substr($_[0], 0, 1, '') }
  print "$c $x\n"; # j aphy

This seems consistent.

--
Jeff "japhy" Pinyan japhy@​pobox.com http​://www.pobox.com/~japhy/
RPI Acacia brother #734 http​://www.perlmonks.org/ http​://www.cpan.org/
** Look for "Regular Expressions in Perl" published by Manning, in 2002 **
<stu> what does y/// stand for? <tenderpuss> why, yansliterate of course.

[p5pRT]

From @ysth

I'm not sure this is a bug.

The elements of @​_ are aliases to the elements passed to the function.

True 'nuf. But look again and explain why changing @​Y affects @​_.
(Also note the comment about perl croaking in the OPs second example.)

sub F \{
                      print join\('\,'\, @​\_\)\, "\\n";
    @​X = \(\);
                      print join\('\,'\, @​\_\)\, "\\n";
    @​Y = qw/ magic /;
                      print join\('\,'\, @​\_\)\, "\\n";
\}
@​X = qw/ A B C D E /;
F\('start'\, @​X\, 'end'\);

With the curious output​:
start,A,B,C,D,E,end
start,,,,,,end
start,magic,,,,,end

[p5pRT]

From [Unknown Contact. See original ticket]

On Jan 28, Yitzchak Scott-Thoennes said​:

I'm not sure this is a bug.

The elements of @​_ are aliases to the elements passed to the function.

True 'nuf. But look again and explain why changing @​Y affects @​_.
(Also note the comment about perl croaking in the OPs second example.)

sub F \{
                      print join\('\,'\, @​\_\)\, "\\n";
    @​X = \(\);
                      print join\('\,'\, @​\_\)\, "\\n";
    @​Y = qw/ magic /;
                      print join\('\,'\, @​\_\)\, "\\n";
\}
@​X = qw/ A B C D E /;
F\('start'\, @​X\, 'end'\);

With the curious output​:
start,A,B,C,D,E,end
start,,,,,,end
start,magic,,,,,end

Well, it doesn't appear to futz up in bleadperl...

--
Jeff "japhy" Pinyan japhy@​pobox.com http​://www.pobox.com/~japhy/
RPI Acacia brother #734 http​://www.perlmonks.org/ http​://www.cpan.org/
** Look for "Regular Expressions in Perl" published by Manning, in 2002 **
<stu> what does y/// stand for? <tenderpuss> why, yansliterate of course.

[p5pRT]

From @schwern

On Mon, Jan 28, 2002 at 10​:18​:43PM -0500, Jeff 'japhy' Pinyan wrote​:

On Jan 28, Yitzchak Scott-Thoennes said​:

I'm not sure this is a bug.

The elements of @​_ are aliases to the elements passed to the function.

True 'nuf. But look again and explain why changing @​Y affects @​_.
(Also note the comment about perl croaking in the OPs second example.)

sub F \{
                      print join\('\,'\, @​\_\)\, "\\n";
    @​X = \(\);
                      print join\('\,'\, @​\_\)\, "\\n";
    @​Y = qw/ magic /;
                      print join\('\,'\, @​\_\)\, "\\n";
\}
@​X = qw/ A B C D E /;
F\('start'\, @​X\, 'end'\);

With the curious output​:
start,A,B,C,D,E,end
start,,,,,,end
start,magic,,,,,end

Well, it doesn't appear to futz up in bleadperl...

It does @​14190.

--

Michael G. Schwern <schwern@​pobox.com> http​://www.pobox.com/~schwern/
Perl Quality Assurance <perl-qa@​perl.org> Kwalitee Is Job One
"His plagiarism was limited only by this faulty technique."
  -- Peter Schickele

[p5pRT]

From @iabyn

sub F \{
                      print join\('\,'\, @​\_\)\, "\\n";
    @​X = \(\);
                      print join\('\,'\, @​\_\)\, "\\n";
    @​Y = qw/ magic /;
                      print join\('\,'\, @​\_\)\, "\\n";
\}
@​X = qw/ A B C D E /;
F\('start'\, @​X\, 'end'\);

With the curious output​:
start,A,B,C,D,E,end
start,,,,,,end
start,magic,,,,,end

Yes it's a bug, and no, it can't easily be fixed. It's all because stuff
pushed onto the stack (and copied into @​_) isn't reference counted.

The individual SVs that make up the elements of @​X start off with a refcnt
of 1. They are then pushed onto the stack (still with refcnt == 1),
then the pointers on the stack are copied into @​_. So now both @​X and @​_
have pointers to the SVs (but with refcnt == 1). Now the elements of
@​X are deleted​: since they have a refcnt of 1, the are freed up.
@​_ now contains pointers to SVs in the free SV slabs !!! When a new SV is
created ("magic"), one of the free SVs is grabbed, which happens to be the
same as that pointed to by the first slot in @​_.
Bad, bad, bad.

Fixed in Perl 6? :-(

[p5pRT]

From [Unknown Contact. See original ticket]

However, in my sample code, @​_ is corrupt, not modified. If you pass an
array (@​X) to your foo function, and the array (@​X) is shortened by
side-effect in foo, then the elements of @​_ that pointed to now-truncated
elements of the array point to corrupt memory. You may core dump. That is
a bug.

It appears that the aliasing of the elements that occurs when @​_ is created
becomes corrupted if the aliases arrays are shorted.

-Justin
vallon@​bear.com

-----Original Message-----
From​: Jeff 'japhy' Pinyan [SMTP​:jeffp@​crusoe.net]
Sent​: Monday, January 28, 2002 8​:34 PM
To​: Michael G Schwern
Cc​: Justin Vallon; perl5-porters@​perl.org
Subject​: Re​: [ID 20020128.003] @​_ gets corrupted when F(@​X) shortens
@​X

On Jan 28, Michael G Schwern said​:

Confirmed from bleadperl back to 5.003_07 at least.

I'm not sure this is a bug.

The elements of @​_ are aliases to the elements passed to the function.

  ($x, $y, $z) = (1,2,3);
  foo($x, $y, $z);

  sub foo {
  print "[", join("][", @​_), "]\n"; # [1][2][3]
  ($x, $y, $z) = ($y, $z, $x);
  print "[", join("][", @​_), "]\n"; # [2][3][1]
  }

@​_ is its separate array, it holds its own elements, albeit aliases to
other scalars. Its size remains the same until you change its size, but
the elements can end up being different when they are changed from their
sources, just as the sources can change when the elements of @​_ are
changed​:

  $x = "japhy";
  $c = prechop($x);

  sub prechop { substr($_[0], 0, 1, '') }
  print "$c $x\n"; # j aphy

This seems consistent.

--
Jeff "japhy" Pinyan japhy@​pobox.com http​://www.pobox.com/~japhy/
RPI Acacia brother #734 http​://www.perlmonks.org/ http​://www.cpan.org/
** Look for "Regular Expressions in Perl" published by Manning, in 2002 **
<stu> what does y/// stand for? <tenderpuss> why, yansliterate of course.

***********************************************************************
Bear Stearns is not responsible for any recommendation, solicitation,
offer or agreement or any information about any transaction, customer
account or account activity contained in this communication.
***********************************************************************