perl segfaults when freeing deeply nested structures

[p5pRT]

Migrated from rt.perl.org#44225 (status was 'resolved')

Searchable as RT44225$

[p5pRT]

From perl-5.8.0@ton.iguana.be

Created by perl-5.8.0@ton.iguana.be

The following program builds a double linked list, and then
destroys it again. On exit it dumps core. Effect happens in
many but not all versions off 5.8.0, no problems in 5.6.1.

Increasing the 10000 to 1000000 makes more perl versions
crash (e.g. my 5.6.1 begins crashing too). That version
uses about 160M for me, so it's no out of memory effect.

I was unable to reduce the program further.

#! /usr/bin/perl -w
my $head = {};
$head->{"next"} = $head;
$head->{"previous"} = $head;

for (1..10000) {
  my $node = {};

  # attach node in front
  $prev = $head;
  my $next = $prev->{"next"};
  $prev->{"next"} = $node;
  $node->{"next"} = $next;
  $node->{"previous"} = $prev;
  $next->{"previous"} = $node;
}

# Make nodes single connected
$prev = $head;
my $next = $prev->{"next"};
while ($next != $head) {
  undef($next->{"previous"});
  $prev = $next;
  $next = $prev->{"next"};
}
undef($prev->{"next"});

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl v5.8.0:

Configured by ton at Tue Nov 12 01:56:18 CET 2002.

Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration:
  Platform:
    osname=linux, osvers=2.4.19, archname=i686-linux-thread-multi-64int-ld
    uname='linux quasar 2.4.19 #5 wed oct 2 02:34:25 cest 2002 i686 unknown '
    config_args=''
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=define use64bitall=undef uselongdouble=define
    usemymalloc=y, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -fomit-frame-pointer',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -I/usr/local/include'
    ccversion='', gccversion='2.95.3 20010315 (release)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long long', ivsize=8, nvtype='long double', nvsize=12, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lndbm -ldb -ldl -lm -lpthread -lc -lposix -lcrypt -lutil
    perllibs=-lnsl -ldl -lm -lpthread -lc -lposix -lcrypt -lutil
    libc=/lib/libc-2.2.4.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.2.4'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
    cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    


@INC for perl v5.8.0:
    /usr/lib/perl5/5.8.0/i686-linux-thread-multi-64int-ld
    /usr/lib/perl5/5.8.0
    /usr/lib/perl5/site_perl/5.8.0/i686-linux-thread-multi-64int-ld
    /usr/lib/perl5/site_perl/5.8.0
    /usr/lib/perl5/site_perl
    .


Environment for perl v5.8.0:
    HOME=/home/ton
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/ton/bin.Linux:/home/ton/bin:/home/ton/bin.SampleSetup:/usr/local/bin:/usr/local/sbin:/usr/local/jre/bin:/home/oracle/product/9.0.1/bin:/usr/local/ar/bin:/usr/games/bin:/usr/X11R6/bin:/usr/share/bin:/usr/bin:/usr/sbin:/bin:/sbin:.
    PERL_BADLANG (unset)
    SHELL=/bin/bash

[p5pRT]

From enache@rdslink.ro

On Sat, May 03, 2003 at 02​:53​:08PM -0000, perl-5.8.0@​ton.iguana.be (via RT) wrote​:

The following program builds a double linked list, and then
destroys it again. On exit it dumps core. Effect happens in
many but not all versions off 5.8.0, no problems in 5.6.1.

Increasing the 10000 to 1000000 makes more perl versions
crash (e.g. my 5.6.1 begins crashing too). That version
uses about 160M for me, so it's no out of memory effect.

Simple hard stack overflow caused be deep recursion.
(22095.pl below is your test with 10000 changed to 100000).

Solution​: adjust your limits.

ex.

$ perl 22095.pl
Segmentation fault
$ ulimit -s 40000
$ perl 22095.pl
$

Regards,
Adi

[p5pRT]

From perl5-porters@ton.iguana.be

In article <20030505152443.GA2169@​ratsnest.hole>,
  Enache Adrian <enache@​rdslink.ro> writes​:

On Sat, May 03, 2003 at 02​:53​:08PM -0000, perl-5.8.0@​ton.iguana.be (via RT) wrote​:

The following program builds a double linked list, and then
destroys it again. On exit it dumps core. Effect happens in
many but not all versions off 5.8.0, no problems in 5.6.1.

Increasing the 10000 to 1000000 makes more perl versions
crash (e.g. my 5.6.1 begins crashing too). That version
uses about 160M for me, so it's no out of memory effect.

Simple hard stack overflow caused be deep recursion.
(22095.pl below is your test with 10000 changed to 100000).

Solution​: adjust your limits.

Mmm, since this happens in a module that should be always useable without
doing external things, that's not so great as a general solution. I'll
sever both links while walking the structure, which should also solve it
though at a speed hit.

Since my program has no recursion, I assume the recursion is
happening inside perl, probabably code doing something like​:

dec_thingy {
  return if --self->refcount > 0;
  for (self->all_references) {
  dec_thingy($_);
  }
  free(self);
}

which would then be called recursively for as many times as my linear
list is long ?

Since e.g. on linux the default stack limit I get is only 8192, maybe
worth considering making that iterative ? Otherwise this will generally
bite if you drop a deep structure.

This also begs the question why 5.6.1 on the same machine can handle
much bigger structures before failing.

[p5pRT]

From sky@nanisky.com

On måndag, maj 5, 2003, at 18​:50 Europe/Stockholm, Ton Hospel wrote​:

Since e.g. on linux the default stack limit I get is only 8192, maybe
worth considering making that iterative ? Otherwise this will generally
bite if you drop a deep structure.

Patches welcome :)

Arthur

[p5pRT]

From perl5-porters@ton.iguana.be

In article <61BDB378-81FE-11D7-9B8A-000393CB5BC4@​nanisky.com>,
  Arthur Bergman <sky@​nanisky.com> writes​:

On måndag, maj 5, 2003, at 18​:50 Europe/Stockholm, Ton Hospel wrote​:

Since e.g. on linux the default stack limit I get is only 8192, maybe
worth considering making that iterative ? Otherwise this will generally
bite if you drop a deep structure.

Patches welcome :)

Probably this thing should be looked into anyways even if it's kept
recursive. 8192 (the default stack limit on my linux box) is 8 Meg,
and the structure in my example is only 10000 entries. That seems
to imply about 800 bytes per frame, which is excessive.

[p5pRT]

From reto.stamm@xilinx.com

Created by reto@xilinx.com

The following script core dumps as it tries to collect $hash.

# start of script
{
  my $hash = {};
  my $in_hash = $hash;
  for (my $i = 0; $i < 250000; $i ++) {
  $in_hash->{INSIDE} = {};
  $in_hash = $in_hash->{INSIDE};
  }
}
print "All cleaned up now.";
# end of script

This seems to happen on 32 bit Linux and 32 bit Solaris.

Looking at the stack trace, it looks like it's analyzing the hash
recursively.

Perl Info

Flags:
    category=core
    severity=high

Site configuration information for perl v5.8.0:

Configured by bhcompile'
cf_email='bhcompile at Mon Sep 15 10:01:56 EDT 2003.

Summary of my rderl (revision 5.0 version 8 subversion 0) configuration:
  Platform:
    osname=linux, osvers=2.4.21-1.1931.2.393.entsmp, 
archname=i386-linux-thread-multi
    uname='linux por'
    config_args='-des -Doptimize=-O2 -g -pipe -march=i386 -mcpu=i686 
-Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red 
Hat, Inc. -Dinstallprefix=/usr -Dprefix=/usr -Darchname=i386-linux 
-Dvendorprefix=/usr -Dsiteprefix=/usr 
-Dotherlibdirs=/usr/lib/perl5/5.8.0 -Duseshrplib -Dusethreads 
-Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db 
-Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio 
-Dinstallusrbinperl -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less 
-isr'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef'
 useithreads=define usemultiplicity=
    useperlio= d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=un uselongdouble=
    usemymalloc=, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS 
-DDEBUGGING -fno-strict-aliasing -I/usr/local/include 
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
    optimize='',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING 
-fno-strict-aliasing -I/usr/local/include -I/usr/include/gdbm'
    ccversion='', gccversion='3.2.3 20030502 (Red Hat Linux 3.2.3-19)', 
gccosandvers=''
gccversion='3.2.3 200305'
    intsize=o, longsize=s, ptrsize=l, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long'
k', ivsize=4'
ivtype, nvtype='double'
o_no', nvsize=, Off_t='', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='gcc'
l', ldflags =' -L/usr/local/lib'
ldflags_use'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lgdbm -ldb -ldl -lm -lpthread -lc -lcrypt -lutil
    perllibs=
    libc=/lib/libc-2.3.2.so, so=so, useshrplib=true, libperl=libper
    gnulibc_version='2.3.2'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so', d_dlsymun=undef, ccdlflags='-rdynamic 
-Wl,-rpath,/usr/lib/perl5/5.8.0/i386-linux-thread-multi/CORE'
    cccdlflags='-fPIC'
ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5', lddlflags='s 
Unicode/Normalize XS/A'

Locally applied patches:
    MAINT18379


@INC for perl v5.8.0:
    /usr/lib/perl5/5.8.0/i386-linux-thread-multi
    /usr/lib/perl5/5.8.0
    /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.0
    /usr/lib/perl5/site_perl
    /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.0
    /usr/lib/perl5/vendor_perl
    /usr/lib/perl5/5.8.0/i386-linux-thread-multi
    /usr/lib/perl5/5.8.0
    .


Environment for perl v5.8.0:
    HOME=/home/reto
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LANGVAR=en_US.UTF-8
    
LD_LIBRARY_PATH=/home/reto/sb/HEAD/rtf/bin/lin:/build/xfndry/HEAD/rtf/bin/lin:/build/xfndry/HEAD/rtf/bin/lin
    LOGDIR (unset)
    
PATH=.:/devl/swtools:/home/reto/gnu/bin:/home/reto/gnu/firefox:/usr/X/bin:/usr/kerberos/bin:/home/reto/sb/HEAD/rtf/bin/lin:/build/xfndry/HEAD/rtf/bin/lin:/home/reto/sb/HEAD/env/bin/lin:/build/xfndry/HEAD/env/bin/lin:/home/reto/sb/HEAD/rtf:/build/xfndry/HEAD/rtf:/build/xfndry/HEAD/rtf/bin/lin:/devl/swtools:/bin:/usr/bin:/usr/local/bin
    PERL_BADLANG (unset)
    SHELL=/usr/local/bin/tcsh
    dlflags='-share (unset)

[p5pRT]

From chromatic@wgz.org

On Wednesday 22 March 2006 14​:43, Reto Stamm wrote​:

The following script core dumps as it tries to collect $hash.

# start of script
{
my $hash = {};
my $in_hash = $hash;
for (my $i = 0; $i < 250000; $i ++) {
$in_hash->{INSIDE} = {};
$in_hash = $in_hash->{INSIDE};
}
}
print "All cleaned up now.";
# end of script

This seems to happen on 32 bit Linux and 32 bit Solaris.

Looking at the stack trace, it looks like it's analyzing the hash
recursively.

Looking at the stack trace shows tens of thousands of calls. I think it's
recursing past what the C stack can handle. There may be no easy fix, though
rewriting the functions S_hfreeentries, Perl_hv_undef, Perl_sv_clear,
Perl_sv_free2, and Perl_hv_free_ent for iteration instead of recursion might
do it.

-- c

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From christian@pflanze.mine.nu

Created by christian@pflanze.mine.nu

See the code below which segfaults.

I don't see any cyclic reference, and running LengthOfStream
(SequenceStream 15000) in an endless loop runs just fine without
leaking, so cyclic references are not the problem.

It is clearly related to the stack size, increasing the stack limit
using ulimit raises the length of streams that can be processed.

It should be possible to create such streams of infinite length, being
built and deallocated while being traversed. This bug means that the
functional stream programming paradigm (or probably anything else
which creates closures in a repeated fashion?) cannot be applied to
perl 5.

use strict;

sub Delay ( & ) {
  [ 0, $_[0] ]
}

sub Force ( $ ) {
  my $p=shift;
  $$p[0] ? $$p[1]
  : do {
  $$p[1]= &{ $$p[1] };
  $$p[0]= 1;
  $$p[1]
  };
}

sub Cons ( $ $ ) {
  [ @​_ ]
}

sub Car ( $ ) {
  $_[0][0]
}

sub Cdr ( $ ) {
  $_[0][1]
}

our $Nil= [];

sub LengthOfStream ( $ ) {
  my ($strm)=@​_;
  my $len=0;
  while(1) {
  my $p= Force($strm);
  if ($p eq $Nil) {
  return $len;
  } else {
  $len++;
  $strm= Cdr($p);
  }
  }
}

sub SequenceStream ( $ ) {
  my ($n)=@​_;
  Delay {
  ($n < 0) ? $Nil
  #​: Cons $n, SequenceStream($n-1);
  : [ $n, SequenceStream($n-1) ];#no change
  };
}

if (0) {
  while(1) {
  print LengthOfStream (SequenceStream 15000),"\n";
  }
}
# above is no problem. but​:
if (0) {
  print LengthOfStream (SequenceStream 15000),"\n";
  print LengthOfStream (SequenceStream 15000),"\n";
  print LengthOfStream (SequenceStream 15000),"\n";
  print LengthOfStream (SequenceStream 25000),"\n";
  # now here a segfault happens
  print LengthOfStream (SequenceStream 25000),"\n";
}
if (1) {
  my $s= (SequenceStream 100000);
  print LengthOfStream $s ,"\n";
  print "Still alive\n";
  undef $s;# <-- segfaults
  print "Still alive?\n";
}

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl v5.8.7:

Configured by Debian Project at Thu Jun  9 00:28:22 EST 2005.

Summary of my perl5 (revision 5 version 8 subversion 7) configuration:
  Platform:
    osname=linux, osvers=2.4.27-ti1211, archname=i386-linux-thread-multi
    uname='linux kosh 2.4.27-ti1211 #1 sun sep 19 18:17:45 est 2004 i686 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i386-linux -Dprefix=/usr -Dprivlib=/usr/share/perl/5.8 -Darchlib=/usr/lib/perl/5.8 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.8.7 -Dsitearch=/usr/local/lib/perl/5.8.7 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dlibperl=libperl.so.5.8.7 -Dd_dosuid -des'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='3.3.6 (Debian 1:3.3.6-6)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=/lib/libc-2.3.2.so, so=so, useshrplib=true, libperl=libperl.so.5.8.7
    gnulibc_version='2.3.2'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    


@INC for perl v5.8.7:
    /etc/perl
    /usr/local/lib/perl/5.8.7
    /usr/local/share/perl/5.8.7
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.8
    /usr/share/perl/5.8
    /usr/local/lib/site_perl
    /usr/local/lib/perl/5.8.4
    /usr/local/share/perl/5.8.4
    /usr/local/lib/perl/5.8.3
    /usr/local/share/perl/5.8.3
    .


Environment for perl v5.8.7:
    HOME=/home/chris
    LANG=de_CH
    LANGUAGE (unset)
    LC_CTYPE=de_CH
    LC_NUMERIC=C
    LD_LIBRARY_PATH=:/home/chris/lib
    LOGDIR (unset)
    PATH=/usr/local/Gambit-C/bin:/opt/j2sdk_nb/j2sdk1.4.2/bin/:/home/chris/local/bin:/home/chris/bin:/root/local/bin:/root/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/bin/X11:/usr/local/sbin:/usr/local/bin:/sbin:/usr/sbin:/usr/local/sbin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

[p5pRT]

From christian@pflanze.mine.nu

  (It looks like the below text which I already sent as followup somehow
got lost, so I'm posting it through RT again.)

It seems like it might be a problem with parameter passing through @​_.
The following change prevents the segfault in all cases.

# either use this definition instead​:

# sub LengthOfStream ( $ ) {
# my ($strm)=@​_;
# #@​_=();# does not help
# undef $_[0];# <------ helps
# my $len=0;
# while(1) {
# my $p= Force($strm);
# if ($p eq $Nil) {
# return $len;
# } else {
# $len++;
# $strm= Cdr($p);
# }
# }
# }

# or this​:

sub LengthOfStream_ {
  my ($strm,$len)=@​_;
  #@​_=(); # does not help
  undef $_[0];# helps.
  my $p= Force($strm);
  if ($p eq $Nil) {
  $len
  } else {
  @​_=(Cdr($p), $len+1);
  goto \&LengthOfStream; # tail call
  }
}

sub LengthOfStream ( $ ) {
  push @​_,0;
  goto \&LengthOfStream_;
}

But it has, of course, the ugly side effect of erasing the variable
being passed by the caller​:

my $s= SequenceStream 100000;
print LengthOfStream $s,"\n"; # -> 100001
# but now $s is undef

I find it strange, that this case also requires the undef $_[0] (and
@​_=() is not enough!)​:

print LengthOfStream SequenceStream 100000,"\n";

Any idea?

What I also don't understand is why this leaks memory and eventually
terminates because of an out of memory condition (it should just run
forever)​:

sub InfiniteStream {
  my ($startnumber)=@​_;
  #undef $_[0]; Modification of a read-only value attempted -- of course.
  #@​_=();
  Delay {
  Cons $startnumber, InfiniteStream($startnumber+1);
  };
}

LengthOfStream InfiniteStream(0);

Any idea here?

Christian.

[p5pRT]

christian@pflanze.mine.nu - Status changed from 'new' to 'open'

[p5pRT]

From @iabyn

On Fri, Apr 28, 2006 at 04​:23​:49AM -0700, Christian wrote​:

See the code below which segfaults.

It's not actually closure-related; it's just that perl currently uses a
recursive method to free recursive data structures, which can blow the C
stack on deeply nested ones. For example the following will crash perl
just as happily as your example​:

  my $s;
  for (1..100_000) {
  $s = [$_, $s ];
  }
  undef $s;

It's on our list of things to fix one day...

--
Spock (or Data) is fired from his high-ranking position for not being able
to understand the most basic nuances of about one in three sentences that
anyone says to him.
  -- Things That Never Happen in "Star Trek" #19

[p5pRT]

From @iabyn

On Fri, Apr 28, 2006 at 02​:25​:25PM +0200, Christian wrote​:

It seems like it might be a problem with parameter passing through
@​_. The following change prevents the segfault in all cases.
sub LengthOfStream_ {
my ($strm,$len)=@​_;
#@​_=(); # does not help
undef $_[0];# helps.

This is avoiding the segfault by avoidly building a recursive data
structure.

--
The optimist believes that he lives in the best of all possible worlds.
As does the pessimist.

[p5pRT]

From christian@pflanze.mine.nu

It seems like it might be a problem with parameter passing through
@​_. The following change prevents the segfault in all cases.

# either use this definition instead​:

# sub LengthOfStream ( $ ) {
# my ($strm)=@​_;
# #@​_=();# does not help
# undef $_[0];# <------ helps
# my $len=0;
# while(1) {
# my $p= Force($strm);
# if ($p eq $Nil) {
# return $len;
# } else {
# $len++;
# $strm= Cdr($p);
# }
# }
# }

# or this​:

sub LengthOfStream_ {
  my ($strm,$len)=@​_;
  #@​_=(); # does not help
  undef $_[0];# helps.
  my $p= Force($strm);
  if ($p eq $Nil) {
  $len
  } else {
  @​_=(Cdr($p), $len+1);
  goto \&LengthOfStream; # tail call
  }
}

sub LengthOfStream ( $ ) {
  push @​_,0;
  goto \&LengthOfStream_;
}

But it has, of course, the ugly side effect of erasing the variable
being passed by the caller​:

my $s= SequenceStream 100000;
print LengthOfStream $s,"\n"; # -> 100001
# but now $s is undef

I find it strange, that this case also requires the undef $_[0] (and
@​_=() is not enough!)​:

print LengthOfStream SequenceStream 100000,"\n";

Any idea?

What I also don't understand is why this leaks memory and eventually
terminates because of an out of memory condition (it should just run
forever)​:

sub InfiniteStream {
  my ($startnumber)=@​_;
  #undef $_[0]; Modification of a read-only value attempted -- of course.
  #@​_=();
  Delay {
  Cons $startnumber, InfiniteStream($startnumber+1);
  };
}

LengthOfStream InfiniteStream(0);

Any idea here?

Christian.

[p5pRT]

From xmltwig@gmail.com

Created by xmltwig@gmail.com

I create a tree with a single root and thousands of children. The link
from the root to the first child is strong, as is the link from each
node to its next sibling. The link from each node to the root is weakened.

The program crashes when the root goes out of scope.

See the attached code which tries to find the exact number. Change the
$NB_ELT_INIT and the $STEP values to find it.

Note that the limit can vary (23790 and 23810 in my limited tests).
I had the same problem with XML​::Twig, which also builds a tree, at
about the same number of nodes. Twig uses weaken several times for each
node though, so that might be an element. Note that it is not the total
number of weaken that counts, just the number in the block in which the
variables are in scope.

This behaviour happens in Scalar​::Util 1.18 and 1.19
perl -v​:
This is perl, v5.8.8 built for i486-linux-gnu-thread-multi

I have also had reports of bugs in XML​::Twig that look suspiciously
like they could come from this problem, on various machines (just today​:
Solaris in c.l.p.m, perl 5.8.5 and 5.8.8, and perl v5.8.7 built for
i386-freebsd-64int by mail).

I hope that helps

--
mirod

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl v5.8.8:

Configured by Debian Project at Tue Mar  6 01:52:23 UTC 2007.

Summary of my perl5 (revision 5 version 8 subversion 8) configuration:
  Platform:
    osname=linux, osvers=2.6.15.7, archname=i486-linux-gnu-thread-multi
    uname='linux rothera 2.6.15.7 #1 smp sat sep 30 10:21:42 utc 2006
i686 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN
-Dcccdlflags=-fPIC -Darchname=i486-linux-gnu -Dprefix=/usr
-Dprivlib=/usr/share/perl/5.8 -Darchlib=/usr/lib/perl/5.8
-Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5
-Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl/5.8.8
-Dsitearch=/usr/local/lib/perl/5.8.8 -Dman1dir=/usr/share/man/man1
-Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1
-Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl
-Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Uusenm
-Duseshrplib -Dlibperl=libperl.so.5.8.8 -Dd_dosuid -des'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define
usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS
-DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN
-fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='4.1.2 (Ubuntu 4.1.2-0ubuntu4)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=/lib/libc-2.5.so, so=so, useshrplib=true, libperl=libperl.so.5.8.8
    gnulibc_version='2.5'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:



@INC for perl v5.8.8:
    /etc/perl
    /usr/local/lib/perl/5.8.8
    /usr/local/share/perl/5.8.8
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.8
    /usr/share/perl/5.8
    /usr/local/lib/site_perl
    /usr/local/lib/perl/5.8.7
    /usr/local/share/perl/5.8.7
    .


Environment for perl v5.8.8:
    HOME=/home/mrodrigu
    LANG=en_US.UTF-8
    LANGUAGE=en_IT:en
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/mrodrigu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/usr/games:/home/mrodrigu/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash