5.14.0-RC2 op/threads.t failures

[p5pRT]

Migrated from rt.perl.org#90006 (status was 'resolved')

Searchable as RT90006$

[p5pRT]

From [email protected]

I've compiled my perl with the following options (full perl -V output
below)​:
-Dusethreads -Duselargefiles -Dcccdlflags= -Dlddlflags+=-fPIC -Duseshrplib
-Dlibperl=libperl.so -Dusemymalloc

The failing test output is​:

[ahartmai@​ahartmai-nb perl-5.14.0-RC2]$
LD_LIBRARY_PATH=/home/ahartmai/perl5/perlbrew/build/perl-5.14.0-RC2; prove
-e./perl -vl t/op/threads.t
t/op/threads.t ..
1..24
ok 1 - delete() under threads
ok 2 - weaken ref under threads
ok 3 - weaken ref \#2 under threads
# parent 17409​: continue
# kid 1 before sort
# parent 17409​: continue
# parent 17409​: waiting for join
# kid 2 before sort
# kid 2 after sort, sleeping 1
# kid 1 after sort, sleeping 1
# kid 2 exit
# kid 1 exit
# parent 17409​: thread exited
# parent 17409​: waiting for join
# parent 17409​: thread exited
ok 4
ok 5 - cloning constant subs
ok 6 - Ensure PL_linestr can be cloned
ok 7 - threads in CHECK block
ok 8 - threads in INIT block
ok 9 - Bug \#41138
Failed 15/24 subtests

Test Summary Report

---

t/op/threads.t (Wstat​: 11 Tests​: 9 Failed​: 0)
  Non-zero wait status​: 11
  Parse errors​: Bad plan. You planned 24 tests but ran 9.
Files=1, Tests=9, 3 wallclock secs ( 0.04 usr 0.00 sys + 3.04 cusr 0.20
csys = 3.28 CPU)
Result​: FAIL

[ahartmai@​ahartmai-nb perl-5.14.0-RC2]$ ./perl -Ilib -V
Summary of my perl5 (revision 5 version 14 subversion 0) configuration​:

  Platform​:
  osname=linux, osvers=2.6.38-8-generic,
archname=x86_64-linux-thread-multi
  uname='linux ahartmai-nb 2.6.38-8-generic #42-ubuntu smp mon apr 11
03​:31​:24 utc 2011 x86_64 x86_64 x86_64 gnulinux '
  config_args='-de
-Dprefix=/home/ahartmai/perl5/perlbrew/perls/perl-5.14.0-RC2 -Dusethreads
-Duselargefiles -Dcccdlflags= -Dlddlflags+=-fPIC -Duseshrplib
-Dlibperl=libperl.so -Dusemymalloc'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=define, usemultiplicity=define
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=y, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
  optimize='-O2',
  cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include'
  ccversion='', gccversion='4.5.2', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
  libpth=/usr/local/lib /lib /usr/lib /usr/lib/x86_64-linux-gnu /lib64
/usr/lib64
  libs=-lnsl -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
  perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
  libc=, so=so, useshrplib=true, libperl=libperl.so
  gnulibc_version='2.13'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E
-Wl,-rpath,/home/ahartmai/perl5/perlbrew/perls/perl-5.14.0-RC2/lib/5.14.0/x86_64-linux-thread-multi/CORE'
  cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib
-fstack-protector'

Characteristics of this binary (from libperl)​:
  Compile-time options​: MULTIPLICITY MYMALLOC PERL_DONT_CREATE_GVSV
  PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
  PERL_PRESERVE_IVUV USE_64_BIT_ALL USE_64_BIT_INT
  USE_ITHREADS USE_LARGE_FILES USE_PERLIO
USE_PERL_ATOF
  USE_REENTRANT_API
  Locally applied patches​:
  RC2
  Built under linux
  Compiled at May 6 2011 14​:28​:09
  %ENV​:

PERLBREW_PATH="/home/ahartmai/perl5/perlbrew/bin​:/home/ahartmai/perl5/perlbrew/perls/current/bin"
  PERLBREW_PERL="perl-5.12.3"
  PERLBREW_ROOT="/home/ahartmai/perl5/perlbrew"
  PERLBREW_VERSION="0.19"
  @​INC​:
  lib

/home/ahartmai/perl5/perlbrew/perls/perl-5.14.0-RC2/lib/site_perl/5.14.0/x86_64-linux-thread-multi
  /home/ahartmai/perl5/perlbrew/perls/perl-5.14.0-RC2/lib/site_perl/5.14.0

/home/ahartmai/perl5/perlbrew/perls/perl-5.14.0-RC2/lib/5.14.0/x86_64-linux-thread-multi
  /home/ahartmai/perl5/perlbrew/perls/perl-5.14.0-RC2/lib/5.14.0
  .

Best regards, Alex (abraxxa)

[p5pRT]

From @obra

Reproduced locally with Ubuntu natty x86_64 and

./Configure -des -Dusethreads -Duselargefiles -Dcccdlflags=
-Dlddlflags+=-fPIC -Duseshrplib -Dlibperl=libperl.so -Dusemymalloc

t/op/threads...................................................perl​:
malloc.c​:3096​: sYSMALLOc​: Assertion `(old_top == (((mbinptr) (((char *)
&((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk,
fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned
long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 *
(sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) &&
((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)'
failed.

On Fri 6.May'11 at 6​:03​:41 -0700, Alexander Hartmaier wrote​:

# New Ticket Created by Alexander Hartmaier
# Please include the string​: [perl #90006]
# in the subject line of all future correspondence about this issue.
# <URL​: http​://rt.perl.org/rt3/Ticket/Display.html?id=90006 >

I've compiled my perl with the following options (full perl -V output
below)​:
-Dusethreads -Duselargefiles -Dcccdlflags= -Dlddlflags+=-fPIC -Duseshrplib
-Dlibperl=libperl.so -Dusemymalloc

The failing test output is​:

[ahartmai@​ahartmai-nb perl-5.14.0-RC2]$
LD_LIBRARY_PATH=/home/ahartmai/perl5/perlbrew/build/perl-5.14.0-RC2; prove
-e./perl -vl t/op/threads.t
t/op/threads.t ..
1..24
ok 1 - delete() under threads
ok 2 - weaken ref under threads
ok 3 - weaken ref \#2 under threads
# parent 17409​: continue
# kid 1 before sort
# parent 17409​: continue
# parent 17409​: waiting for join
# kid 2 before sort
# kid 2 after sort, sleeping 1
# kid 1 after sort, sleeping 1
# kid 2 exit
# kid 1 exit
# parent 17409​: thread exited
# parent 17409​: waiting for join
# parent 17409​: thread exited
ok 4
ok 5 - cloning constant subs
ok 6 - Ensure PL_linestr can be cloned
ok 7 - threads in CHECK block
ok 8 - threads in INIT block
ok 9 - Bug \#41138
Failed 15/24 subtests

Test Summary Report
-------------------
t/op/threads.t (Wstat​: 11 Tests​: 9 Failed​: 0)
Non-zero wait status​: 11
Parse errors​: Bad plan. You planned 24 tests but ran 9.
Files=1, Tests=9, 3 wallclock secs ( 0.04 usr 0.00 sys + 3.04 cusr 0.20
csys = 3.28 CPU)
Result​: FAIL

[ahartmai@​ahartmai-nb perl-5.14.0-RC2]$ ./perl -Ilib -V
Summary of my perl5 (revision 5 version 14 subversion 0) configuration​:

Platform​:
osname=linux, osvers=2.6.38-8-generic,
archname=x86_64-linux-thread-multi
uname='linux ahartmai-nb 2.6.38-8-generic #42-ubuntu smp mon apr 11
03​:31​:24 utc 2011 x86_64 x86_64 x86_64 gnulinux '
config_args='-de
-Dprefix=/home/ahartmai/perl5/perlbrew/perls/perl-5.14.0-RC2 -Dusethreads
-Duselargefiles -Dcccdlflags= -Dlddlflags+=-fPIC -Duseshrplib
-Dlibperl=libperl.so -Dusemymalloc'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=y, bincompat5005=undef
Compiler​:
cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
optimize='-O2',
cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include'
ccversion='', gccversion='4.5.2', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries​:
ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib /usr/lib/x86_64-linux-gnu /lib64
/usr/lib64
libs=-lnsl -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
libc=, so=so, useshrplib=true, libperl=libperl.so
gnulibc_version='2.13'
Dynamic Linking​:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E
-Wl,-rpath,/home/ahartmai/perl5/perlbrew/perls/perl-5.14.0-RC2/lib/5.14.0/x86_64-linux-thread-multi/CORE'
cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib
-fstack-protector'

Characteristics of this binary (from libperl)​:
Compile-time options​: MULTIPLICITY MYMALLOC PERL_DONT_CREATE_GVSV
PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
PERL_PRESERVE_IVUV USE_64_BIT_ALL USE_64_BIT_INT
USE_ITHREADS USE_LARGE_FILES USE_PERLIO
USE_PERL_ATOF
USE_REENTRANT_API
Locally applied patches​:
RC2
Built under linux
Compiled at May 6 2011 14​:28​:09
%ENV​:

PERLBREW_PATH="/home/ahartmai/perl5/perlbrew/bin​:/home/ahartmai/perl5/perlbrew/perls/current/bin"
PERLBREW_PERL="perl-5.12.3"
PERLBREW_ROOT="/home/ahartmai/perl5/perlbrew"
PERLBREW_VERSION="0.19"
@​INC​:
lib

/home/ahartmai/perl5/perlbrew/perls/perl-5.14.0-RC2/lib/site_perl/5.14.0/x86_64-linux-thread-multi
/home/ahartmai/perl5/perlbrew/perls/perl-5.14.0-RC2/lib/site_perl/5.14.0

/home/ahartmai/perl5/perlbrew/perls/perl-5.14.0-RC2/lib/5.14.0/x86_64-linux-thread-multi
/home/ahartmai/perl5/perlbrew/perls/perl-5.14.0-RC2/lib/5.14.0
.

Best regards, Alex (abraxxa)

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @obra

./Configure -des -Dusethreads -Dusemymalloc
make
cd t; LD_LIBRARY_PATH=.. ../perl -I../lib op/threads.t

is enough to trigger the segfault. on both 5.14.0-RC2 and maint-5.12 on
Ubuntu natty.

As far as I can tell, this is _not_ a regression. That means that it's
release critical.

[p5pRT]

From @tonycoz

Backtrace when built with -DEBUGGING -Duseithreads -Dusemymalloc​:

rogram received signal SIGSEGV, Segmentation fault.
Perl_padlist_dup (my_perl=0x7ffff00b88a0, srcpad=<value optimised out>,
  param=0x7fffffffde70) at pad.c​:1956
1956 const char sigil = SvPVX_const(names[ix])[0];
(gdb) bt
#0 Perl_padlist_dup (my_perl=0x7ffff00b88a0, srcpad=<value optimised out>,
  param=0x7fffffffde70) at pad.c​:1956
#1 0x000000000050efbf in S_sv_dup_common (my_perl=0x7ffff00b88a0,
  sstr=0x96a820, param=0x7fffffffde70) at sv.c​:12114
#2 0x000000000050ffde in Perl_sv_dup (my_perl=0x7ffff00b88a0,
  sstr=<value optimised out>, param=0x7fffffffde70) at sv.c​:12140
#3 0x00000000005110e7 in Perl_mg_dup (my_perl=0x7ffff00b88a0, mg=0x9684c8,
  param=0x7fffffffde70) at sv.c​:11483
#4 0x000000000050f60a in S_sv_dup_common (my_perl=0x7ffff00b88a0,
  sstr=0x96ad90, param=0x7fffffffde70) at sv.c​:11899
#5 0x0000000000510070 in Perl_sv_dup_inc (my_perl=<value optimised out>,
  sstr=<value optimised out>, param=<value optimised out>) at sv.c​:12134
#6 0x00000000004df3a9 in Perl_he_dup (my_perl=0x7ffff00b88a0, e=0x95e290,
  shared=<value optimised out>, param=0x7fffffffde70) at hv.c​:187
#7 0x000000000050f157 in S_sv_dup_common (my_perl=0x7ffff00b88a0,
  sstr=0x914958, param=0x7fffffffde70) at sv.c​:12017
#8 0x000000000050ffde in Perl_sv_dup (my_perl=0x7ffff00b88a0,
  sstr=<value optimised out>, param=0x7fffffffde70) at sv.c​:12140
#9 0x000000000050fa47 in S_sv_dup_common (my_perl=0x7ffff00b88a0,
  sstr=0x93f830, param=0x7fffffffde70) at sv.c​:11936
#10 0x000000000050ffde in Perl_sv_dup (my_perl=0x7ffff00b88a0,
  sstr=<value optimised out>, param=0x7fffffffde70) at sv.c​:12140
---Type <return> to continue, or q <return> to quit---
#11 0x000000000051cda4 in perl_clone (proto_perl=0x8f0010, flags=2)
  at sv.c​:12933
#12 0x00007ffff689d51d in S_ithread_create (my_perl=0x8f0010,
  cv=<value optimised out>) at threads.xs​:776
#13 XS_threads_create (my_perl=0x8f0010, cv=<value optimised out>)
  at threads.xs​:1087
#14 0x00000000004f7df1 in Perl_pp_entersub (my_perl=0x8f0010) at
pp_hot.c​:3046
#15 0x00000000004b9d1d in Perl_runops_debug (my_perl=0x8f0010) at
dump.c​:2267
#16 0x0000000000441da3 in S_run_body (my_perl=0x8f0010) at perl.c​:2350
#17 perl_run (my_perl=0x8f0010) at perl.c​:2268
#18 0x000000000041a17c in main (argc=3, argv=0x7fffffffe2a8,
  env=0x7fffffffe2c8) at perlmain.c​:120
(gdb)

[p5pRT]

From @iabyn

On Sat, May 07, 2011 at 04​:18​:03PM -0400, Jesse Vincent wrote​:

./Configure -des -Dusethreads -Dusemymalloc
make
cd t; LD_LIBRARY_PATH=.. ../perl -I../lib op/threads.t

is enough to trigger the segfault. on both 5.14.0-RC2 and maint-5.12 on
Ubuntu natty.

As far as I can tell, this is _not_ a regression. That means that it's
release critical.

Now fixed by the commits

  692fcd3
  50f626a
  f144f1e

It's basically various issues relating to running out of memory while
cloning a new thread. The perl malloc expects a certain environment to be
present and correct, and that may not be the case is the malloc occurs
during the early stages of the clone.

The commits above fix the specific issues, although I suspect there's a
can of worms waiting to be opened there, should one dig deeper. Indeed,
at one point I noticed that malloc's attempt to output an "Out of
memory!" warning to STDERR using PerlIO was triggering a recursive call to
malloc() from the bowels of PerlIO, and eventually blowing the stack. But
I think that sort sort of issue is for another day. The specific
threads/clone-related issue reported has been fixed.

--
print+qq&$}$"$/$s$,$a$d$g$s$@​$.$q$,$​:$.$q$^$,$@​$a$$;$.$q$m&if+map{m,^\d{0\,},,${$​::{$'}}=chr($"+=$&amp;||1)}q&10m22,42}6​:17a22.3@​3;^2dg3q/s"&=~m*\d\*.*g

[p5pRT]

@iabyn - Status changed from 'open' to 'resolved'