perlio & -Dm dumps core

[p5pRT]

Migrated from rt.perl.org#9027 (status was 'resolved')

Searchable as RT9027$

[p5pRT]

From @vanstyn

Created by @vanstyn

crypt% ./perl -Dm -e 1
Segmentation fault (core dumped)
crypt%

Stacktrace of the core file shows a stackbusting recursion of
these three​:
#15 0x0810e10a in PerlIO_allocate () at perlio.c​:446
#16 0x08112b04 in Perl_PerlIO_stderr () at perlio.c​:970
#17 0x0806356f in malloc (nbytes=256) at malloc.c​:1106

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl v5.7.3:

Configured by hv at Thu Apr 25 12:24:47 BST 2002.

Summary of my perl5 (revision 5.0 version 7 subversion 3 patch 16151) configuration:
  Platform:
    osname=linux, osvers=2.4.2-2, archname=i686-linux
    uname='linux crypt.compulink.co.uk 2.4.2-2 #1 sun apr 8 20:41:30 edt 2001 i686 unknown '
    config_args='-des -Dprefix=/opt/bleadperl-m -Doptimize=-g -O6 -Dusedevel -Uversiononly -Dusemymalloc'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=y, bincompat5005=define
  Compiler:
    cc='cc', ccflags ='-DDEBUGGING -fno-strict-aliasing -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
    optimize='-g -O6',
    cppflags='-DDEBUGGING -fno-strict-aliasing -I/usr/include/gdbm'
    ccversion='', gccversion='2.96 20000731 (Red Hat Linux 7.1 2.96-81)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lndbm -lgdbm -ldl -lm -lc -lcrypt -lutil
    perllibs=-lnsl -ldl -lm -lc -lcrypt -lutil
    libc=/lib/libc-2.2.2.so, so=so, useshrplib=false, libperl=libperl.a
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
    cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    DEVEL16151


@INC for perl v5.7.3:
    lib
    /opt/bleadperl-m/lib/5.7.3/i686-linux
    /opt/bleadperl-m/lib/5.7.3
    /opt/bleadperl-m/lib/site_perl/5.7.3/i686-linux
    /opt/bleadperl-m/lib/site_perl/5.7.3
    /opt/bleadperl-m/lib/site_perl
    .


Environment for perl v5.7.3:
    HOME=/home/hv
    LANG=C
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/hv/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

[p5pRT]

From @jhi

On Thu, Apr 25, 2002 at 02​:47​:03PM +0100, Hugo van der Sanden wrote​:

This is a bug report for perl from hv@​crypt.compulink.co.uk,
generated with the help of perlbug 1.33 running under perl v5.7.3@​16151.

-----------------------------------------------------------------
[Please enter your report here]

crypt% ./perl -Dm -e 1
Segmentation fault (core dumped)
crypt%

Stacktrace of the core file shows a stackbusting recursion of
these three​:
#15 0x0810e10a in PerlIO_allocate () at perlio.c​:446
#16 0x08112b04 in Perl_PerlIO_stderr () at perlio.c​:970
#17 0x0806356f in malloc (nbytes=256) at malloc.c​:1106

Interesting​: tru64@​16153 doesn't enter a recursion, but instead
finishes nicely.

--
$jhi++; # http​://www.iki.fi/jhi/
  # There is this special biologist word we use for 'stable'.
  # It is 'dead'. -- Jack Cohen

[p5pRT]

From [Unknown Contact. See original ticket]

Jarkko Hietaniemi <jhi@​iki.fi> writes​:

----- Forwarded message from Hugo van der Sanden <hv@​crypt.compulink.co.uk> -----

Subject​: [ID 20020425.009] perlio & -Dm dumps core
From​: Hugo van der Sanden <hv@​crypt.compulink.co.uk>
Date​: Thu, 25 Apr 2002 14​:47​:03 +0100
Message-Id​: <200204251347.g3PDl3f25967@​crypt.compulink.co.uk>
To​: perl5-porters@​perl.org
Reply-To​: hv@​crypt0.demon.co.uk

This is a bug report for perl from hv@​crypt.compulink.co.uk,
generated with the help of perlbug 1.33 running under perl v5.7.3@​16151.

-----------------------------------------------------------------
[Please enter your report here]

crypt% ./perl -Dm -e 1
Segmentation fault (core dumped)
crypt%

I am not sure what to say here. -Dm tries to print something when
you malloc() and is using PerlIO *. PerlIO * needs to malloc its
data structures. How can it do anything but recurse?

We either need to have another malloc() for PerlIO to use,
or -Dm needs to be disabled while in PerlIO, or ....

Stacktrace of the core file shows a stackbusting recursion of
these three​:
#15 0x0810e10a in PerlIO_allocate () at perlio.c​:446
#16 0x08112b04 in Perl_PerlIO_stderr () at perlio.c​:970
#17 0x0806356f in malloc (nbytes=256) at malloc.c​:1106

--
Nick Ing-Simmons
http​://www.ni-s.u-net.com/

[p5pRT]

From @paulg1973

Nick Ing-Simmons [mailto​:nick.ing-simmons@​elixent.com] wrote​:

Jarkko Hietaniemi <jhi@​iki.fi> writes​:

----- Forwarded message from Hugo van der Sanden
<hv@​crypt.compulink.co.uk> -----
[snip]
-----------------------------------------------------------------
[Please enter your report here]

crypt% ./perl -Dm -e 1
Segmentation fault (core dumped)
crypt%

I am not sure what to say here. -Dm tries to print something when
you malloc() and is using PerlIO *. PerlIO * needs to malloc its
data structures. How can it do anything but recurse?

We either need to have another malloc() for PerlIO to use,
or -Dm needs to be disabled while in PerlIO, or ....

I submit that the way out is to first define the layers of abstraction. If
it is as follows​:

<Perl Compiler & Interpreter>
<PerlIO>
<Storage Management>
<C Libraries>
<OS>

then it is pretty clear that <Storage Management> shouldn't call "up" the
hierarchy. If it wants to print something, it's got to use simpler
mechanism.

Seems to me the problem arises from a muddled definition of the abstract
machine.

PG

[p5pRT]

From @vanstyn

Jarkko Hietaniemi <jhi@​iki.fi> wrote​:
:On Thu, Apr 25, 2002 at 02​:47​:03PM +0100, Hugo van der Sanden wrote​:
:> crypt% ./perl -Dm -e 1
:> Segmentation fault (core dumped)
:> crypt%
:>
:> Stacktrace of the core file shows a stackbusting recursion of
:> these three​:
:> #15 0x0810e10a in PerlIO_allocate () at perlio.c​:446
:> #16 0x08112b04 in Perl_PerlIO_stderr () at perlio.c​:970
:> #17 0x0806356f in malloc (nbytes=256) at malloc.c​:1106
:
:Interesting​: tru64@​16153 doesn't enter a recursion, but instead
:finishes nicely.

This recursion only occurs with usemymalloc, and it happens because
the perl.h DEBUG_m macro is safed against recursion but malloc.c
overrides it with an unsafe variant. The patch below duplicates the
safe version, and allows me to get further, but changing the macro
only gets C< perl -Dm -e 1 > to the next segfault 484 malloc debug
lines later. Along the way there are a couple of warnings​:

0x814f400​: (00248) malloc 1008 bytes
0x8144a30​: (00249) malloc 1 bytes
0x8144a38​: (00250) realloc 46 bytes the hard way
0x8150440​: (00251) malloc 46 bytes
0x8144a38​: (00252) free
0x8150448​: (00253) realloc 54 bytes the hard way
0x8143d00​: (00254) malloc 54 bytes
0x8150448​: (00255) free
Unaligned `next' pointer in the free chain 0x3fbff0a at 0x814f400
0x8143d08​: (00256) free
[...]
0x8144b08​: (00483) free
Unaligned pointer in the free chain 0x3fbff0a
0x8151848​: (00484) free
Segmentation fault (core dumped)

I guess this is a reentrancy problem​: note that not long before the
first warning, 0x814f400 has been malloced (and not since freed),
but the complaint says it is in the free chain. My guess is that
this occurs because the DEBUG_m for malloc is inside the critical
section; the patch moves it to just after the critical section, and
now the simple C< perl -Dm -e 1 > runs to completion (with a total
of 570 malloc debug lines, or 1171 with PERL_DESTRUCT_LEVEL 2).

The 'unaligned' warnings are still in the critical section, but
as far as I can see they're just a shade shy of a panic, so I
haven't touched them. I'm more worried about the
  PerlIO_puts(PerlIO_stderr(),s);
style used to print the 'Out of memory' diagnostic before exiting;
if this isn't safe against additional malloc, I think it is important
that it be made so (either by changing the PerlIO implementation,
or by providing a new interface that we could use here), since
a core dump at this stage is going to snowball the initial problem
by filling up your disk.

Hugo

Inline Patch

--- malloc.c.old	Mon Mar 11 04:15:42 2002
+++ malloc.c	Thu May  2 04:03:47 2002
@@ -356,9 +356,16 @@
 
 #ifdef DEBUGGING
 #  undef DEBUG_m
-#  define DEBUG_m(a)  \
+#  define DEBUG_m(a) 							\
     STMT_START {							\
-	if (PERL_GET_INTERP) { dTHX; if (DEBUG_m_TEST) { a; } }	\
+	if (PERL_GET_INTERP) {						\
+	    dTHX;							\
+	    if (DEBUG_m_TEST) {						\
+		PL_debug &= ~DEBUG_m_FLAG;				\
+		a;							\
+		PL_debug |= DEBUG_m_FLAG;				\
+	    }								\
+	}								\
     } STMT_END
 #endif
 
@@ -1103,11 +1110,6 @@
   		return (NULL);
 	}
 
-	DEBUG_m(PerlIO_printf(Perl_debug_log,
-			      "0x%"UVxf": (%05lu) malloc %ld bytes\n",
-			      PTR2UV(p), (unsigned long)(PL_an++),
-			      (long)size));
-
 	/* remove from linked list */
 #if defined(RCHECK)
 	if ((PTR2UV(p)) & (MEM_ALIGNBYTES - 1)) {
@@ -1127,6 +1129,11 @@
   	nextf[bucket] = p->ov_next;
 
 	MALLOC_UNLOCK;
+
+	DEBUG_m(PerlIO_printf(Perl_debug_log,
+			      "0x%"UVxf": (%05lu) malloc %ld bytes\n",
+			      PTR2UV(p), (unsigned long)(PL_an++),
+			      (long)size));
 
 #ifdef IGNORE_SMALL_BAD_FREE
 	if (bucket >= FIRST_BUCKET_WITH_CHECK)