BBC List::MoreUtils::XS broken by 16ada235c332e017667585e1a5a00ce43a31c529

[p5pRT]

Migrated from rt.perl.org#132578 (status was 'resolved')

Searchable as RT132578$

[p5pRT]

From [email protected]

Created by [email protected]

List​::MoreUtils​::XS has acquired a new test failure at core commit
16ada23 "fix GvSV refcounting in sort".
On affected core versions, its test suite shows

t/xs/occurrences.t .......... 1/? perl​: sv.c​:6506​: Perl_sv_clear​: Assertion `((svtype)((sv)->sv_flags & 0xff)) != (svtype)0xff' failed.
t/xs/occurrences.t .......... All 8 subtests passed

Examining this is complicated by another test failure that shows up
earlier, with debugging builds​:

t/xs/arrayify.t ............. panic​: XSUB List​::MoreUtils​::XS​::arrayify (XS.c) failed to extend arg stack​: base=23e9968, sp=23e99f8, hwm=23e99e0
t/xs/arrayify.t ............. Dubious, test returned 255 (wstat 65280, 0xff00)
No subtests run

I'm pretty sure that the arrayify.t failure would happen with 5.27.6,
though the distro tested just fine on a non-debugging 5.27.6. I'll have a
look at that later. There are also some compilation warnings on threading
builds, picking up definite bugs, that don't cause test failures.
Those are also preexisting and out of scope for this ticket.

Perl Info

Flags:
    category=core
    severity=high

Site configuration information for perl 5.27.7:

Configured by zefram at Tue Dec 12 12:07:39 GMT 2017.

Summary of my perl5 (revision 5 version 27 subversion 7) configuration:
  Commit id: 16ada235c332e017667585e1a5a00ce43a31c529
  Platform:
    osname=linux
    osvers=3.16.0-4-amd64
    archname=x86_64-linux-thread-multi
    uname='linux barba.rous.org 3.16.0-4-amd64 #1 smp debian 3.16.43-2+deb8u2 (2017-06-26) x86_64 gnulinux '
    config_args='-des -Dprefix=/home/zefram/usr/perl/perl_install/perl-git-blead-i64-f52 -Duselargefiles -Dusethreads -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dusedevel -Uversiononly -Ui_db -DDEBUGGING'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
    optimize='-O2 -g'
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='4.9.2'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
    libs=-lpthread -lnsl -ldb -ldl -lm -lcrypt -lutil -lc
    perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.19.so
    so=so
    useshrplib=true
    libperl=libperl.so
    gnulibc_version='2.19'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E -Wl,-rpath,/home/zefram/usr/perl/perl_install/perl-git-blead-i64-f52/lib/5.27.7/x86_64-linux-thread-multi/CORE'
    cccdlflags='-fPIC'
    lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector-strong'



@INC for perl 5.27.7:
    /home/zefram/usr/perl/perl_install/perl-git-blead-i64-f52/lib/site_perl/5.27.7/x86_64-linux-thread-multi
    /home/zefram/usr/perl/perl_install/perl-git-blead-i64-f52/lib/site_perl/5.27.7
    /home/zefram/usr/perl/perl_install/perl-git-blead-i64-f52/lib/5.27.7/x86_64-linux-thread-multi
    /home/zefram/usr/perl/perl_install/perl-git-blead-i64-f52/lib/5.27.7


Environment for perl 5.27.7:
    HOME=/home/zefram
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/zefram/usr/perl/perl_install/perl-git-blead-i64-f52/bin:/home/zefram/pub/x86_64-unknown-linux-gnu/bin:/home/zefram/pub/common/bin:/usr/bin:/bin:/usr/local/bin:/usr/games
    PERL5LIB=
    PERL5OPT=
    PERL5_CPANPLUS_IS_RUNNING=32597
    PERL5_CPAN_IS_RUNNING=32597
    PERLDOC=-oman
    PERL_BADLANG (unset)
    SHELL=/usr/bin/zsh

[p5pRT]

From [email protected]

I wrote​:

List​::MoreUtils​::XS has acquired a new test failure at core commit
16ada23 "fix GvSV refcounting in sort".

I believe the failure arises because List​::MoreUtils​::XS itself has
a bunch of refcount-breaking writes to GvSV of *a and *b, apparently
cribbed from pp_sort. Some of the subs localise their changes to $a and
$b, but some do not. In particular, the "reduce" functions are liable
to set $a to a mortal scalar, and leave $a side-effected to refer to
the now-dead scalar. pp_sort later gets invoked, and some of the new
refcounting logic notices that the $a scalar that it's trying to save
and restore is already dead. There are multiple existing bugs there
for List​::MoreUtils​::XS to fix.

-zefram

[p5pRT]

From [email protected]

I've opened [rt.cpan.org #123868] for List-MoreUtils-XS, regarding its
refcounting bugs.

The other assertion failure that I found already has an rt.cpan.org
ticket, with a patch.

-zefram

[p5pRT]

From @jkeenan

On Tue, 12 Dec 2017 21​:14​:59 GMT, zefram@​fysh.org wrote​:

I've opened [rt.cpan.org #123868] for List-MoreUtils-XS, regarding its
refcounting bugs.

The other assertion failure that I found already has an rt.cpan.org
ticket, with a patch.

-zefram

List-MoreUtils-XS installs for me without problem on perl-5.29.2. And it appears to be doing well on other Perl versions and platforms (http​://matrix.cpantesters.org/?dist=List-MoreUtils-XS).

I don't see any remaining core distribution issue, so I'm closing this ticket.

Thank you very much.
--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

@jkeenan - Status changed from 'open' to 'resolved'