Blead breaks Sereal-Encoder-3.015

[p5pRT]

Migrated from rt.perl.org#131852 (status was 'resolved')

Searchable as RT131852$

[p5pRT]

From @jkeenan

A recent commit to Perl 5 blead has broken CPAN distribution
Sereal-Encoder-3.015.

This was called to my attention by Carlos Guevara; see, e.g.,
http​://www.cpantesters.org/cpan/report/c7f78c0e-797f-11e7-82e1-3ed0897f352c.

Errors have been observed at commit fde84d2, but the offending commit
has not yet been identified. The errors are too extensive to reproduce
here; see link above.

Thank you very much.
Jim Keenan

[p5pRT]

From @jkeenan

Summary of my perl5 (revision 5 version 27 subversion 3) configuration​:
  Commit id​: d819dc5
  Platform​:
  osname=linux
  osvers=4.4.0-89-generic
  archname=x86_64-linux
  uname='linux zareason 4.4.0-89-generic #112-ubuntu smp mon jul 31 19​:38​:41 utc 2017 x86_64 x86_64 x86_64 gnulinux '
  config_args='-des -Dusedevel -Uversiononly -Dprefix=/home/jkeenan/testing/blead -Dman1dir=none -Dman3dir=none'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=undef
  usemultiplicity=undef
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='cc'
  ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
  optimize='-O2'
  cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='5.4.0 20160609'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='cc'
  ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/5/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /lib64 /usr/lib64
  libs=-lpthread -lnsl -ldb -ldl -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.23.so
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version='2.23'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E'
  cccdlflags='-fPIC'
  lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:
  Compile-time options​:
  HAS_TIMES
  PERLIO_LAYERS
  PERL_COPY_ON_WRITE
  PERL_DONT_CREATE_GVSV
  PERL_MALLOC_WRAP
  PERL_OP_PARENT
  PERL_PRESERVE_IVUV
  PERL_USE_DEVEL
  USE_64_BIT_ALL
  USE_64_BIT_INT
  USE_LARGE_FILES
  USE_LOCALE
  USE_LOCALE_COLLATE
  USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC
  USE_LOCALE_TIME
  USE_PERLIO
  USE_PERL_ATOF
  Built under linux
  Compiled at Aug 7 2017 08​:54​:17
  %ENV​:
  PERLBREW_BASHRC_VERSION="0.78"
  PERLBREW_HOME="/home/jkeenan/.perlbrew"
  PERLBREW_MANPATH="/home/jkeenan/perl5/perlbrew/perls/perl-5.26.0/man"
  PERLBREW_PATH="/home/jkeenan/perl5/perlbrew/bin​:/home/jkeenan/perl5/perlbrew/perls/perl-5.26.0/bin"
  PERLBREW_PERL="perl-5.26.0"
  PERLBREW_ROOT="/home/jkeenan/perl5/perlbrew"
  PERLBREW_VERSION="0.78"
  PERL_WORKDIR="/home/jkeenan/gitwork/perl"
  @​INC​:
  lib
  /home/jkeenan/testing/blead/lib/perl5/site_perl/5.27.3/x86_64-linux
  /home/jkeenan/testing/blead/lib/perl5/site_perl/5.27.3
  /home/jkeenan/testing/blead/lib/perl5/5.27.3/x86_64-linux
  /home/jkeenan/testing/blead/lib/perl5/5.27.3

[p5pRT]

From @demerphq

Thanks. I will investigate eventually if noone else does.

Yves

On 7 August 2017 at 15​:16, James E Keenan <perlbug-followup@​perl.org> wrote​:

# New Ticket Created by James E Keenan
# Please include the string​: [perl #131852]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=131852 >

A recent commit to Perl 5 blead has broken CPAN distribution
Sereal-Encoder-3.015.

This was called to my attention by Carlos Guevara; see, e.g.,
http​://www.cpantesters.org/cpan/report/c7f78c0e-797f-11e7-82e1-3ed0897f352c.

Errors have been observed at commit fde84d2, but the offending commit
has not yet been identified. The errors are too extensive to reproduce
here; see link above.

Thank you very much.
Jim Keenan

--
perl -Mre=debug -e "/just|another|perl|hacker/"

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @iabyn

On Mon, Aug 07, 2017 at 03​:22​:54PM +0200, demerphq wrote​:

Thanks. I will investigate eventually if noone else does.

The first to fail is 010_desperate.t with the first failure being
(ignore the test numbers - this is from a stripped down script)​:

ok 1 - (plain) defined​: blessed regexp with reuse
not ok 2 - (plain) correct​: blessed regexp with reuse
# Failed test '(plain) correct​: blessed regexp with reuse'
# at /tmp/010_desperate.t line 68.
# got​: '"=\363rl\3\0B,cbar(\352(?^ix​:foo))\b"'
# expected​: '"=\363rl\3\0B,cbar(\261cfoobix)\b"'
1..2

Which is trying to serialise this​:

  do { my $qr= bless qr/foo/ix,"bar"; [ $qr, $qr ] },

I don't know much about the internals of Sereal​::Encoder, but I would
hazard a guess that the change is due to the fact that REGEXP SVs
now have the POK flag set again after a recent commit of mine. (They used
to up until 5.18, then didn't due to an issue with PVLV-as-REGEXPs, and
now do again).

This code​:

  my $qr= bless qr/foo/ix,"bar";
  use Devel​::Peek;
  Dump $qr;

shows the following main difference​:

- FLAGS = (OBJECT,FAKE)
+ FLAGS = (OBJECT,POK,FAKE,pPOK)

REGEXPs have also changed, with various parts the SV stored as​:

Previously​:

  if SVt_REGEXP​:

  sv_any​: the 'struct regexp*' pointer
  sv_u​: the 'struct regexp*' pointer
  sv_any->xpv_len_u.xpvlenu_pv​: the string "(?^ix​:foo))"

  if SVt_PVLV​:

  sv_any​: the XPVLV* pointer
  sv_u​: the 'struct regexp*' pointer
  sv_any->xpv_len_u.xpvlenu_pv​: the string "(?^ix​:foo))"

Now​:

  if SVt_REGEXP​:

  sv_any​: the 'struct regexp*' pointer
  sv_u​: the string "(?^ix​:foo))"

  if SVt_PVLV​:

  sv_any​: the XPVLV* pointer
  sv_any->xpv_len_u.xpvlenu_rx​: the 'struct regexp*' pointer
  sv_u​: the string "(?^ix​:foo))"

Now the ReANY() macro includes a test of sv->sv_type == SVt_PVLV
and returns sv_any or sv_any->xpv_len_u.xpvlenu_rx accordingly.

--
"Procrastination grows to fill the available time"
  -- Mitchell's corollary to Parkinson's Law

[p5pRT]

From @jkeenan

On Mon, 07 Aug 2017 13​:23​:07 GMT, demerphq wrote​:

Thanks. I will investigate eventually if noone else does.

Yves

On 7 August 2017 at 15​:16, James E Keenan <perlbug-followup@​perl.org>
wrote​:

# New Ticket Created by James E Keenan
# Please include the string​: [perl #131852]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=131852 >

A recent commit to Perl 5 blead has broken CPAN distribution
Sereal-Encoder-3.015.

This was called to my attention by Carlos Guevara; see, e.g.,
http​://www.cpantesters.org/cpan/report/c7f78c0e-797f-11e7-82e1-
3ed0897f352c.

Errors have been observed at commit fde84d2, but the offending
commit
has not yet been identified. The errors are too extensive to
reproduce
here; see link above.

Thank you very much.
Jim Keenan

Still present in perl 5 blead as of today (125cf57)

--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

From @demerphq

On 1 November 2017 at 17​:50, James E Keenan via RT
<perlbug-followup@​perl.org> wrote​:

Still present in perl 5 blead as of today (125cf57)

fixed in Sereal 4.002

We now have this logic​:

# if ( PERL_VERSION > 27 || (PERL_VERSION == 27 && PERL_SUBVERSION >= 3) )
  /* Commit df6b4bd
  * changed the behavior mentioned below, so that the POK flag is
on again. Sigh.
  * So this branch is a deliberate NO-OP, it just makes the
conditions easier to read.*/
# elif ( PERL_VERSION > 17 || (PERL_VERSION == 17 && PERL_SUBVERSION >= 6) )
  /* With commit 8d919b0 (perl.git) and
  * release 5.17.6, regular expression are no longer SvPOK (IOW
are no longer
  * considered to be containing a string).
  * This breaks some of the REGEXP detection logic in srl_dump_sv, so
  * we need yet another CPP define. */
# define REGEXP_NO_LONGER_POK
# endif

The amount of macro goo to write backwards portable Regexp code is
pretty high, that is part of a larger
block of checks that amounts to about 40 lines. :-)

Yves
--
perl -Mre=debug -e "/just|another|perl|hacker/"

[p5pRT]

From @jkeenan

On Sat, 11 Nov 2017 09​:36​:41 GMT, demerphq wrote​:

On 1 November 2017 at 17​:50, James E Keenan via RT
<perlbug-followup@​perl.org> wrote​:

Still present in perl 5 blead as of today
(125cf57)

fixed in Sereal 4.002

We now have this logic​:

# if ( PERL_VERSION > 27 || (PERL_VERSION == 27 && PERL_SUBVERSION

= 3) )
/* Commit df6b4bd
* changed the behavior mentioned below, so that the POK flag is
on again. Sigh.
* So this branch is a deliberate NO-OP, it just makes the
conditions easier to read.*/
# elif ( PERL_VERSION > 17 || (PERL_VERSION == 17 &&
PERL_SUBVERSION >= 6) )
/* With commit 8d919b0
(perl.git) and
* release 5.17.6, regular expression are no longer SvPOK (IOW
are no longer
* considered to be containing a string).
* This breaks some of the REGEXP detection logic in srl_dump_sv,
so
* we need yet another CPP define. */
# define REGEXP_NO_LONGER_POK
# endif

The amount of macro goo to write backwards portable Regexp code is
pretty high, that is part of a larger
block of checks that amounts to about 40 lines. :-)

Yves

Confirmed as of blead commit 3d2ba98.

Marking ticket Resolved.

Thank you very much.
--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

@jkeenan - Status changed from 'open' to 'resolved'