Crash: Embedded perl on FreeBSD: Perl__inverse_folds

[jacquesg]

Description

We embed multiple perl interpreters into some of our applications. After upgrading from 5.26.0 to 5.30.2 (also confirmed that it exists on 5.31.11, 5.28.2 is still unknown), we see the following intermittent crash

* thread #1, name = 'perl', stop reason = signal SIGSEGV: invalid address (fault address: 0x408c58a18)
frame #0: 0x00000000005968db perl`Perl__inverse_folds(my_perl=<unavailable>, cp=97, first_folds_to=0x00007fffffffb40c, remaining_folds_to=0x00007fffffffb410) at utf8.c:3436:16
3433 /* 'index' is guaranteed to be non-negative, as this is an inversion map
3434 * that covers all possible inputs. See [perl #133365] */
3435 SSize_t index = _invlist_search(PL_utf8_foldclosures, cp);
-> 3436 I32 base = _Perl_IVCF_invmap[index];
3437
3438 PERL_ARGS_ASSERT__INVERSE_FOLDS;
3439
(lldb) bt
* thread #1, name = 'perl', stop reason = signal SIGSEGV: invalid address (fault address: 0x408c58a18)
* frame #0: 0x00000000005968db perl`Perl__inverse_folds(my_perl=<unavailable>, cp=97, first_folds_to=0x00007fffffffb40c, remaining_folds_to=0x00007fffffffb410) at utf8.c:3436:16
frame #1: 0x00000000004bce96 perl`S_regclass(my_perl=0x0000000801c22000, pRExC_state=0x00007fffffffbf60, flagp=0x00007fffffffb7bc, depth=9, stop_at_1=<unavailable>, allow_mutiple_chars=true, silence_non_portable=<unavailable>, strict=<unavailable>, optimizable=<unavailable>, ret_invlist=0x0000000000000000) at regcomp.c:19332:53
frame #2: 0x00000000004b02b8 perl`S_regatom(my_perl=0x0000000801c22000, pRExC_state=0x00007fffffffbf60, flagp=0x00007fffffffb7bc, depth=<unavailable>) at regcomp.c:13487:15
frame #3: 0x00000000004ae0c1 perl`S_regbranch [inlined] S_regpiece(my_perl=<unavailable>, pRExC_state=<unavailable>) at regcomp.c:12600:11
frame #4: 0x00000000004ae0a1 perl`S_regbranch(my_perl=0x0000000801c22000, pRExC_state=0x00007fffffffbf60, flagp=0x00007fffffffb8a8, first=<unavailable>, depth=8) at regcomp.c:12520
frame #5: 0x0000000000490370 perl`S_reg(my_perl=0x0000000801c22000, pRExC_state=0x00007fffffffbf60, paren=2, flagp=0x00007fffffffba1c, depth=<unavailable>) at regcomp.c:12222:10
frame #6: 0x00000000004b00ce perl`S_regatom(my_perl=0x0000000801c22000, pRExC_state=<unavailable>, flagp=0x00007fffffffbbec, depth=<unavailable>) at regcomp.c:13509:15
frame #7: 0x00000000004ae0c1 perl`S_regbranch [inlined] S_regpiece(my_perl=<unavailable>, pRExC_state=<unavailable>) at regcomp.c:12600:11
frame #8: 0x00000000004ae0a1 perl`S_regbranch(my_perl=0x0000000801c22000, pRExC_state=0x00007fffffffbf60, flagp=0x00007fffffffbcd8, first=<unavailable>, depth=4) at regcomp.c:12520
frame #9: 0x0000000000490370 perl`S_reg(my_perl=0x0000000801c22000, pRExC_state=0x00007fffffffbf60, paren=0, flagp=0x00007fffffffc0cc, depth=<unavailable>) at regcomp.c:12222:10
frame #10: 0x000000000048cfe8 perl`Perl_re_op_compile(my_perl=0x0000000801c22000, patternp=<unavailable>, pat_count=<unavailable>, expr=<unavailable>, eng=0x0000000000994358, old_re=0x0000000000000000, is_bare_re=0x0000000000000000, orig_rx_flags=0, pm_flags=0) at regcomp.c:7854:9
frame #11: 0x000000000042af4e perl`Perl_pmruntime(my_perl=0x0000000801c22000, o=0x0000000817fdfe10, expr=<unavailable>, repl=0x0000000000000000, flags=<unavailable>, floor=<unavailable>) at op.c:8359:6
frame #12: 0x0000000000486067 perl`Perl_yyparse(my_perl=0x0000000801c22000, gramtype=<unavailable>) at perly.y:1293:23
frame #13: 0x000000000054df6c perl`S_doeval_compile(my_perl=0x0000000801c22000, gimme=<unavailable>, outside=<unavailable>, seq=<unavailable>, hh=<unavailable>) at pp_ctl.c:3540:77
frame #14: 0x000000000054c7d7 perl`Perl_pp_require [inlined] S_require_file(my_perl=<unavailable>, sv=<unavailable>) at pp_ctl.c:4363:9
frame #15: 0x000000000054b41d perl`Perl_pp_require(my_perl=0x0000000801c22000) at pp_ctl.c:4387
frame #16: 0x00000000004f3d06 perl`Perl_runops_standard(my_perl=0x0000000801c22000) at run.c:41:26
frame #17: 0x000000000044ec83 perl`Perl_call_sv(my_perl=0x0000000801c22000, sv=0x000000080fb9ec30, flags=13) at perl.c:3089:6
frame #18: 0x000000000044b2e8 perl`Perl_call_list(my_perl=0x0000000801c22000, oldscope=15, paramList=0x0000000810876c78) at perl.c:5128:6
frame #19: 0x0000000000438e38 perl`S_process_special_blocks(my_perl=0x0000000801c22000, floor=603, fullname=<unavailable>, gv=0x000000080f9ea4e0, cv=0x000000080fb9ec30) at op.c:11733:6
frame #20: 0x0000000000430436 perl`Perl_newATTRSUB_x(my_perl=0x0000000801c22000, floor=603, o=<unavailable>, proto=<unavailable>, attrs=0x0000000000000000, block=0x000000081276b830, o_is_gv=<unavailable>) at op.c:11659:21
frame #21: 0x0000000000431923 perl`Perl_utilize(my_perl=0x0000000801c22000, aver=<unavailable>, floor=603, version=<unavailable>, idop=<unavailable>, arg=<unavailable>) at op.c:8821:5
frame #22: 0x0000000000484c5d perl`Perl_yyparse(my_perl=0x0000000801c22000, gramtype=<unavailable>) at perly.y:347:6
frame #23: 0x000000000054df6c perl`S_doeval_compile(my_perl=0x0000000801c22000, gimme=<unavailable>, outside=<unavailable>, seq=<unavailable>, hh=<unavailable>) at pp_ctl.c:3540:77
frame #24: 0x000000000054c7d7 perl`Perl_pp_require [inlined] S_require_file(my_perl=<unavailable>, sv=<unavailable>) at pp_ctl.c:4363:9
frame #25: 0x000000000054b41d perl`Perl_pp_require(my_perl=0x0000000801c22000) at pp_ctl.c:4387
frame #26: 0x00000000004f3d06 perl`Perl_runops_standard(my_perl=0x0000000801c22000) at run.c:41:26
frame #27: 0x000000000044ec83 perl`Perl_call_sv(my_perl=0x0000000801c22000, sv=0x00000008120a5900, flags=13) at perl.c:3089:6
frame #28: 0x000000000044b2e8 perl`Perl_call_list(my_perl=0x0000000801c22000, oldscope=11, paramList=0x000000081233f480) at perl.c:5128:6
frame #29: 0x0000000000438e38 perl`S_process_special_blocks(my_perl=0x0000000801c22000, floor=491, fullname=<unavailable>, gv=0x00000008120a5c48, cv=0x00000008120a5900) at op.c:11733:6
frame #30: 0x0000000000430436 perl`Perl_newATTRSUB_x(my_perl=0x0000000801c22000, floor=491, o=<unavailable>, proto=<unavailable>, attrs=0x0000000000000000, block=0x0000000819d48a90, o_is_gv=<unavailable>) at op.c:11659:21
frame #31: 0x0000000000431923 perl`Perl_utilize(my_perl=0x0000000801c22000, aver=<unavailable>, floor=491, version=<unavailable>, idop=<unavailable>, arg=<unavailable>) at op.c:8821:5
frame #32: 0x0000000000484c5d perl`Perl_yyparse(my_perl=0x0000000801c22000, gramtype=<unavailable>) at perly.y:347:6
frame #33: 0x000000000054df6c perl`S_doeval_compile(my_perl=0x0000000801c22000, gimme=<unavailable>, outside=<unavailable>, seq=<unavailable>, hh=<unavailable>) at pp_ctl.c:3540:77
frame #34: 0x000000000054c7d7 perl`Perl_pp_require [inlined] S_require_file(my_perl=<unavailable>, sv=<unavailable>) at pp_ctl.c:4363:9
frame #35: 0x000000000054b41d perl`Perl_pp_require(my_perl=0x0000000801c22000) at pp_ctl.c:4387
frame #36: 0x00000000004f3d06 perl`Perl_runops_standard(my_perl=0x0000000801c22000) at run.c:41:26
frame #37: 0x000000000044ec83 perl`Perl_call_sv(my_perl=0x0000000801c22000, sv=0x00000008103dd3c0, flags=13) at perl.c:3089:6
frame #38: 0x000000000044b2e8 perl`Perl_call_list(my_perl=0x0000000801c22000, oldscope=7, paramList=0x00000008103ddb28) at perl.c:5128:6
frame #39: 0x0000000000438e38 perl`S_process_special_blocks(my_perl=0x0000000801c22000, floor=379, fullname=<unavailable>, gv=0x000000080fb22f30, cv=0x00000008103dd3c0) at op.c:11733:6
frame #40: 0x0000000000430436 perl`Perl_newATTRSUB_x(my_perl=0x0000000801c22000, floor=379, o=<unavailable>, proto=<unavailable>, attrs=0x0000000000000000, block=0x0000000819d48690, o_is_gv=<unavailable>) at op.c:11659:21
frame #41: 0x0000000000431923 perl`Perl_utilize(my_perl=0x0000000801c22000, aver=<unavailable>, floor=379, version=<unavailable>, idop=<unavailable>, arg=<unavailable>) at op.c:8821:5
frame #42: 0x0000000000484c5d perl`Perl_yyparse(my_perl=0x0000000801c22000, gramtype=<unavailable>) at perly.y:347:6
frame #43: 0x000000000054df6c perl`S_doeval_compile(my_perl=0x0000000801c22000, gimme=<unavailable>, outside=<unavailable>, seq=<unavailable>, hh=<unavailable>) at pp_ctl.c:3540:77
frame #44: 0x000000000054c7d7 perl`Perl_pp_require [inlined] S_require_file(my_perl=<unavailable>, sv=<unavailable>) at pp_ctl.c:4363:9
frame #45: 0x000000000054b41d perl`Perl_pp_require(my_perl=0x0000000801c22000) at pp_ctl.c:4387
frame #46: 0x00000000004f3d06 perl`Perl_runops_standard(my_perl=0x0000000801c22000) at run.c:41:26
frame #47: 0x000000000044ec83 perl`Perl_call_sv(my_perl=0x0000000801c22000, sv=0x000000081c9ac8d0, flags=13) at perl.c:3089:6
frame #48: 0x000000000044b2e8 perl`Perl_call_list(my_perl=0x0000000801c22000, oldscope=3, paramList=0x00000008123f9c78) at perl.c:5128:6
frame #49: 0x0000000000438e38 perl`S_process_special_blocks(my_perl=0x0000000801c22000, floor=258, fullname=<unavailable>, gv=0x0000000816fd3ee8, cv=0x000000081c9ac8d0) at op.c:11733:6
frame #50: 0x0000000000430436 perl`Perl_newATTRSUB_x(my_perl=0x0000000801c22000, floor=258, o=<unavailable>, proto=<unavailable>, attrs=0x0000000000000000, block=0x0000000819d481f8, o_is_gv=<unavailable>) at op.c:11659:21
frame #51: 0x0000000000431923 perl`Perl_utilize(my_perl=0x0000000801c22000, aver=<unavailable>, floor=258, version=<unavailable>, idop=<unavailable>, arg=<unavailable>) at op.c:8821:5
frame #52: 0x0000000000484c5d perl`Perl_yyparse(my_perl=0x0000000801c22000, gramtype=<unavailable>) at perly.y:347:6
frame #53: 0x000000000054df6c perl`S_doeval_compile(my_perl=0x0000000801c22000, gimme=<unavailable>, outside=<unavailable>, seq=<unavailable>, hh=<unavailable>) at pp_ctl.c:3540:77
frame #54: 0x000000000054d85d perl`Perl_pp_entereval(my_perl=<unavailable>) at pp_ctl.c:4519:9
frame #55: 0x00000000004f3d06 perl`Perl_runops_standard(my_perl=0x0000000801c22000) at run.c:41:26
frame #56: 0x000000000044e4aa perl`S_run_body(my_perl=0x0000000801c22000, oldscope=<unavailable>) at perl.c:0
frame #57: 0x000000000044e291 perl`perl_run(my_perl=0x0000000801c22000) at perl.c:2682:2
frame #58: 0x00000000004213e0 perl`main(argc=<unavailable>, argv=<unavailable>, env=0x00007fffffffe150) at perlmain.c:127:9
frame #59: 0x000000000042115d perl`_start + 141

Steps to Reproduce
Unknown

Expected behavior
Should not seg fault.

Perl configuration

Summary of my perl5 (revision 5 version 31 subversion 11) configuration:
   
  Platform:
    osname=freebsd
    osvers=11.3-release
    archname=amd64-freebsd-thread-multi
    uname='freebsd freebsd11-2.build.lan 11.3-release freebsd 11.3-release #0 r349754: fri jul 5 04:45:24 utc 2019 root@releng2.nyi.freebsd.org:usrobjusrsrcsysgeneric amd64 '
    config_args='-de -Duserelocatableinc -Dusethreads -Dlocincpth=none -Dloclibpth=none -Dprefix=.../.. -Dbin=bin -Dcc=cc -Dld=clang++ -Dccflags=-DPERL_RELOCATABLE_INCPUSH -fexceptions -maes -fPIC -D_REENTRANT -Aldflags=  -Wl,--rpath,/usr/local/Striata/lib -Wl,--rpath,XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX -Alddlflags= -shared  -Wl,--rpath,/usr/local/Striata/lib -Wl,--rpath,XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX -Dprivlib=.../../lib -Dprivlibexp=.../../lib -Darchlib=.../../lib/freebsd -Darchlibexp=.../../lib/freebsd -Dsitelib=.../../site/lib -Dsitelibexp=.../../site/lib -Dsitearch=.../../site/lib/freebsd -Dsitearchexp=.../../site/lib/freebsd -Dvendorprefix=.../../vendor -Dvendorlib=.../../vendor/lib -Dvendorlibexp=.../../vendor/lib -Dvendorarch=.../../vendor/lib/freebsd -Dvendorarchexp=.../../vendor/lib/freebsd -Dstartperl=#!/usr/home/build/dev/clang-64/medusa-freebsd64/medusa/build/perl/cpan/perl/install/perl/bin/perl -Dperlpath=/usr/home/build/dev/clang-64/medusa-freebsd64/medusa/build/perl/cpan/perl/install/perl/bin/perl -Accflags=-DPERL_USE_SAFE_PUTENV  -Doptimize=-O3 -DDEBUGGING=-g -Duse64bitint -Duse64bitall=define'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-DPERL_RELOCATABLE_INCPUSH -fexceptions -maes -fPIC -D_REENTRANT -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector-strong -D_FORTIFY_SOURCE=2'
    optimize='-O3 -g'
    cppflags='-DPERL_RELOCATABLE_INCPUSH -fexceptions -maes -fPIC -D_REENTRANT -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector-strong'
    ccversion=''
    gccversion='4.2.1 Compatible FreeBSD Clang 8.0.0 (tags/RELEASE_800/final 356365)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='clang++'
    ldflags ='-pthread -Wl,-E   -Wl,--rpath,/usr/local/Striata/lib -Wl,--rpath,XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX -fstack-protector-strong'
    libpth=/usr/lib /usr/local/lib /usr/lib/clang/8.0.0/lib /usr/lib
    libs=-lpthread -ldl -lm -lcrypt -lutil
    perllibs=-lpthread -ldl -lm -lcrypt -lutil
    libc=
    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags='-DPIC -fPIC'
    lddlflags='-shared   -shared -Wl,--rpath,/usr/local/Striata/lib -Wl,--rpath,XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX -fstack-protector-strong'


Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_TIMES
    MULTIPLICITY
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_IMPLICIT_CONTEXT
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_RELOCATABLE_INCPUSH
    PERL_USE_SAFE_PUTENV
    USE_64_BIT_ALL
    USE_64_BIT_INT
    USE_ITHREADS
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
    USE_REENTRANT_API
  Built under freebsd
  Compiled at May  8 2020 17:59:29
  @INC:
    /usr/home/build/dev/clang-64/medusa-freebsd64/medusa/build/perl/cpan/perl/install/perl/vendor/lib/freebsd
    /usr/home/build/dev/clang-64/medusa-freebsd64/medusa/build/perl/cpan/perl/install/perl/vendor/lib
    /usr/home/build/dev/clang-64/medusa-freebsd64/medusa/build/perl/cpan/perl/install/perl/site/lib/freebsd
    /usr/home/build/dev/clang-64/medusa-freebsd64/medusa/build/perl/cpan/perl/install/perl/site/lib
    /usr/home/build/dev/clang-64/medusa-freebsd64/medusa/build/perl/cpan/perl/install/perl/lib/freebsd
    /usr/home/build/dev/clang-64/medusa-freebsd64/medusa/build/perl/cpan/perl/install/perl/lib

[jacquesg]

For what its worth, it seems like this occurs after some interpreters that we created in process have been destroyed.

[jkeenan]

Description

We embed multiple perl interpreters into some of our applications. After upgrading from 5.26.0 to 5.30.2 (also confirmed that it exists on 5.31.11, 5.28.2 is still unknown), we see the following intermittent crash

* thread #1, name = 'perl', stop reason = signal SIGSEGV: invalid address (fault address: 0x408c58a18)
frame #0: 0x00000000005968db perl`Perl__inverse_folds(my_perl=<unavailable>, cp=97, first_folds_to=0x00007fffffffb40c, remaining_folds_to=0x00007fffffffb410) at utf8.c:3436:16
3433 /* 'index' is guaranteed to be non-negative, as this is an inversion map
3434 * that covers all possible inputs. See [perl #133365] */
3435 SSize_t index = _invlist_search(PL_utf8_foldclosures, cp);
-> 3436 I32 base = _Perl_IVCF_invmap[index];
3437
3438 PERL_ARGS_ASSERT__INVERSE_FOLDS;
3439
(lldb) bt
* thread #1, name = 'perl', stop reason = signal SIGSEGV: invalid address (fault address: 0x408c58a18)
* frame #0: 0x00000000005968db perl`Perl__inverse_folds(my_perl=<unavailable>, cp=97, first_folds_to=0x00007fffffffb40c, remaining_folds_to=0x00007fffffffb410) at utf8.c:3436:16
frame #1: 0x00000000004bce96 perl`S_regclass(my_perl=0x0000000801c22000, pRExC_state=0x00007fffffffbf60, flagp=0x00007fffffffb7bc, depth=9, stop_at_1=<unavailable>, allow_mutiple_chars=true, silence_non_portable=<unavailable>, strict=<unavailable>, optimizable=<unavailable>, ret_invlist=0x0000000000000000) at regcomp.c:19332:53

[snip]
/usr/home/build/dev/clang-64/medusa-freebsd64/medusa/build/perl/cpan/perl/install/perl/lib

For reference (only):

$ ack Perl__inverse_folds .
utf8.c
3406:Perl__inverse_folds(pTHX_ const UV cp, U32 * first_folds_to,

embed.h
924:#define _inverse_folds(a,b,c)	Perl__inverse_folds(aTHX_ a,b,c)

proto.h
129:PERL_CALLCONV Size_t	Perl__inverse_folds(pTHX_ const UV cp, U32 * first_folds_to, const U32 ** remaining_folds_to)

[jkeenan]

Description
We embed multiple perl interpreters into some of our applications. After upgrading from 5.26.0 to 5.30.2 (also confirmed that it exists on 5.31.11, 5.28.2 is still unknown), we see the following intermittent crash

* thread #1, name = 'perl', stop reason = signal SIGSEGV: invalid address (fault address: 0x408c58a18)
frame #0: 0x00000000005968db perl`Perl__inverse_folds(my_perl=<unavailable>, cp=97, first_folds_to=0x00007fffffffb40c, remaining_folds_to=0x00007fffffffb410) at utf8.c:3436:16
3433 /* 'index' is guaranteed to be non-negative, as this is an inversion map
3434 * that covers all possible inputs. See [perl #133365] */
3435 SSize_t index = _invlist_search(PL_utf8_foldclosures, cp);
-> 3436 I32 base = _Perl_IVCF_invmap[index];
3437
3438 PERL_ARGS_ASSERT__INVERSE_FOLDS;
3439
(lldb) bt
* thread #1, name = 'perl', stop reason = signal SIGSEGV: invalid address (fault address: 0x408c58a18)
* frame #0: 0x00000000005968db perl`Perl__inverse_folds(my_perl=<unavailable>, cp=97, first_folds_to=0x00007fffffffb40c, remaining_folds_to=0x00007fffffffb410) at utf8.c:3436:16
frame #1: 0x00000000004bce96 perl`S_regclass(my_perl=0x0000000801c22000, pRExC_state=0x00007fffffffbf60, flagp=0x00007fffffffb7bc, depth=9, stop_at_1=<unavailable>, allow_mutiple_chars=true, silence_non_portable=<unavailable>, strict=<unavailable>, optimizable=<unavailable>, ret_invlist=0x0000000000000000) at regcomp.c:19332:53

[snip]
/usr/home/build/dev/clang-64/medusa-freebsd64/medusa/build/perl/cpan/perl/install/perl/lib

For reference (only):

$ ack Perl__inverse_folds .
utf8.c
3406:Perl__inverse_folds(pTHX_ const UV cp, U32 * first_folds_to,

embed.h
924:#define _inverse_folds(a,b,c)	Perl__inverse_folds(aTHX_ a,b,c)

proto.h
129:PERL_CALLCONV Size_t	Perl__inverse_folds(pTHX_ const UV cp, U32 * first_folds_to, const U32 ** remaining_folds_to)

Also in the core distribution, but a bit more generally:

$ ack -il inverse_folds . |sort
dist/Devel-PPPort/parts/base/5027011
dist/Devel-PPPort/parts/embed.fnc
embed.fnc
embed.h
proto.h
regcomp.c
regexec.c
t/re/fold_grind.pl
utf8.c

[tonycoz]

This looks like the same type of issue as #17154

[jacquesg]

I believe it is similar yes, both the reporter in #17154 and ourselves embed perl. Unfortunately the patch in #17154 is present in 5.30.11 which we've also tried to no success.

[khwilliamson]

I have a fix that I hope fixes the problem. If you can reproduce this with any regularity, you could try out https://github.com/Perl/perl5/commits/smoke-me/khw-intrpvar

[jacquesg]

I've applied the patch locally, will put it under some stress.

[jacquesg]

So far no new crashes on 5.31.11. I've applied the patch to 5.30.2 as well, will know soon enough.

[pgul]

Hi.
I also faced this bug (or very similar). Following simple program crashes with segfault or "Out of memory" on Ubuntu 20.04, Fedora 32 and MacOS with perl 5.30.2 (not each run, but often, I think about 50% of calls), but works good with perl 5.26:

#include <EXTERN.h>
#include <perl.h>
#include <XSUB.h>

int main(int argc, char** argv, char** env) {
  int i;

  PERL_SYS_INIT3(&argc, &argv, &env);

  for (i=0; i<2; i++) {
    PerlInterpreter *perl;
    char *args[] = { "perl",  "-e", "sub some_func { 'some line' =~ /[ab]/ }", NULL };

    perl = perl_alloc();
    PL_perl_destruct_level = 1;
    perl_construct(perl);
    perl_parse(perl, NULL, 3, args, (char **)NULL);
    perl_run(perl);
    PL_perl_destruct_level = 1;
    perl_destruct(perl);
    perl_free(perl);
  }

  PERL_SYS_TERM();
  return 0;
}

Stack trace is:

#0  Perl_croak_no_mem () at util.c:1790
#1  0x00007ffff7d29593 in Perl_safesysmalloc (size=size@entry=93824992293010) at util.c:204
#2  0x00007ffff7d60d08 in Perl_sv_grow (my_perl=my_perl@entry=0x555555588860, sv=sv@entry=0x55555555c028,
    newlen=93824992293010, newlen@entry=93824992293009) at sv.c:1599
#3  0x00007ffff7cf9eca in S_initialize_invlist_guts (initial_size=<optimized out>, invlist=0x55555555c028,
    my_perl=0x555555588860) at regcomp.c:9201
#4  Perl__new_invlist (my_perl=my_perl@entry=0x555555588860, initial_size=<optimized out>, initial_size@entry=11728124036625)
    at regcomp.c:9201
#5  0x00007ffff7cfb2ad in Perl__invlist_intersection_maybe_complement_2nd (my_perl=my_perl@entry=0x555555588860,
    a=<optimized out>, b=<optimized out>, complement_b=complement_b@entry=true, i=i@entry=0x7fffffffd2f8) at regcomp.c:9840
#6  0x00007ffff7cfb895 in S_populate_ANYOF_from_invlist (my_perl=my_perl@entry=0x555555588860, node=0x55555558fa3c,
    invlist_ptr=invlist_ptr@entry=0x7fffffffd2f8) at regcomp.c:14917
#7  0x00007ffff7d0b64d in S_populate_ANYOF_from_invlist (invlist_ptr=0x7fffffffd2f8, node=<optimized out>,
    my_perl=0x555555588860) at regcomp.c:14872
#8  S_regclass (my_perl=my_perl@entry=0x555555588860, pRExC_state=pRExC_state@entry=0x7fffffffd720,
    flagp=flagp@entry=0x7fffffffd480, depth=depth@entry=5, stop_at_1=stop_at_1@entry=false,
    allow_mutiple_chars=<optimized out>, allow_mutiple_chars@entry=true, silence_non_portable=<optimized out>,
    strict=<optimized out>, optimizable=<optimized out>, ret_invlist=<optimized out>) at regcomp.c:19167
#9  0x00007ffff7d1a7ac in S_regatom (depth=4, flagp=0x7fffffffd480, pRExC_state=0x7fffffffd720, my_perl=0x555555588860)
    at regcomp.c:13407
#10 S_regpiece (my_perl=my_perl@entry=0x555555588860, pRExC_state=pRExC_state@entry=0x7fffffffd720,
    flagp=flagp@entry=0x7fffffffd514, depth=depth@entry=3) at regcomp.c:12520
#11 0x00007ffff7d1f1d7 in S_regbranch (my_perl=my_perl@entry=0x555555588860, pRExC_state=pRExC_state@entry=0x7fffffffd720,
    flagp=flagp@entry=0x7fffffffd5b4, first=first@entry=1, depth=depth@entry=2) at regcomp.c:12440
#12 0x00007ffff7d11da4 in S_reg (my_perl=my_perl@entry=0x555555588860, pRExC_state=pRExC_state@entry=0x7fffffffd720,
    paren=paren@entry=0, flagp=flagp@entry=0x7fffffffd6e4, depth=depth@entry=1) at regcomp.c:12145
#13 0x00007ffff7d16ce3 in Perl_re_op_compile (my_perl=0x555555588860, patternp=<optimized out>, pat_count=<optimized out>,
    expr=<optimized out>, eng=0x7ffff7f97260 <PL_core_reg_engine>, old_re=0x0, is_bare_re=0x0, orig_rx_flags=0, pm_flags=0)
    at regcomp.c:7734
#14 0x00007ffff7cb1bfe in Perl_pmruntime (my_perl=my_perl@entry=0x555555588860, o=0x55555558ec98, expr=0x55555558ec58, repl=<optimized out>, flags=flags@entry=1, floor=<optimized out>) at op.c:7128
#15 0x00007ffff7cecf98 in Perl_yyparse (my_perl=my_perl@entry=0x555555588860, gramtype=gramtype@entry=258) at perly.y:1234
#16 0x00007ffff7d8b36b in S_doeval_compile (my_perl=my_perl@entry=0x555555588860, gimme=gimme@entry=2 '\002', outside=outside@entry=0x55555558df28, seq=<optimized out>, hh=hh@entry=0x0) at pp_ctl.c:3526
#17 0x00007ffff7d987c2 in Perl_pp_entereval (my_perl=0x555555588860) at pp_ctl.c:4502
#18 0x00007ffff7cb835d in Perl_eval_sv (my_perl=my_perl@entry=0x555555588860, sv=sv@entry=0x55555558dd30, flags=<optimized out>, flags@entry=2) at perl.c:3157
#19 0x00007ffff7cb853d in Perl_eval_pv (my_perl=0x555555588860, p=<optimized out>, croak_on_error=1) at perl.c:3221
#20 0x0000555555555264 in main (argc=<optimized out>, argv=<optimized out>, env=<optimized out>) at perlembed.c:18

I'll try patch proposed by @khwilliamson and report feedback.

[pgul]

It's not easy to apply the patch to 5.30.2. If I apply the changes from 1c98fee then some tests fail.
@khwilliamson could you please create the fix for 5.30.2 version?

[jacquesg]

I've not seen any more crashes after applying the patch. Will this still make it into 5.32.0?

[xsawyerx]

I might be willing to make an exception for this. (RC0 was out yesterday, so it will go mid-RC.) It depends on approval from @tonycoz and @khwilliamson.

[tonycoz]

It looks fine to me.

Note that this can't be backported if not included in 5.32 (and can't be backported to 5.30) since it would break binary compatibility.