Is behaviour of lexicals which are auto-vivified after being strings correct?

[p5pRT]

Migrated from rt.perl.org#27591 (status was 'resolved')

Searchable as RT27591$

[p5pRT]

From @jlokier

Created by @jlokier

perl -le 'for (1..10) { my $x; $x .= "A"; $$x .= "B"; print length($x).", ".length($$x).", $x, $$x"; }'

Prints​:

1, 1, A, B
1, 2, A, BB
1, 3, A, BBB

What a stonking great pair of surprises!

Firstly, that $x can be both a string _and_ a reference at the same
time. That's a surprise to me, but probably not to long-time Perl 5
133t ha%0rz.

Secondly, that $$x is magically not destroyed and recreated, even
though $x is! That, surely, is just plain wrong?

To illustrate why it's dubious, try this​:

perl -le 'for (1..3) { my $x; $$x .= "B"; print length($x).", ".length($$x).", $x, $$x"; }'

The output is​:

17, 1, SCALAR(0x804c988), B
17, 1, SCALAR(0x804c988), B
17, 1, SCALAR(0x804c988), B

Note how _now_ we are still auto-vivifying $$x and appending to it,
but each time round the loop $$x is destroyed and recreated anew. As
expected.

There is some serious mystery in the Perl lexical variable
implementation, and I'm sure it's not compatible with what we expect
lexical variables to do.

It's nice that taking a reference to a lexical (\$x) keeps the lexical
alive. But lexicals that don't go away when there aren't any
program-visible references to them are extremely impolite. Object
users _expect_ objects to be destroyed when a lexical scope is left -
that's one of the lovely things about Perl's lexical model - file
handles being a fine example of those objects. (I've just tried, and
as expected, objects accessed through $$x aren't destroyed when the
scope of $x is left).

More illustrative is this code​:

perl -le 'for (1..3) { my $x="A"; $$x .= "B"; print "$x, $$x"; $x="A"; $$x .= "D"; print "$x, $$x"; }'

A, B
C, D
A, BB
C, DD
A, BBB
C, DDD

Oh. That looks a lot like the string is used as a hash key of some kind.
It looks quite deliberate.

Is there something I've not noticed in the documentation?
Is it even useful? Hmm, lexicals that behave like C local statics...

Seriously, I understand Perl 5 has a _lot_ of oddities in the lexical
variables department, as I seem to be discovering at a remarkeably
high rate. (And I've been using Perl 5 for years; it's strange how
they're all showing up now). I never noticed these in the manual.
Are they know, expected, normal behaviour, or are they really bugs?

In particular, "man perlsub" explains a lot about lexical variables
and mentions nothing about lexicals that retain their value (or aspects
of it) when a scope is exited and reentered, in fact it says this​:

  The parameter list to my() may be assigned to if desired, which allows
  you to initialize your variables. (If no initializer is given for a
  particular variable, it is created with the undefined value.)
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

One could be forgiven for believing that it means C<undef>, not
undefined as in behaviour :)

Btw, C<use strict 'refs'> does prevent these kind of assignments. It's
not the end of the world.

-- Jamie

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl v5.8.0:

Configured by bhcompile'
cf_email='bhcompile at Wed Aug 13 11:45:59 EDT 2003.

Summary of my rderl (revision 5.0 version 8 subversion 0) configuration:
  Platform:
    osname=linux, osvers=2.4.21-1.1931.2.382.entsmp, archname=i386-linux-thread-multi
    uname='linux str'
    config_args='-des -Doptimize=-O2 -g -pipe -march=i386 -mcpu=i686 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dinstallprefix=/usr -Dprefix=/usr -Darchname=i386-linux -Dvendorprefix=/usr -Dsiteprefix=/usr -Dotherlibdirs=/usr/lib/perl5/5.8.0 -Duseshrplib -Dusethreads -Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef'
 useithreads=define usemultiplicity=
    useperlio= d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=un uselongdouble=
    usemymalloc=, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
    optimize='',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include -I/usr/include/gdbm'
    ccversion='', gccversion='3.2.2 20030222 (Red Hat Linux 3.2.2-5)', gccosandvers=''
gccversion='3.2.2 200302'
    intsize=r, longsize=r, ptrsize=5, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long'
k', ivsize=4'
ivtype='l, nvtype='double'
o_nonbl', nvsize=, Off_t='', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='gcc'
l', ldflags =' -L/u'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lgdbm -ldb -ldl -lm -lpthread -lc -lcrypt -lutil
    perllibs=
    libc=/lib/libc-2.3.2.so, so=so, useshrplib=true, libperl=libper
    gnulibc_version='2.3.2'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so', d_dlsymun=undef, ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5/5.8.0/i386-linux-thread-multi/CORE'
    cccdlflags='-fPIC'
ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5', lddlflags='s Unicode/Normalize XS/A'

Locally applied patches:
    MAINT18379


@INC for perl v5.8.0:
    /usr/lib/perl5/5.8.0/i386-linux-thread-multi
    /usr/lib/perl5/5.8.0
    /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.0
    /usr/lib/perl5/site_perl
    /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.0
    /usr/lib/perl5/vendor_perl
    /usr/lib/perl5/5.8.0/i386-linux-thread-multi
    /usr/lib/perl5/5.8.0
    .


Environment for perl v5.8.0:
    HOME=/home/jamie
    LANG=en_GB.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/jamie/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash
    dlflags='-share (unset)

[p5pRT]

From @jlokier

Btw, C<use strict 'refs'> does prevent these kind of assignments. It's
not the end of the world.

Doh! (Slaps face with a wet fish, then realises it was a brick shaped
like a fish.)

Ok, I made a mistake for the first time in my life. Please close the
bug. It is a bug in me, and that's a different list. Please don't
bring it up at parties. Delete it from the archive, I don't want my
mother to see this. I'm going to get cosmetic surgery and a name change.

For the non-record, eyes only, I got hot and bothered when this test
turned into a swap storm​:

  time perl -e 'my $y = "x"x10000; for (1..1000000) { my $x=""; $$x .= $y }'

(What I meant of course was C<my $x=\my $y>).
For brevity, I tend to omit C<use strict> from one-liners.

-- Jamie

[p5pRT]

@rgs - Status changed from 'new' to 'resolved'

[p5pRT]

From perl5-porters@ton.iguana.be

In article <rt-3.0.8-27591-81224.5.31251743874563@​perl.org>,
  Jamie Lokier (via RT) <perlbug-followup@​perl.org> writes​:

# New Ticket Created by Jamie Lokier
# Please include the string​: [perl #27591]
# in the subject line of all future correspondence about this issue.
# <URL​: http​://rt.perl.org​:80/rt3/Ticket/Display.html?id=27591 >

This is a bug report for perl from jamie@​shareable.org,
generated with the help of perlbug 1.34 running under perl v5.8.0.

-----------------------------------------------------------------
[Please enter your report here]

perl -le 'for (1..10) { my $x; $x .= "A"; $$x .= "B"; print length($x).", ".length($$x).", $x, $$x"; }'

Prints​:

1, 1, A, B
1, 2, A, BB
1, 3, A, BBB

What a stonking great pair of surprises!

Not a bug. You are using symbolic references here, so you are manipulating the global
variable $A through your lexical, which is just the NAME of the var.

"use strict" will "fix" this for you. It will error then.

[p5pRT]

From @jlokier

(Ton Hospel) via RT wrote​:

Not a bug. You are using symbolic references here, so you are
manipulating the global variable $A through your lexical, which is
just the NAME of the var.

Yes, thank you.
This should be closed now - I hope you saw my other followup.

"use strict" will "fix" this for you. It will error then.

Yes, it was a failure to use strict for a quick one-liner that had me
confused.

-- Jamie

[p5pRT]

From @mjdominus

"use strict" will "fix" this for you. It will error then.

Yes, it was a failure to use strict for a quick one-liner that had me
confused.

I don't see how it could have been, because your original message said​:

Btw, C<use strict 'refs'> does prevent these kind of assignments. It's
not the end of the world.

It seems to me that the real problem here was that you were using
'strict refs' without understanding what it was supposed to be doing,
or why using strict 'refs' would prevent 'these kinds of
assignments'. The one and only purpose of strict refs is to prevent
exactly 'those kind of assignments', and you must have seen the message​:

  Can't use string ("A") as a SCALAR ref while "strict refs" in use

which I would have thought was a tipoff as to what was happening.

[p5pRT]

From @mjdominus

A, B
C, D
A, BB
C, DD
A, BBB
C, DDD

Oh. That looks a lot like the string is used as a hash key of some kind.

Actually, that is quite an insightful remark.

[p5pRT]

From @jlokier

Mark-Jason Dominus via RT wrote​:

    Can't use string \("A"\) as a SCALAR ref while "strict refs" in use

which I would have thought was a tipoff as to what was happening.

It did, after I'd hit send.

Prior, I'd written C<my $x=""; $$x .= $long_string lots_of_times> and
it just didn't occur to me to think of the non-strict refs when that
went awry. After it I was blind until it dawned on my, just after posting.

Fwiw, I do understand exactly what C<use strit 'refs'> is for. I am
used to accessing globs and I/O slots and package hashes and such
things, so I grok the general idea. It just became strangely
invisible to me for a while.

It didn't help that I'd just been shown some genuine weird behaviour
with real lexicals​: C<for (1..10) { my $x=10 if $_==1;print $x;$x=20 }>.

That looked similar to what I was seeing with this report, hence
thinking with tunnel vision.

The moral I take away​: check bug report examples with C<use strict>
before posting, in case I missed an obvious Perlism.

Thanks for your time; may the next one be a real bug :)

-- Jamie