Skip to content

gh-108732: include comprehension locals in frame.f_locals #109026

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 7, 2023

Conversation

carljm
Copy link
Member

@carljm carljm commented Sep 6, 2023

This applies the same fix that was made for locals() in #105715 to sys._getframe().f_locals as well. This is mostly relevant because pdb uses the latter, so this fix makes these comprehension locals (in module- or class-scope comprehensions) visible to pdb.

I also made one small fix in passing to PyFrame_FastToLocalsWithError; the assertion and the null check were in the wrong order, since the assertion would segfault before ever hitting the null check, if f were null. (Of course if assertions are turned off, the order doesn't matter either way.)

Co-authored-by: Radislav Chugunov <[email protected]>
@vstinner
Copy link
Member

vstinner commented Sep 7, 2023

freebsd Failing

This failure is unrelated to this PR. Apparently, a test_asyncio test is unable on FreeBSD:

FAIL: test_start_tls_server_1 (test.test_asyncio.test_sslproto.SelectorStartTLSTests.test_start_tls_server_1)

I created issue #109051 to track it.

Copy link
Member

@JelleZijlstra JelleZijlstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good apart from a suggestion in the NEWS entry.

@carljm carljm added the needs backport to 3.12 only security fixes label Sep 7, 2023
@carljm carljm merged commit f2584ea into python:main Sep 7, 2023
@miss-islington
Copy link
Contributor

Thanks @carljm for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

@carljm carljm deleted the framelocals branch September 7, 2023 14:56
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Sep 7, 2023
…onGH-109026)

(cherry picked from commit f2584ea)

Co-authored-by: Carl Meyer <[email protected]>
Co-authored-by: Radislav Chugunov <[email protected]>
Co-authored-by: Jelle Zijlstra <[email protected]>
@bedevere-bot
Copy link

GH-109097 is a backport of this pull request to the 3.12 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.12 only security fixes label Sep 7, 2023
Yhg1s pushed a commit that referenced this pull request Sep 12, 2023
…109026) (#109097)

gh-108732: include comprehension locals in frame.f_locals (GH-109026)
(cherry picked from commit f2584ea)

Co-authored-by: Carl Meyer <[email protected]>
Co-authored-by: Radislav Chugunov <[email protected]>
Co-authored-by: Jelle Zijlstra <[email protected]>
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot x86 Gentoo Installed with X 3.12 has failed when building commit d533ab1.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1136/builds/320) and take a look at the build logs.
  4. Check if the failure is related to this commit (d533ab1) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1136/builds/320

Failed tests:

  • test_io

Failed subtests:

  • test_readinto_buffer_overflow - test.test_io.CMiscIOTest.test_readinto_buffer_overflow
  • test_daemon_threads_shutdown_stderr_deadlock - test.test_io.CMiscIOTest.test_daemon_threads_shutdown_stderr_deadlock
  • test_text_encoding - test.test_io.CMiscIOTest.test_text_encoding
  • test_nonblock_pipe_write_bigbuf - test.test_io.PyMiscIOTest.test_nonblock_pipe_write_bigbuf
  • test_daemon_threads_shutdown_stdout_deadlock - test.test_io.CMiscIOTest.test_daemon_threads_shutdown_stdout_deadlock
  • test_text_encoding - test.test_io.PyMiscIOTest.test_text_encoding
  • test_nonblock_pipe_write_bigbuf - test.test_io.CMiscIOTest.test_nonblock_pipe_write_bigbuf
  • test_nonblock_pipe_write_smallbuf - test.test_io.PyMiscIOTest.test_nonblock_pipe_write_smallbuf
  • test_nonblock_pipe_write_smallbuf - test.test_io.CMiscIOTest.test_nonblock_pipe_write_smallbuf

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/test_io.py", line 4269, in test_nonblock_pipe_write_bigbuf
    self._test_nonblock_pipe_write(16*1024)
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/test_io.py", line 4308, in _test_nonblock_pipe_write
    received.append(rf.read())
                    ^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/_pyio.py", line 1068, in read
    return self._read_unlocked(size)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/_pyio.py", line 1080, in _read_unlocked
    chunk = self.raw.readall()
            ^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/_pyio.py", line 1686, in readall
    result += chunk
MemoryError


Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/test_io.py", line 4275, in test_nonblock_pipe_write_smallbuf
    self._test_nonblock_pipe_write(1024)
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/test_io.py", line 4308, in _test_nonblock_pipe_write
    received.append(rf.read())
                    ^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/_pyio.py", line 1068, in read
    return self._read_unlocked(size)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/_pyio.py", line 1080, in _read_unlocked
    chunk = self.raw.readall()
            ^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/_pyio.py", line 1686, in readall
    result += chunk
MemoryError


Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/test_io.py", line 4420, in test_text_encoding
    proc = assert_python_ok('-X', 'utf8=0', '-c', code)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/support/script_helper.py", line 166, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/support/script_helper.py", line 149, in _assert_python
    res, cmd_line = run_python_until_end(*args, **env_vars)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/support/script_helper.py", line 134, in run_python_until_end
    proc = subprocess.Popen(cmd_line, stdin=subprocess.PIPE,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/subprocess.py", line 1906, in _execute_child
    part = os.read(errpipe_read, 50000)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
MemoryError


Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/test_io.py", line 4269, in test_nonblock_pipe_write_bigbuf
    self._test_nonblock_pipe_write(16*1024)
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/test_io.py", line 4308, in _test_nonblock_pipe_write
    received.append(rf.read())
                    ^^^^^^^^^
MemoryError


Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/test_io.py", line 4301, in _test_nonblock_pipe_write
    wf.write(msg)
BlockingIOError: [Errno 12] write could not complete without blocking


Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/test_io.py", line 4305, in _test_nonblock_pipe_write
    self.assertEqual(e.args[0], errno.EAGAIN)
AssertionError: 12 != 11


Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/test_io.py", line 4437, in test_readinto_buffer_overflow
    self.assertRaises(ValueError, bufio.readinto, b)
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/unittest/case.py", line 778, in assertRaises
    return context.handle('assertRaises', args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/unittest/case.py", line 238, in handle
    callable_obj(*args, **kwargs)
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/test_io.py", line 4434, in read
    return b'x' * 10**6
           ~~~~~^~~~~~~
MemoryError


Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/test_io.py", line 4269, in test_nonblock_pipe_write_bigbuf
    self._test_nonblock_pipe_write(16*1024)
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/test_io.py", line 4294, in _test_nonblock_pipe_write
    with rf, wf:
BlockingIOError: [Errno 11] write could not complete without blocking


Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/test_io.py", line 4483, in test_daemon_threads_shutdown_stdout_deadlock
    self.check_daemon_threads_shutdown_deadlock('stdout')
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/test_io.py", line 4467, in check_daemon_threads_shutdown_deadlock
    res, _ = run_python_until_end("-c", code)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/support/script_helper.py", line 134, in run_python_until_end
    proc = subprocess.Popen(cmd_line, stdin=subprocess.PIPE,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/subprocess.py", line 1906, in _execute_child
    part = os.read(errpipe_read, 50000)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
MemoryError


Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/test_io.py", line 4488, in test_daemon_threads_shutdown_stderr_deadlock
    self.check_daemon_threads_shutdown_deadlock('stderr')
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/test_io.py", line 4467, in check_daemon_threads_shutdown_deadlock
    res, _ = run_python_until_end("-c", code)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/support/script_helper.py", line 139, in run_python_until_end
    out, err = proc.communicate()
               ^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/subprocess.py", line 1209, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/subprocess.py", line 2128, in _communicate
    data = os.read(key.fd, 32768)
           ^^^^^^^^^^^^^^^^^^^^^^
MemoryError


Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/test_io.py", line 4275, in test_nonblock_pipe_write_smallbuf
    self._test_nonblock_pipe_write(1024)
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.installed/build/target/lib/python3.12/test/test_io.py", line 4308, in _test_nonblock_pipe_write
    received.append(rf.read())
                    ^^^^^^^^^
MemoryError

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot x86 Gentoo Non-Debug with X 3.12 has failed when building commit d533ab1.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1191/builds/317) and take a look at the build logs.
  4. Check if the failure is related to this commit (d533ab1) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1191/builds/317

Failed tests:

  • test_io

Failed subtests:

  • test_readinto_buffer_overflow - test.test_io.CMiscIOTest.test_readinto_buffer_overflow
  • test_daemon_threads_shutdown_stderr_deadlock - test.test_io.CMiscIOTest.test_daemon_threads_shutdown_stderr_deadlock
  • test_text_encoding - test.test_io.CMiscIOTest.test_text_encoding
  • test_daemon_threads_shutdown_stdout_deadlock - test.test_io.CMiscIOTest.test_daemon_threads_shutdown_stdout_deadlock
  • test_nonblock_pipe_write_bigbuf - test.test_io.CMiscIOTest.test_nonblock_pipe_write_bigbuf
  • test_nonblock_pipe_write_smallbuf - test.test_io.CMiscIOTest.test_nonblock_pipe_write_smallbuf

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/test/test_io.py", line 4437, in test_readinto_buffer_overflow
    self.assertRaises(ValueError, bufio.readinto, b)
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/unittest/case.py", line 778, in assertRaises
    return context.handle('assertRaises', args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/unittest/case.py", line 238, in handle
    callable_obj(*args, **kwargs)
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/test/test_io.py", line 4434, in read
    return b'x' * 10**6
           ~~~~~^~~~~~~
MemoryError


Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/test/test_io.py", line 4420, in test_text_encoding
    proc = assert_python_ok('-X', 'utf8=0', '-c', code)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/test/support/script_helper.py", line 166, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/test/support/script_helper.py", line 149, in _assert_python
    res, cmd_line = run_python_until_end(*args, **env_vars)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/test/support/script_helper.py", line 134, in run_python_until_end
    proc = subprocess.Popen(cmd_line, stdin=subprocess.PIPE,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/subprocess.py", line 1906, in _execute_child
    part = os.read(errpipe_read, 50000)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
MemoryError


Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/test/test_io.py", line 4488, in test_daemon_threads_shutdown_stderr_deadlock
    self.check_daemon_threads_shutdown_deadlock('stderr')
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/test/test_io.py", line 4467, in check_daemon_threads_shutdown_deadlock
    res, _ = run_python_until_end("-c", code)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/test/support/script_helper.py", line 139, in run_python_until_end
    out, err = proc.communicate()
               ^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/subprocess.py", line 1209, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/subprocess.py", line 2128, in _communicate
    data = os.read(key.fd, 32768)
           ^^^^^^^^^^^^^^^^^^^^^^
MemoryError


Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/test/test_io.py", line 4269, in test_nonblock_pipe_write_bigbuf
    self._test_nonblock_pipe_write(16*1024)
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/test/test_io.py", line 4308, in _test_nonblock_pipe_write
    received.append(rf.read())
                    ^^^^^^^^^
MemoryError


Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/test/test_io.py", line 4483, in test_daemon_threads_shutdown_stdout_deadlock
    self.check_daemon_threads_shutdown_deadlock('stdout')
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/test/test_io.py", line 4467, in check_daemon_threads_shutdown_deadlock
    res, _ = run_python_until_end("-c", code)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/test/support/script_helper.py", line 134, in run_python_until_end
    proc = subprocess.Popen(cmd_line, stdin=subprocess.PIPE,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/subprocess.py", line 1906, in _execute_child
    part = os.read(errpipe_read, 50000)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
MemoryError


Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/test/test_io.py", line 4275, in test_nonblock_pipe_write_smallbuf
    self._test_nonblock_pipe_write(1024)
  File "/buildbot/buildarea/cpython/3.12.ware-gentoo-x86.nondebug/build/Lib/test/test_io.py", line 4308, in _test_nonblock_pipe_write
    received.append(rf.read())
                    ^^^^^^^^^
MemoryError

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants