Skip to content

Heap Types In the Cross-Interpreter Data Registry are Improperly Shared #110310

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ericsnowcurrently opened this issue Oct 3, 2023 · 0 comments
Closed

Heap Types In the Cross-Interpreter Data Registry are Improperly Shared #110310

ericsnowcurrently opened this issue Oct 3, 2023 · 0 comments
Assignees
@ericsnowcurrently
Labels
3.12 only security fixes 3.13 bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-subinterpreters type-bug An unexpected behavior, bug, or error

Comments

@ericsnowcurrently
Copy link
Member

ericsnowcurrently commented Oct 3, 2023 •
edited by bedevere-app bot
Loading

This can cause a weakref for a class to be decref'ed in the wrong interpreter (or even when the GIL isn't held). The solution is to have a separate registry for each interpreter for heap types. Static types would stay global.

Linked PRs
@ericsnowcurrently ericsnowcurrently added type-bug An unexpected behavior, bug, or error interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-subinterpreters 3.12 only security fixes 3.13 bugs and security fixes labels Oct 3, 2023
@ericsnowcurrently ericsnowcurrently self-assigned this Oct 3, 2023
@bedevere-app bedevere-app bot mentioned this issue Oct 3, 2023
Merged
ericsnowcurrently added a commit that referenced this issue Oct 4, 2023
@ericsnowcurrently
gh-110310: Add a Per-Interpreter XID Registry for Heap Types (gh-110311)
80dc39e 
We do the following:

* add a per-interpreter XID registry (PyInterpreterState.xidregistry)
* put heap types there (keep static types in _PyRuntimeState.xidregistry)
* clear the registries during interpreter/runtime finalization
* avoid duplicate entries in the registry (when _PyCrossInterpreterData_RegisterClass() is called more than once for a type)
* use Py_TYPE() instead of PyObject_Type() in _PyCrossInterpreterData_Lookup()

The per-interpreter registry helps preserve isolation between interpreters.  This is important when heap types are registered, which is something we haven't been doing yet but I will likely do soon.
@github-project-automation github-project-automation bot moved this from Todo to Done in Subinterpreters Oct 4, 2023
ericsnowcurrently added a commit to ericsnowcurrently/cpython that referenced this issue Oct 11, 2023
@ericsnowcurrently
[3.12] pythongh-110310: Add a Per-Interpreter XID Registry for Heap T…
71c96b5 
…ypes (pythongh-110311)

We do the following:

* add a per-interpreter XID registry (PyInterpreterState.xidregistry)
* put heap types there (keep static types in _PyRuntimeState.xidregistry)
* clear the registries during interpreter/runtime finalization
* avoid duplicate entries in the registry (when _PyCrossInterpreterData_RegisterClass() is called more than once for a type)
* use Py_TYPE() instead of PyObject_Type() in _PyCrossInterpreterData_Lookup()

The per-interpreter registry helps preserve isolation between interpreters.  This is important when heap types are registered, which is something we haven't been doing yet but I will likely do soon.

(cherry-picked from commit 80dc39e)
This was referenced Oct 11, 2023
Closed
Merged
ericsnowcurrently added a commit that referenced this issue Nov 28, 2023
@ericsnowcurrently
[3.12] gh-110310: Add a Per-Interpreter XID Registry for Heap Types (g…
1e1a30f 
…h-110311) (gh-110714)

We do the following:

* add a per-interpreter XID registry (PyInterpreterState.xidregistry)
* put heap types there (keep static types in _PyRuntimeState.xidregistry)
* clear the registries during interpreter/runtime finalization
* avoid duplicate entries in the registry (when _PyCrossInterpreterData_RegisterClass() is called more than once for a type)
* use Py_TYPE() instead of PyObject_Type() in _PyCrossInterpreterData_Lookup()

The per-interpreter registry helps preserve isolation between interpreters.  This is important when heap types are registered, which is something we haven't been doing yet but I will likely do soon.

(cherry-picked from commit 80dc39e)
Glyphack pushed a commit to Glyphack/cpython that referenced this issue Sep 2, 2024
@ericsnowcurrently @Glyphack
pythongh-110310: Add a Per-Interpreter XID Registry for Heap Types (p…
8ed5f7e 
…ythongh-110311)

We do the following:

* add a per-interpreter XID registry (PyInterpreterState.xidregistry)
* put heap types there (keep static types in _PyRuntimeState.xidregistry)
* clear the registries during interpreter/runtime finalization
* avoid duplicate entries in the registry (when _PyCrossInterpreterData_RegisterClass() is called more than once for a type)
* use Py_TYPE() instead of PyObject_Type() in _PyCrossInterpreterData_Lookup()

The per-interpreter registry helps preserve isolation between interpreters.  This is important when heap types are registered, which is something we haven't been doing yet but I will likely do soon.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ericsnowcurrently ericsnowcurrently

Labels
3.12 only security fixes 3.13 bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-subinterpreters type-bug An unexpected behavior, bug, or error
Projects
Subinterpreters
Status: Done
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ericsnowcurrently