Skip to content

Memory leaks in free-threaded build at shutdown #122697

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
colesbury opened this issue Aug 5, 2024 · 0 comments
Closed

Memory leaks in free-threaded build at shutdown #122697

colesbury opened this issue Aug 5, 2024 · 0 comments
Labels
3.14 bugs and security fixes topic-free-threading type-bug An unexpected behavior, bug, or error

Comments

@colesbury
Copy link
Contributor

colesbury commented Aug 5, 2024

Bug report

The -X showrefcount isn't reporting leaked blocks on exit and we're hiding some leaked objects on shutdown. Note that we're still able to catch most leaks via the refleaks test.

-X showrefcount

_Py_GetGlobalAllocatedBlocks() is returning zero because the main interpreter is already NULL. This works in the default build because we've stashed away the number of leaked blocks in _PyRuntime.obmalloc.interpreter_leaks. We should do the same thing in the free-threaded build by updating _PyInterpreterState_FinalizeAllocatedBlocks().

Leaked objects on exit

Some objects that are referenced by global modules and use deferred reference counting are leaked on exit. Objects that use deferred reference counting are only collected by the GC. However, these object still have valid, live references when we run the GC for the final time. For example, one leaked object is a member_descriptor whose last reference is removed in clear_static_type_objects.

I think we'll cause too many problems if we try to invoke the GC later. Instead, I think that the free-threaded GC should disable deferred reference counting on all objects when its invoked for the final time.

Linked PRs

@colesbury colesbury added type-bug An unexpected behavior, bug, or error topic-free-threading 3.14 bugs and security fixes labels Aug 5, 2024
colesbury added a commit to colesbury/cpython that referenced this issue Aug 5, 2024
We were not properly accounting for interpreter memory leaks at
shutdown and had two sources of leaks:

 * Objects that use deferred reference counting and were reachable via
   static types outlive the final GC. We now disable deferred reference
   counting on all objects if we are calling the GC due to interpreter
   shutdown.

 * `_PyMem_FreeDelayed` did not properly check for interpreter shutdown
   so we had some memory blocks that were enqueued to be freed, but
   never actually freed.
colesbury added a commit to colesbury/cpython that referenced this issue Aug 6, 2024
colesbury added a commit to colesbury/cpython that referenced this issue Aug 7, 2024
colesbury added a commit that referenced this issue Aug 8, 2024
We were not properly accounting for interpreter memory leaks at
shutdown and had two sources of leaks:

 * Objects that use deferred reference counting and were reachable via
   static types outlive the final GC. We now disable deferred reference
   counting on all objects if we are calling the GC due to interpreter
   shutdown.

 * `_PyMem_FreeDelayed` did not properly check for interpreter shutdown
   so we had some memory blocks that were enqueued to be freed, but
   never actually freed.

 * `_PyType_FinalizeIdPool` wasn't called at interpreter shutdown.
blhsing pushed a commit to blhsing/cpython that referenced this issue Aug 22, 2024
…122703)

We were not properly accounting for interpreter memory leaks at
shutdown and had two sources of leaks:

 * Objects that use deferred reference counting and were reachable via
   static types outlive the final GC. We now disable deferred reference
   counting on all objects if we are calling the GC due to interpreter
   shutdown.

 * `_PyMem_FreeDelayed` did not properly check for interpreter shutdown
   so we had some memory blocks that were enqueued to be freed, but
   never actually freed.

 * `_PyType_FinalizeIdPool` wasn't called at interpreter shutdown.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.14 bugs and security fixes topic-free-threading type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

1 participant