Skip to content

Assertion errors in Py_DEBUG mode when forgetting to untrack GC objects #111777

Closed
@Yhg1s

Description

@Yhg1s

This has been distilled from a crash we found at Google while upgrading to Python 3.11. It's a complicated setup and I have only been able to partially reproduce the issue, but I believe my solution is correct.

When extension types set Py_TPFLAGS_HAVE_GC they should call PyObject_GC_Untrack() before starting destruction of the object so that the GC module doesn't see partially destructed objects if it happens to trigger during it. Python 3.11 started warning when types didn't do this correctly:

cpython/Modules/gcmodule.c

Lines 2398 to 2407 in d78c872

if (_PyObject_GC_IS_TRACKED(op)) {
#ifdef Py_DEBUG
if (PyErr_WarnExplicitFormat(PyExc_ResourceWarning, "gc", 0,
"gc", NULL, "Object of type %s is not untracked before destruction",
((PyObject*)op)->ob_type->tp_name)) {
PyErr_WriteUnraisable(NULL);
}
#endif
gc_list_remove(g);
}

There's two subtle issues here: one is that Py_DECREF(), and thus object deallocation, can be called with a pending exception (it's not unreasonable or even uncommon to do so). The PyErr_WarnExplicitFormat call, however, may (and likely will) call other Python code, which will trigger an assertion that no exception is pending, usually this one:

cpython/Objects/typeobject.c

Lines 4784 to 4785 in d78c872

/* We may end up clearing live exceptions below, so make sure it's ours. */
assert(!PyErr_Occurred());

The code should store any pending exception before calling anything that might care about the exception.

The other subtle issue is that the code warns before untracking the object. The warning (which can call arbitrary Python code via warnings.showwarning) can easily trigger a GC run, which then turns up seeing the still-erroneously-tracked object in an invalid state. The act of warning about a potential issue is triggering the issue (but only with Py_DEBUG defined). As far as I can see there's no reason not to untrack before raising the warning, which would avoid the problem. I have a PR that fixes both these issues.

Linked PRs

Metadata

Metadata

Assignees

Labels

3.11only security fixes3.12only security fixes3.13bugs and security fixes

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions