os.stat() and os.DirEntry.stat() don't check for fill_time() exception: _Py_CheckSlotResult: Slot * of type int succeeded with an exception set

[vstinner]

Bug report

$ ./python -m test -v test_tools.test_freeze -u all
== CPython 3.12.0rc3+ (heads/3.12-dirty:4a0c118d6a, Sep 20 2023, 16:55:13) [GCC 13.2.1 20230728 (Red Hat 13.2.1-1)]
== Linux-6.4.14-200.fc38.x86_64-x86_64-with-glibc2.37 little-endian
== Python build: debug
== cwd: /home/vstinner/python/3.12/build/test_python_768929æ
== CPU count: 12
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 1.16 Run tests sequentially
0:00:00 load avg: 1.16 [1/1] test_tools.test_freeze
test_freeze_simple_script (test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script) ... creating the script to be frozen at /tmp/tmp_as044dn/app.py
copying the source tree into /tmp/tmp_as044dn/cpython...

^C

Fatal Python error: _Py_CheckSlotResult: Slot * of type int succeeded with an exception set
Python runtime state: initialized
KeyboardInterrupt

Current thread 0x00007fd397f9e740 (most recent call first):
  File "/home/vstinner/python/3.12/Lib/shutil.py", line 376 in copystat
  File "/home/vstinner/python/3.12/Lib/shutil.py", line 536 in _copytree
  File "/home/vstinner/python/3.12/Lib/shutil.py", line 588 in copytree
  File "/home/vstinner/python/3.12/Lib/shutil.py", line 524 in _copytree
  File "/home/vstinner/python/3.12/Lib/shutil.py", line 588 in copytree
  ...

I suppose that long_mul() was called indirectly by os_DirEntry_stat_impl(). Traceback when I put a breakpoint in gdb:

(gdb) where
#0  long_mul (a=0x7fffe97e8ac0, b=0x7fffea5d23c0) at Objects/longobject.c:3956
#1  0x00000000004ca219 in binary_op1 (v=1695221791, w=1000000000, op_slot=16, op_name=0x7adfe4 "*") at Objects/abstract.c:882
#2  0x00000000004cab6b in PyNumber_Multiply (v=1695221791, w=1000000000) at Objects/abstract.c:1100
#3  0x000000000070a0ac in fill_time (module=<module at remote 0x7fffea5ce570>, v=<os.stat_result at remote 0x7fffe97fb150>, s_index=7, 
    f_index=10, ns_index=13, sec=1695221791, nsec=606287902) at ./Modules/posixmodule.c:2400
#4  0x000000000070a398 in _pystat_fromstructstat (module=<module at remote 0x7fffea5ce570>, st=0x7ffffffe8730)
    at ./Modules/posixmodule.c:2508
#5  0x0000000000713e1d in DirEntry_fetch_stat (module=<module at remote 0x7fffea5ce570>, self=0x7fffe97d5ae0, follow_symlinks=0)
    at ./Modules/posixmodule.c:14639
#6  0x0000000000713e64 in DirEntry_get_lstat (defining_class=0xb6dd60, self=0x7fffe97d5ae0) at ./Modules/posixmodule.c:14650
#7  0x0000000000713f2a in os_DirEntry_stat_impl (self=0x7fffe97d5ae0, defining_class=0xb6dd60, follow_symlinks=1)
    at ./Modules/posixmodule.c:14685
#8  0x000000000071dadc in os_DirEntry_stat (self=0x7fffe97d5ae0, defining_class=0xb6dd60, args=0x7ffff7fbafb0, nargs=0, kwnames=0x0)
    at ./Modules/clinic/posixmodule.c.h:10788
#9  0x00000000004feba4 in method_vectorcall_FASTCALL_KEYWORDS_METHOD (func=<method_descriptor at remote 0x7fffea5fc6b0>, 
    args=0x7ffff7fbafa8, nargsf=9223372036854775809, kwnames=0x0) at Objects/descrobject.c:387
#10 0x00000000004ed499 in _PyObject_VectorcallTstate (tstate=0xb5a8e0 <_PyRuntime+475616>, 
    callable=<method_descriptor at remote 0x7fffea5fc6b0>, args=0x7ffff7fbafa8, nargsf=9223372036854775809, kwnames=0x0)
    at ./Include/internal/pycore_call.h:92
#11 0x00000000004ee17d in PyObject_Vectorcall (callable=<method_descriptor at remote 0x7fffea5fc6b0>, args=0x7ffff7fbafa8, 
    nargsf=9223372036854775809, kwnames=0x0) at Objects/call.c:325
#12 0x0000000000644123 in _PyEval_EvalFrameDefault (tstate=0xb5a8e0 <_PyRuntime+475616>, frame=0x7ffff7fbaf38, throwflag=0)
    at Python/bytecodes.c:2711
#13 0x000000000062f7a0 in _PyEval_EvalFrame (tstate=0xb5a8e0 <_PyRuntime+475616>, frame=0x7ffff7fba1c0, throwflag=0)
    at ./Include/internal/pycore_ceval.h:88

Linked PRs

• gh-109613: _pystat_fromstructstat() checks for exceptions #109618
• [3.12] gh-109613: _pystat_fromstructstat() checks for exceptions (GH-109618) #109641
• [3.11] gh-109613: _pystat_fromstructstat() checks for exceptions (#109618) #109668

[vstinner]

I can reproduce this bug with this script:

import os
import signal
import time

start_time = time.monotonic()
deadline = start_time + 10.0
print("Stress test for 10 seconds...")

class GotSignal(Exception):
    pass

got_signals = 0
def sig_alarm(signum, frame):
    global got_signals
    got_signals += 1
    raise GotSignal()

# Send SIGALRM every 10 ms!
signal.signal(signal.SIGALRM, sig_alarm)
signal.setitimer(signal.ITIMER_REAL, 0.010, 0.010)

while time.monotonic() < deadline:
    try:
        for entry in os.scandir('Modules'):
            entry.stat(follow_symlinks=False)
    except GotSignal:
        pass

signal.setitimer(signal.ITIMER_REAL, 0, 0)
dt = time.monotonic() - start_time
print(f"Got {got_signals} SIGALRM signals in {dt:.1f} seconds")

[vstinner]

Output:

$ ./python stress_fill_time.py 
Stress test for 10 seconds...
Fatal Python error: _Py_CheckSlotResult: Slot * of type int succeeded with an exception set
Python runtime state: initialized
Traceback (most recent call last):
  File "/home/vstinner/python/main/stress_fill_time.py", line 16, in sig_alarm
    raise GotSignal()
GotSignal
Abandon (core dumped)

[chgnrdv]

Looks very similar to #102890.

[vstinner]

The problem is that _pystat_fromstructstat() only checks for exceptions just before exit, not after setting each item. Especially, it doesn't check for fill_time() exception, whereas fill_time() can raise an exception by calling a signal handler.

[vstinner]

I closed issue #102890 as a duplicate of this issue.

[vstinner]

@chgnrdv script to reproduce the issue using a thread: #102890 (comment)