Race in bytes_hash under free-threading

[hawkinsp]

Bug report

Bug description:

Run the following reproducer under Python 3.13.1t with free-threading enabled:

import concurrent.futures
import functools
import threading

num_threads = 8

def closure(b, x):
  b.wait()
  x.__hash__()

with concurrent.futures.ThreadPoolExecutor(max_workers=num_threads) as executor:
  for _ in range(100):
    b = threading.Barrier(num_threads)
    x = b"hi"
    x = x * 100000
    for _ in range(num_threads):
      executor.submit(functools.partial(closure, b, x))

Get:

WARNING: ThreadSanitizer: data race (pid=1186406)
  Write of size 8 at 0x7fb6ecd80028 by thread T8:
    #0 bytes_hash /usr/local/google/home/phawkins/p/cpython/Objects/bytesobject.c:1590:21 (python3.13+0x1ddaab) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #1 wrap_hashfunc /usr/local/google/home/phawkins/p/cpython/Objects/typeobject.c:8824:11 (python3.13+0x2f8aa6) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #2 wrapperdescr_raw_call /usr/local/google/home/phawkins/p/cpython/Objects/descrobject.c:531:12 (python3.13+0x2001b1) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #3 wrapperdescr_call /usr/local/google/home/phawkins/p/cpython/Objects/descrobject.c:569:14 (python3.13+0x2001b1)
    #4 _PyObject_MakeTpCall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:242:18 (python3.13+0x1ea40c) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #5 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:166:16 (python3.13+0x1eb068) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #6 PyObject_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:327:12 (python3.13+0x1eb068)
    #7 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e24fb) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #8 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #9 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #10 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #11 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x571bb2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #12 partial_vectorcall /usr/local/google/home/phawkins/p/cpython/./Modules/_functoolsmodule.c:252:16 (python3.13+0x571bb2)
    #13 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #14 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #15 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #16 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e46e2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #17 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #18 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #19 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #20 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef5ef) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #21 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef5ef)
    #22 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #23 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #24 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #25 thread_run /usr/local/google/home/phawkins/p/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x564292) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #26 pythread_wrapper /usr/local/google/home/phawkins/p/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4bd637) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)

  Previous read of size 8 at 0x7fb6ecd80028 by thread T4:
    #0 bytes_hash /usr/local/google/home/phawkins/p/cpython/Objects/bytesobject.c:1588:12 (python3.13+0x1dda7d) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #1 wrap_hashfunc /usr/local/google/home/phawkins/p/cpython/Objects/typeobject.c:8824:11 (python3.13+0x2f8aa6) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #2 wrapperdescr_raw_call /usr/local/google/home/phawkins/p/cpython/Objects/descrobject.c:531:12 (python3.13+0x2001b1) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #3 wrapperdescr_call /usr/local/google/home/phawkins/p/cpython/Objects/descrobject.c:569:14 (python3.13+0x2001b1)
    #4 _PyObject_MakeTpCall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:242:18 (python3.13+0x1ea40c) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #5 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:166:16 (python3.13+0x1eb068) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #6 PyObject_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:327:12 (python3.13+0x1eb068)
    #7 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e24fb) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #8 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #9 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #10 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #11 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x571bb2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #12 partial_vectorcall /usr/local/google/home/phawkins/p/cpython/./Modules/_functoolsmodule.c:252:16 (python3.13+0x571bb2)
    #13 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #14 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #15 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #16 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e46e2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #17 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #18 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #19 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #20 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef5ef) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #21 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef5ef)
    #22 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #23 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #24 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #25 thread_run /usr/local/google/home/phawkins/p/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x564292) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #26 pythread_wrapper /usr/local/google/home/phawkins/p/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4bd637) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)

PyBytesObject::ob_shash probably needs to be read and written atomically.

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

• gh-128133: Added relaxed atomics to bytes_hash #128412
• [3.13] gh-128133: use relaxed atomics for hash of bytes (GH-128412) #130022

[colesbury]

I think this is just a matter of using relaxed atomics in bytes_hash to access ob_shash and checking if any of the other uses of ob_shash need atomic operations.

[colesbury]

Patches welcome!

[abhijeetsharma200]

Hi, I would like to work on this issue, but I have not been able to replicate it as shown in the image. I built CPython from the 3.13 branch using the following command to enable free-threading ./configure --without-threads

Please let me know how I can replicate this issue. Thanks!

[colesbury]

The flag for building with free threading enabled is --disable-gil. If you also want a debug build with thread sanitizer, you should use:

./configure --with-pydebug --disable-gil --with-thread-sanitizer

[abhijeetsharma200]

Hi,

Thanks for the comment. However, I am running into the following issue using your command:

./configure --with-pydebug --disable-gil --with-thread-sanitizer

which gives the following error:
checking for --with-mimalloc... configure: error: --disable-gil requires mimalloc memory allocator (--with-mimalloc).

and then using:
./configure --with-pydebug --with-mimalloc --disable-gil --with-thread-sanitizer

gives:
--with-mimalloc... configure: error: mimalloc requires stdatomic.h, use --without-mimalloc to disable mimalloc.

[colesbury]

You'll need a compiler that has support for stdatomic.h. I think GCC 5.1 (9+ years old) or newer should work as will any comparably recent version of clang.

[abhijeetsharma200]

Yup already using that

# gcc --version
gcc (GCC) 14.0.1 20240411 (Red Hat 14.0.1-0)
Copyright (C) 2024 Free Software Foundation, Inc.

[colesbury]

I'm not sure what the issues is then. Make sure you have the dependencies installed and follow the steps at https://devguide.python.org/getting-started/setup-building/#build-dependencies.

Also, it's worth making sure that the compiler used by configure is the same as the run you are running manually. That's not always the case!