gh-123471: Make concurrent iteration over itertools.repeat safe under free-threading

[eendebakpt]

If needed we can add a unit test similar to the one in https://github.com/python/cpython/pull/131212/files

• Issue: Make concurrent iteration over pairwise, combinations, permutations, cwr, product, etc. from itertools safe under free-threading #123471

[rhettinger]

Is it possible to create tests demonstrating code the previously failed that can now be counted on to succeed?

[eendebakpt]

@rhettinger The issue that this PR solves is the following: when defining the iterator as repeat(data, 245) and concurrent iteration with multiple threads in this part of the code:

    if (ro->cnt > 0)
        ro->cnt--;

we can have multiple threads passing the condition ro->cnt > 0 (if ro->cnt is 1), and then decrementing ro->cnt to a negative value. The iterator will then nevert be exhausted (even though this is not the intend of the user).

A test for this is:

import unittest
from threading import Thread, Barrier
from itertools import repeat
from test.support import threading_helper


threading_helper.requires_working_threading(module=True)

class ItertoolsThreading(unittest.TestCase):

    @threading_helper.reap_threads
    def test_repeat(self):
        number_of_threads = 20
        number_of_iterations = 700_000
        barrier = Barrier(number_of_threads)
        def work(it):
            barrier.wait()
            while True:
                try:
                    _ = next(it)
                except StopIteration:
                    break

        data = 3
        for it in range(number_of_iterations):
            print(f'iteration {it}')
            repeat_iterator = repeat(data, 245)
            worker_threads = []
            for ii in range(number_of_threads):
                worker_threads.append(
                    Thread(target=work, args=[repeat_iterator]))

            with threading_helper.start_threads(worker_threads):
                pass

            barrier.reset()


if __name__ == "__main__":
    unittest.main()

On my system this creates a deadlock (typically after a couple of 1000) iterations in a free-threading debug build.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot iOS ARM64 Simulator 3.x (tier-3) has failed when building commit 9d127e8.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1380/builds/3222) and take a look at the build logs.
4. Check if the failure is related to this commit (9d127e8) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1380/builds/3222

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/Users/buildbot/Library/Developer/XCTestDevices/7817F54E-F383-435E-A96B-7848C5D4910E/data/Containers/Bundle/Application/6FD8422A-BA2D-42A0-9F8A-786BED06BB0E/iOSTestbed.app/python/lib/python3.14/threading.py", line 1079, in _bootstrap_inner
    self._context.run(self.run)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/7817F54E-F383-435E-A96B-7848C5D4910E/data/Containers/Bundle/Application/6FD8422A-BA2D-42A0-9F8A-786BED06BB0E/iOSTestbed.app/python/lib/python3.14/threading.py", line 1021, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/7817F54E-F383-435E-A96B-7848C5D4910E/data/Containers/Bundle/Application/6FD8422A-BA2D-42A0-9F8A-786BED06BB0E/iOSTestbed.app/python/lib/python3.14/test/test_interpreters/test_stress.py", line 30, in task
    interp = interpreters.create()
  File "/Users/buildbot/Library/Developer/XCTestDevices/7817F54E-F383-435E-A96B-7848C5D4910E/data/Containers/Bundle/Application/6FD8422A-BA2D-42A0-9F8A-786BED06BB0E/iOSTestbed.app/python/lib/python3.14/test/support/interpreters/__init__.py", line 76, in create
    id = _interpreters.create(reqrefs=True)
interpreters.InterpreterError: interpreter creation failed
k

[freakboy3742]

The iOS failure is caused by #127108