Catch accidentally exhausting iterators by repeatedly looping through them

[technillogue]

Is your feature request related to a problem? Please describe

iterator = (ord(char) for char in string.printable)
for spam in range(10):
      for eggs in iterator: # this won't work as expected and is really hard to catch!
            print(chr(spam*eggs))

Describe the solution you'd like

W\d+: Repeatedly looping through iterator (looping-through-iterator)

Additional context

I realize it's easy to have false positives here, but if we limit to cases where it's clear that it's a non-infinite iterator (e.g. list(iterable), comprehensions, for loops without return/break/continue) I think the main problem is that I could have used range(1) above and that would give a false positive, but that seems like fairly bad style anyway.

[PCManticore]

Yeah, this might be tricky to get right, but it would an useful check to have.

[ranadheerg]

I am interested in working on this issue. Please assign it to me

[Pierre-Sassoulas]

Done, thank you For your interest !

[ranadheerg]

Hi I've started working on a new checker to detect reuse of iterators and would love some feedback before I open a PR.

Summary

This checker aims to detect cases where a generator or iterator is reused in ways that may lead to bugs or silent data loss.

Approach

• Detects iterator assignments from generator expressions, iter(), map(), etc.
• Tracks use across for, while, and next() calls
• Detects reuse inside function calls and methods (e.g., if an iterator is passed and reused)
• Handles some cases involving instance attributes and aliasing

Example Test Cases

Reuse in function:

def process(it):
    for x in it:
        print(x)

it = (x for x in range(3))
process(it)
process(it)  # <-- should warn

There are more cases that I am trying to extend this warning to. Some of them would be

multiple assignments, unpacking, and aliasing

Iterator as a class attribute

Here's a test case where the iterator is stored as an instance attribute. I'm checking whether this triggers a warning on second use:

class A:
    def __init__(self):
        self.iterator = (x for x in range(3))

a = A()
for i in a.iterator:
    pass
for i in a.iterator:  # Might not warn yet (complex access)
    pass

Partial consumption (Use next() once and then use for loop after that) should warn
Here's a test case where the iterator is consumed before used in for loop. I'm checking whether this triggers a warning on second use:

it = (x for x in range(3))
print(next(it))  # Partially consumed
for x in it:  # Should warn
    print(x)

I'm still working on edge cases like:

1. Multiple assignments and unpacking

2. Aliasing

3. Iterator stored in containers (e.g., lists, dicts)

Would appreciate any feedback on whether this direction makes sense for Pylint, and how deep this checker should go in detecting reuse.

Thanks!

[DanielNoord]

I think your examples make a lot of sense, but I would probably start with the smallest of smallest warning you can think of.
For example, re-use of an iterator within the same scope. Otherwise, there are simply too many edge cases to consider. In your initial example, for example, what should happen if instead of print there would be a break. Or a break within an if. That would mean the iterator isn't exhausted and therefore you would raise a false positive.

[Pierre-Sassoulas]

Agree with Daniel, let's start small and let's see how the primer react (running the new check on some well established repositories which is included in the continuous integration). Then we can add use cases if the cost both in terms of implementation and inference at run time is not too high.

[ranadheerg]

Thanks for the helpful feedback! That makes sense — I’ll narrow the checker to only detect definite re-use within the same scope for now, where it’s clear the iterator was exhausted (e.g., a for-loop without early exits). I’ll avoid uncertain cases like partial consumption or conditionally broken loops for now.

Once that’s working cleanly, I can revisit edge cases or broader reuse patterns in a follow-up proposal.

Will share a PR soon!

[technillogue]

very excited someone is working on this issue!