Itertools Recipes - iter_index() silently suppresses ValueError

[vbrozik]

Documentation

In Itertools Recipes there is a bug in the iter_index() function. The function silently suppresses ValueError exception raised by a generator given to the argument iterable.

The bug was introduced by this pull request: gh-102088 Optimize iter_index itertools recipe #102360
Commit: 148bde6

Code to reproduce the bug:

def assert_no_value(iterable, forbidden_value):
    """Pass the iterable but raise ValueError if forbidden_value is found."""
    for item in iterable:
        if item == forbidden_value:
            raise ValueError(f'Value {forbidden_value!r} is not allowed.')
        yield item

# Here we should get ValueError exception
# but it is being silently suppressed by iter_index()
list(iter_index(assert_no_value('AABCADEAF', 'B'), 'A'))

Complete notebook reproducing the bug:
https://github.com/vbrozik/python-ntb/blob/main/problems_from_forums/2023-07-24_iter_index.ipynb

Possible solutions which come to my mind:

• revert the commit 148bde6
• check the value of ValueError inside the iter_index() function
• open discussion to determine whether operator.indexOf() should use ValueError

Note: I already mentioned the bug in the package more-itertools which inherited it:
Update recipes.iter_index to match CPython PR 102360 #690

Linked PRs

• gh-107208: Fix iter_index() recipe to not swallow exceptions #108835
• [3.12] gh-107208: Fix iter_index() recipe to not swallow exceptions (gh-108835) #108837

[AlexWaygood]

Cc. @pochmann

[pochmann]

The "Fast path for sequences" btw has the same issue(?), as can be seen with this sequence version of your example:

from collections.abc import Sequence

class AssertNoValue(Sequence):
    def __init__(self, iterable, forbidden_value):
        self.iterable = iterable
        self.forbidden_value = forbidden_value

    def __getitem__(self, index):
        item = self.iterable[index]
        if item == self.forbidden_value:
            raise ValueError(f'Value {self.forbidden_value!r} is not allowed.')
        return item

    def __len__(self):
        return len(self.iterable)

# Here we should get ValueError exception
# but it is being silently suppressed by iter_index()
list(iter_index(AssertNoValue('AABCADEAF', 'B'), 'A'))

Attempt This Online!

Granted, that's perhaps more artificial/unrealistic than an iterator raising a ValueError.

[rhettinger]

Interestingly, indexOf('AABCADEAF', 'X') and indexOf(assert_no_value('AABCADEAF', 'B'), 'X') both raise a ValueError. It seems that the two cases can only be distinguished by examining the exception message.

A more robust way to distinguish between the two cases would be to provide a more refined exception class as was done for FileNotFoundError vs OSError and ModuleNotFoundError vs ImportError:

class ValueNotFoundError(ValueError):
    "Value was not found"

[rhettinger]

The proposal to add ValueNotFoundError did not go well. This is unfortunate because putting in a new exception is a big step that requires a consensus.

Unless someone with more social capital steps forward to champion ValueNotFoundError, it looks like operator.indexOf will remain broken in perpetuity and we will need to roll back @pochmann 's optimization.