Make IteratorSize(::Iterators.Cycle) not always IsInfinite and document why
#54187
Conversation
Co-authored-by: adienes <[email protected]>
| # IsInfinite() would be false if iterator ever becomes empty | ||
| IteratorSize(I) === IsInfinite() ? IsInfinite() : SizeUnknown() | ||
| end | ||
| IteratorSize(it::Cycle) = isempty(it.xs) ? HasLength() : IsInfinite() |
There was a problem hiding this comment.
Do we have other examples where IteratorSize(it) and IteratorSize(typeof(it)) (potentially) disagree? Makes me a bit uncomfortable.
And if it.xs is stateful, it seems that IsInfinite might turn out to be wrong, because isempty(it.xs) might become true.
There was a problem hiding this comment.
Do we have other examples where IteratorSize(it) and IteratorSize(typeof(it)) (potentially) disagree? Makes me a bit uncomfortable.
No, this is the first case, and changes the docs to suggest the possibility.
And if it.xs is stateful, it seems that IsInfinite might turn out to be wrong, because isempty(it.xs) might become true.
IMO that's because Iterators.cycle on stateful iterators is broken. It is explicitly documented to be infinite for non-empty arguments.
nsajko
added
bugfix
|
This PR includes:
Would it be possible to separate this into multiple PRs? |
| function IteratorSize(::Type{Cycle{I}}) where I | ||
| # TODO: find a better way of communicating the size of a cycle | ||
| # IsInfinite() would be false if iterator ever becomes empty | ||
| IteratorSize(I) === IsInfinite() ? IsInfinite() : SizeUnknown() |
There was a problem hiding this comment.
A refinement, to improve precision:
| IteratorSize(I) === IsInfinite() ? IsInfinite() : SizeUnknown() | |
| c = IteratorSize(I) | |
| ((c === IsInfinite()) || (c === HasShape{0}())) ? IsInfinite() : SizeUnknown() |
The idea is that there are two cases when we know the number of elements is not zero (and thus infinite):
- when
IteratorSize(I) === IsInfinite(), already accounted for, just listing it for completeness - when
IteratorSize(I) === HasShape{0}(), because a zero-dimensional shape implies a length of exactly one
Fixes #53169
Deletes an "XXX: this is false if iterator ever becomes empty" comment