-
Notifications
You must be signed in to change notification settings - Fork 21
takeRight and dropRight of views eagerly traverse the underlying collection elements #11275
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Just to point out leading-dot in REPL, in case the feature is not well-known:
|
But that's an important difference. These methods cannot be lazy because they return strict collections. There is no indication in the types that
I don't think that's true. A View represents a reified operation on an Iterator (or on indexed access in case of an IndexedView). Traversing is done with an Iterator, values are computed based on an Iterator, so the complexity is that of accessing the unerlying collection through an Iterator. Any kind of caching in Views is a problem because it goes against the simple model of Views as reified Iterator operations. I don't we should do it unless it's clear from the types (like computing a strict collection or building a View from an Iterator). What we need for |
Yeah, good point.
I agree. |
Small catch: |
Possibly not the best option, but you could convert it to a |
- Add these methods to IterableOnceOps - Implement them in Iterator (with the necessary amount of caching but no more than that) - Move the existing strict implementations from IterableOps to StrictOptimizedIterableOps - Add View.(TakeRight|DropRight) based on Iterator - Move IndexedSeqView implementations of TakeRight, Drop and DropRight up to SeqView - Add new overrides in IndexedSeqView Fixes scala/bug#11275
- Add these methods to IterableOnceOps - Implement them in Iterator (with the necessary amount of caching but no more than that) - Move the existing strict implementations from IterableOps to StrictOptimizedIterableOps - Add View.(TakeRight|DropRight) based on Iterator - Move IndexedSeqView implementations of TakeRight, Drop and DropRight up to SeqView - Add new overrides in IndexedSeqView Fixes scala/bug#11275
I took the direct route with dedicated |
- Move the existing strict implementations from IterableOps to StrictOptimizedIterableOps - Add View.(TakeRight|DropRight) with private Iterator-based implementations that cache as little and as late as possible - Move IndexedSeqView implementations of TakeRight, Drop and DropRight up to SeqView - Add new overrides in IndexedSeqView Fixes scala/bug#11275
Uh oh!
There was an error while loading. Please reload this page.
I’m not sure we can do better, but I’d like to draw attention to the current behaviour of
takeRight
anddropRight
applied to views, at least to agree on what we should expect.This REPL session shows the current behaviour:
takeRight
returns aView
whose elements are eagerly forced, and if we look at the implementation, we see that the elements are collected into anArrayBuffer
, and then a view of thatArrayBuffer
is returned. I see two problems with that: (1) the fact that the returned view elements are eagerly evaluated breaks the property of transformation operations applied to views being lazy, and (2) an intermediateArrayBuffer
is used, although the purpose of using views is to not create intermediate data structures.That being said, unlike
take
, we can not implementtakeRight
on views in a fully lazy way because we have to reach the end of the underlying collection to know that we can start emitting elements. So, the best we could do would be to delay the time we traverse the underlying collection to the time when at least one element of the view is effectively accessed. Doing that would fix the two mentioned issues.However, that’s not the end of the story because this solution would break another (maybe informal?) property of views: accessing view elements should have a predictable complexity, and more specifically, it should have the same complexity of accessing elements of the underlying collection. Said otherwise,
head
should beO(1)
on views, but that wouldn’t be the case here because it would require a complete traversal of the underlying collection to get the first element resulting from thetakeRight
operation.So, it seems that the two properties (transformation operations of views should be lazy, and accessing view elements should have the same complexity as accessing the underlying collection elements) can not be satisfied at the same time. Which one should we pick?
My suggestion would be to keep the second property and give up the first one, because we already have some other operations that are not lazy (e.g.
groupBy
,permutations
, …). These operations are not really transformation operations because they don’t return the same collection type but a collection of sub-collections (iegroupBy
returns aMap[K, View[A]]
). Another similar case issorted
, which eagerly evaluates its elements as well (and uses an intermediateArrayBuffer
).If we agree on that, we can still at least improve the current implementation by not creating an intermediate
ArrayBuffer
, likeLazyList
does here: #11083 (comment)The text was updated successfully, but these errors were encountered: