-
Notifications
You must be signed in to change notification settings - Fork 30
Description
Operations on parallel collections do not propagate the taskSupport to the result.
This seems highly unintuitive.
For example, a fellow engineer recently changed code from
par.map(expensiveProcessing)
to
par.filter(isRelevant).map(expensiveProcessing)
losing the taskSupport on the way so that expensiveProcessing
ended up being executed on the global EC which is not intended. Three different engineers (author, code reviewer and me) did not expect this.
Is this on purpose or a bug?
Some research in the library code suggests that there is an intention to set a non-default taskSupport on the result:
-
Combiner has "resultWithTaskSupport" and propagates its
combinerTaskSupport
there. -
ParIterableLike#combinerFactory propagates the taskSupport to
combinerTaskSupport
, but only ifcombiner.canBeShared
(https://github.com/scala/scala-parallel-collections/blob/master/core/src/main/scala/scala/collection/parallel/ParIterableLike.scala#L545)
Not sure whether that makes sense or not, but my intuition would propagate in either case.