scala-parallel-collections/scala.collection.parallel/ParIterableLike

ParIterableLike

trait ParIterableLike[+T, +CC <: ([X] =>> ParIterable[X]), +Repr <: ParIterable[T], +Sequential <: Iterable[T] & IterableOps[T, Iterable, Sequential]] extends IterableOnce[T] with CustomParallelizable[T, Repr] with Parallel with HasNewCombiner[T, Repr]

A template trait for parallel collections of type ParIterable[T].

A template trait for parallel collections of type ParIterable[T].

This is a base trait for Scala parallel collections. It defines behaviour common to all parallel collections. Concrete parallel collections should inherit this trait and ParIterable if they want to define specific combiner factories.

Parallel operations are implemented with divide and conquer style algorithms that parallelize well. The basic idea is to split the collection into smaller parts until they are small enough to be operated on sequentially.

All of the parallel operations are implemented as tasks within this trait. Tasks rely on the concept of splitters, which extend iterators. Every parallel collection defines:

   def splitter: IterableSplitter[T]

which returns an instance of IterableSplitter[T], which is a subtype of Splitter[T]. Splitters have a method remaining to check the remaining number of elements, and method split which is defined by splitters. Method split divides the splitters iterate over into disjunct subsets:

   def split: Seq[Splitter]

which splits the splitter into a sequence of disjunct subsplitters. This is typically a very fast operation which simply creates wrappers around the receiver collection. This can be repeated recursively.

Tasks are scheduled for execution through a scala.collection.parallel.TaskSupport object, which can be changed through the tasksupport setter of the collection.

Method newCombiner produces a new combiner. Combiners are an extension of builders. They provide a method combine which combines two combiners and returns a combiner containing elements of both combiners. This method can be implemented by aggressively copying all the elements into the new combiner or by lazily binding their results. It is recommended to avoid copying all of the elements for performance reasons, although that cost might be negligible depending on the use case. Standard parallel collection combiners avoid copying when merging results, relying either on a two-step lazy construction or specific data-structure properties.

Methods:

   def seq: Sequential
   def par: Repr

produce the sequential or parallel implementation of the collection, respectively. Method par just returns a reference to this parallel collection. Method seq is efficient - it will not copy the elements. Instead, it will create a sequential version of the collection using the same underlying data structure. Note that this is not the case for sequential collections in general - they may copy the elements and produce a different underlying data structure.

The combination of methods toMap, toSeq or toSet along with par and seq is a flexible way to change between different collection types.

Since this trait extends the GenIterable trait, methods like size must also be implemented in concrete collections, while iterator forwards to splitter by default.

Each parallel collection is bound to a specific fork/join pool, on which dormant worker threads are kept. The fork/join pool contains other information such as the parallelism level, that is, the number of processors used. When a collection is created, it is assigned the default fork/join pool found in the scala.parallel package object.

Parallel collections are not necessarily ordered in terms of the foreach operation (see Traversable). Parallel sequences have a well defined order for iterators - creating an iterator and traversing the elements linearly will always yield the same order. However, bulk operations such as foreach, map or filter always occur in undefined orders for all parallel collections.

Existing parallel collection implementations provide strict parallel iterators. Strict parallel iterators are aware of the number of elements they have yet to traverse. It's also possible to provide non-strict parallel iterators, which do not know the number of elements remaining. To do this, the new collection implementation must override isStrictSplitterCollection to false. This will make some operations unavailable.

To create a new parallel collection, extend the ParIterable trait, and implement size, splitter, newCombiner and seq. Having an implicit combiner factory requires extending this trait in addition, as well as providing a companion object, as with regular collections.

Method size is implemented as a constant time operation for parallel collections, and parallel collection operations rely on this assumption.

The higher-order functions passed to certain operations may contain side-effects. Since implementations of bulk operations may not be sequential, this means that side-effects may not be predictable and may produce data-races, deadlocks or invalidation of state if care is not taken. It is up to the programmer to either avoid using side-effects or to use some form of synchronization when accessing mutable data.

Type Params
Repr

the type of the actual collection containing the elements

T

the element type of the collection

trait HasNewCombiner[T, Repr]
trait Parallel
trait CustomParallelizable[T, Repr]
trait Parallelizable[T, Repr]
trait IterableOnce[T]
class Object
trait Matchable
class Any
trait ParIterable[T]
trait ParMap[K, V]
class WithDefault[A, B]
class WithDefault[K, V]
class WithDefault[K, V]
trait ParMap[K, V]
class ParHashMap[K, V]
trait ParMap[K, V]
class ParHashMap[K, V]
class ParTrieMap[K, V]
trait ParSeq[T]
trait ParSeq[T]
class ParRange
class ParVector[T]
trait ParSeq[T]
class ParArray[T]
trait ParSet[T]
trait ParSet[T]
class ParHashSet[T]
trait ParSet[T]
class ParHashSet[T]
trait ParIterable[T]
trait ParIterable[T]
trait ParMapLike[K, V, CC, Repr, Sequential]
trait ParMapLike[K, V, CC, Repr, Sequential]
trait ParMapLike[K, V, CC, Repr, Sequential]
trait ParSeqLike[T, CC, Repr, Sequential]
trait ParSetLike[T, CC, Repr, Sequential]
trait ParSetLike[T, CC, Repr, Sequential]

Type members

Classlikes

protected trait Accessor[R, Tp] extends StrictSplitterCheckTask[R, Tp]

Standard accessor task that iterates over the elements of the collection.

Standard accessor task that iterates over the elements of the collection.

Type Params
R

type of the result of this method (R for result).

Tp

the representation type of the task at hand.

trait BuilderOps[Elem, To]
protected class Copy[U >: T, That](cfactory: CombinerFactory[U, That], val pit: IterableSplitter[T]) extends Transformer[Combiner[U, That], Copy[U, That]]
trait SignallingOps[PI <: DelegatedSignalling]
protected trait StrictSplitterCheckTask[R, Tp] extends Task[R, Tp]
trait TaskOps[R, Tp]
protected trait Transformer[R, Tp] extends Accessor[R, Tp]

Types

type SSCTask[R, Tp] = StrictSplitterCheckTask[R, Tp]

Value members

Abstract methods

def seq: Sequential
def size: Int
def stringPrefix: String

Concrete methods

def ++[U >: T](that: IterableOnce[U]): CC[U]
def /:[S](z: S)(op: (S, T) => S): S
def :\[S](z: S)(op: (T, S) => S): S
def aggregate[S](z: => S)(seqop: (S, T) => S, combop: (S, S) => S): S

Aggregates the results of applying an operator to subsequent elements.

Aggregates the results of applying an operator to subsequent elements.

This is a more general form of fold and reduce. It has similar semantics, but does not require the result to be a supertype of the element type. It traverses the elements in different partitions sequentially, using seqop to update the result, and then applies combop to results from different partitions. The implementation of this operation may operate on an arbitrary number of collection partitions, so combop may be invoked arbitrary number of times.

For example, one might want to process some elements and then produce a Set. In this case, seqop would process an element and append it to the set, while combop would concatenate two sets from different partitions together. The initial value z would be an empty set.

  pc.aggregate(Set[Int]())(_ += process(_), _ ++ _)

Another example is calculating geometric mean from a collection of doubles (one would typically require big doubles for this).

Type Params
S

the type of accumulated results

Value Params
combop

an associative operator used to combine results from different partitions

seqop

an operator used to accumulate results within a partition

z

the initial value for the accumulated result of the partition - this will typically be the neutral element for the seqop operator (e.g. Nil for list concatenation or 0 for summation) and may be evaluated more than once

def collect[S](pf: PartialFunction[T, S]): CC[S]
def copyToArray[U >: T](xs: Array[U]): Unit
def copyToArray[U >: T](xs: Array[U], start: Int): Unit
def copyToArray[U >: T](xs: Array[U], start: Int, len: Int): Unit
def count(p: T => Boolean): Int
def debugBuffer: ArrayBuffer[String]
def drop(n: Int): Repr
def dropWhile(pred: T => Boolean): Repr

Drops all elements in the longest prefix of elements that satisfy the predicate, and returns a collection composed of the remaining elements.

Drops all elements in the longest prefix of elements that satisfy the predicate, and returns a collection composed of the remaining elements.

This method will use indexFlag signalling capabilities. This means that splitters may set and read the indexFlag state. The index flag is initially set to maximum integer value.

Value Params
pred

the predicate used to test the elements

Returns

a collection composed of all the elements after the longest prefix of elements in this parallel iterable that satisfy the predicate pred

def exists(p: T => Boolean): Boolean

Tests whether a predicate holds for some element of this parallel iterable.

Tests whether a predicate holds for some element of this parallel iterable.

This method will use abort signalling capabilities. This means that splitters may send and read abort signals.

Value Params
p

a predicate used to test elements

Returns

true if p holds for some element, false otherwise

def filter(pred: T => Boolean): Repr
def filterNot(pred: T => Boolean): Repr
def find(p: T => Boolean): Option[T]

Finds some element in the collection for which the predicate holds, if such an element exists. The element may not necessarily be the first such element in the iteration order.

Finds some element in the collection for which the predicate holds, if such an element exists. The element may not necessarily be the first such element in the iteration order.

If there are multiple elements obeying the predicate, the choice is nondeterministic.

This method will use abort signalling capabilities. This means that splitters may send and read abort signals.

Value Params
p

predicate used to test the elements

Returns

an option value with the element if such an element exists, or None otherwise

def flatMap[S](f: T => IterableOnce[S]): CC[S]
def fold[U >: T](z: U)(op: (U, U) => U): U

Folds the elements of this sequence using the specified associative binary operator. The order in which the elements are reduced is unspecified and may be nondeterministic.

Folds the elements of this sequence using the specified associative binary operator. The order in which the elements are reduced is unspecified and may be nondeterministic.

Note this method has a different signature than the foldLeft and foldRight methods of the trait Traversable. The result of folding may only be a supertype of this parallel collection's type parameter T.

Type Params
U

a type parameter for the binary operator, a supertype of T.

Value Params
op

a binary operator that must be associative

z

a neutral element for the fold operation, it may be added to the result an arbitrary number of times, not changing the result (e.g. Nil for list concatenation, 0 for addition, or 1 for multiplication)

Returns

the result of applying fold operator op between all the elements and z

def foldLeft[S](z: S)(op: (S, T) => S): S
def foldRight[S](z: S)(op: (T, S) => S): S
def forall(p: T => Boolean): Boolean

Tests whether a predicate holds for all elements of this parallel iterable.

Tests whether a predicate holds for all elements of this parallel iterable.

This method will use abort signalling capabilities. This means that splitters may send and read abort signals.

Value Params
p

a predicate used to test elements

Returns

true if p holds for all elements, false otherwise

def foreach[U](f: T => U): Unit

Applies a function f to all the elements of parallel iterable in an undefined order.

Applies a function f to all the elements of parallel iterable in an undefined order.

Type Params
U

the result type of the function applied to each element, which is always discarded

Value Params
f

function applied to each element

def groupBy[K](f: T => K): ParMap[K, Repr]
def hasDefiniteSize: Boolean
def head: T
def headOption: Option[T]
def init: Repr
protected def initTaskSupport(): Unit
def isEmpty: Boolean
def isStrictSplitterCollection: Boolean

Denotes whether this parallel collection has strict splitters.

Denotes whether this parallel collection has strict splitters.

This is true in general, and specific collection instances may choose to override this method. Such collections will fail to execute methods which rely on splitters being strict, i.e. returning a correct value in the remaining method.

This method helps ensure that such failures occur on method invocations, rather than later on and in unpredictable ways.

final def isTraversableAgain: Boolean
def iterator: Splitter[T]

Creates a new split iterator used to traverse the elements of this collection.

Creates a new split iterator used to traverse the elements of this collection.

By default, this method is implemented in terms of the protected splitter method.

Returns

a split iterator

def last: T
def lastOption: Option[T]
def map[S](f: T => S): CC[S]
def max[U >: T](ord: Ordering[U]): T
def maxBy[S](f: T => S)(cmp: Ordering[S]): T
def min[U >: T](ord: Ordering[U]): T
def minBy[S](f: T => S)(cmp: Ordering[S]): T
def mkString(start: String, sep: String, end: String): String
def mkString(sep: String): String
def mkString: String
def nonEmpty: Boolean
override def par: Repr
def partition(pred: T => Boolean): (Repr, Repr)
def product[U >: T](num: Numeric[U]): U
def reduce[U >: T](op: (U, U) => U): U

Reduces the elements of this sequence using the specified associative binary operator.

Reduces the elements of this sequence using the specified associative binary operator.

The order in which operations are performed on elements is unspecified and may be nondeterministic.

Note this method has a different signature than the reduceLeft and reduceRight methods of the trait Traversable. The result of reducing may only be a supertype of this parallel collection's type parameter T.

Type Params
U

A type parameter for the binary operator, a supertype of T.

Value Params
op

A binary operator that must be associative.

Returns

The result of applying reduce operator op between all the elements if the collection is nonempty.

Throws
UnsupportedOperationException

if this parallel iterable is empty.

def reduceLeft[U >: T](op: (U, T) => U): U
def reduceLeftOption[U >: T](op: (U, T) => U): Option[U]
def reduceOption[U >: T](op: (U, U) => U): Option[U]

Optionally reduces the elements of this sequence using the specified associative binary operator.

Optionally reduces the elements of this sequence using the specified associative binary operator.

The order in which operations are performed on elements is unspecified and may be nondeterministic.

Note this method has a different signature than the reduceLeftOption and reduceRightOption methods of the trait Traversable. The result of reducing may only be a supertype of this parallel collection's type parameter T.

Type Params
U

A type parameter for the binary operator, a supertype of T.

Value Params
op

A binary operator that must be associative.

Returns

An option value containing result of applying reduce operator op between all the elements if the collection is nonempty, and None otherwise.

def reduceRight[U >: T](op: (T, U) => U): U
def reduceRightOption[U >: T](op: (T, U) => U): Option[U]
def repr: Repr
protected def reuse[S, That](oldc: Option[Combiner[S, That]], newc: Combiner[S, That]): Combiner[S, That]

Optionally reuses an existing combiner for better performance. By default it doesn't - subclasses may override this behaviour. The provided combiner oldc that can potentially be reused will be either some combiner from the previous computational task, or None if there was no previous phase (in which case this method must return newc).

Optionally reuses an existing combiner for better performance. By default it doesn't - subclasses may override this behaviour. The provided combiner oldc that can potentially be reused will be either some combiner from the previous computational task, or None if there was no previous phase (in which case this method must return newc).

Value Params
newc

The new, empty combiner that can be used.

oldc

The combiner that is the result of the previous task, or None if there was no previous task.

Returns

Either newc or oldc.

def sameElements[U >: T](that: IterableOnce[U]): Boolean
def scan[U >: T](z: U)(op: (U, U) => U): CC[U]

Computes a prefix scan of the elements of the collection.

Computes a prefix scan of the elements of the collection.

Note: The neutral element z may be applied more than once.

Type Params
U

element type of the resulting collection

Value Params
op

the associative operator for the scan

z

neutral element for the operator op

Returns

a new parallel iterable containing the prefix scan of the elements in this parallel iterable

def scanLeft[S](z: S)(op: (S, T) => S): Iterable[S]
def scanRight[S](z: S)(op: (T, S) => S): Iterable[S]
def slice(unc_from: Int, unc_until: Int): Repr
def span(pred: T => Boolean): (Repr, Repr)

Splits this parallel iterable into a prefix/suffix pair according to a predicate.

Splits this parallel iterable into a prefix/suffix pair according to a predicate.

This method will use indexFlag signalling capabilities. This means that splitters may set and read the indexFlag state. The index flag is initially set to maximum integer value.

Value Params
pred

the predicate used to test the elements

Returns

a pair consisting of the longest prefix of the collection for which all the elements satisfy pred, and the rest of the collection

def splitAt(n: Int): (Repr, Repr)
def sum[U >: T](num: Numeric[U]): U
def tail: Repr
def take(n: Int): Repr
def takeWhile(pred: T => Boolean): Repr

Takes the longest prefix of elements that satisfy the predicate.

Takes the longest prefix of elements that satisfy the predicate.

This method will use indexFlag signalling capabilities. This means that splitters may set and read the indexFlag state. The index flag is initially set to maximum integer value.

Value Params
pred

the predicate used to test the elements

Returns

the longest prefix of this parallel iterable of elements that satisfy the predicate pred

def tasksupport: TaskSupport

The task support object which is responsible for scheduling and load-balancing tasks to processors.

The task support object which is responsible for scheduling and load-balancing tasks to processors.

See also
def tasksupport_=(ts: TaskSupport): Unit

Changes the task support object which is responsible for scheduling and load-balancing tasks to processors.

Changes the task support object which is responsible for scheduling and load-balancing tasks to processors.

A task support object can be changed in a parallel collection after it has been created, but only during a quiescent period, i.e. while there are no concurrent invocations to parallel collection methods.

Here is a way to change the task support of a parallel collection:

import scala.collection.parallel._
val pc = mutable.ParArray(1, 2, 3)
pc.tasksupport = new ForkJoinTaskSupport(
  new java.util.concurrent.ForkJoinPool(2))
See also
def to[C](factory: Factory[T, C]): C
def toArray[U >: T](`evidence$1`: ClassTag[U]): Array[U]
def toBuffer[U >: T]: Buffer[U]
def toIndexedSeq: IndexedSeq[T]
def toIterator: Iterator[T]
def toList: List[T]
def toMap[K, V](ev: T <:< (K, V)): ParMap[K, V]
protected def toParCollection[U >: T, That](cbf: () => Combiner[U, That]): That
protected def toParMap[K, V, That](cbf: () => Combiner[(K, V), That])(ev: T <:< (K, V)): That
def toSeq: ParSeq[T]
def toSet[U >: T]: ParSet[U]
override def toString: String
Definition Classes
Any
def toVector: Vector[T]
def withFilter(pred: T => Boolean): Repr
protected def wrap[R](body: => R): NonDivisible[R]
def zip[U >: T, S](that: ParIterable[S]): CC[(U, S)]
def zip[U >: T, S](that: Iterable[S]): CC[(U, S)]
def zipAll[S, U >: T](that: ParIterable[S], thisElem: U, thatElem: S): CC[(U, S)]
def zipWithIndex[U >: T]: CC[(U, Int)]

Zips this parallel iterable with its indices.

Zips this parallel iterable with its indices.

Type Params
U

the type of the first half of the returned pairs (this is always a supertype of the collection's element type T).

Returns

A new collection of type ParIterable containing pairs consisting of all elements of this parallel iterable paired with their index. Indices start at 0.

Deprecated methods

@deprecated("Use `to(LazyList)` instead.", "0.1.3")
def toStream: Stream[T]
Deprecated
@deprecated("Use `toIterable` instead", "0.1.3")
def toTraversable: ParIterable[T]
Deprecated

Inherited methods

def knownSize: Int
Inherited from
IterableOnce
def stepper[S <: Stepper[_]](shape: StepperShape[T, S]): S
Inherited from
IterableOnce

Implicits

Implicits

implicit protected def builder2ops[Elem, To](cb: Builder[Elem, To]): BuilderOps[Elem, To]
implicit protected def delegatedSignalling2ops[PI <: DelegatedSignalling](it: PI): SignallingOps[PI]
implicit protected def task2ops[R, Tp](tsk: StrictSplitterCheckTask[R, Tp]): TaskOps[R, Tp]