scala.collection.parallel

Package object for parallel collections.

Attributes

Members list

Type members

Classlikes

Attributes

Supertypes
trait Tasks
class Object
trait Matchable
class Any
Show all
Known subtypes

This trait implements scheduling by employing an adaptive work stealing technique.

This trait implements scheduling by employing an adaptive work stealing technique.

Attributes

Supertypes
trait Tasks
class Object
trait Matchable
class Any
Known subtypes

Extension methods for .par on sequential collections.

Extension methods for .par on sequential collections.

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
implicit class CollectionsHaveToParArray[C, T](c: C)(implicit asGto: C => IterableOnce[T])

Adds toParArray method to collection classes.

Adds toParArray method to collection classes.

Attributes

Supertypes
class Object
trait Matchable
class Any
trait Combiner[-Elem, +To] extends Builder[Elem, To], Sizing, Parallel

The base trait for all combiners. A combiner incremental collection construction just like a regular builder, but also implements an efficient merge operation of two builders via combine method. Once the collection is constructed, it may be obtained by invoking the result method.

The base trait for all combiners. A combiner incremental collection construction just like a regular builder, but also implements an efficient merge operation of two builders via combine method. Once the collection is constructed, it may be obtained by invoking the result method.

The complexity of the combine method should be less than linear for best performance. The result method doesn't have to be a constant time operation, but may be performed in parallel.

Type parameters

Elem

the type of the elements added to the builder

To

the type of the collection the builder produces

Attributes

Supertypes
trait Parallel
trait Sizing
trait Builder[Elem, To]
trait Growable[Elem]
trait Clearable
class Object
trait Matchable
class Any
Show all
Known subtypes
trait LazyCombiner[Elem, To, Buff]
trait CombinerFactory[U, Repr]

Attributes

Supertypes
class Object
trait Matchable
class Any

A task support that uses an execution context to schedule tasks.

A task support that uses an execution context to schedule tasks.

It can be used with the default execution context implementation in the scala.concurrent package. It internally forwards the call to either a forkjoin based task support or a thread pool executor one, depending on what the execution context uses.

By default, parallel collections are parameterized with this task support object, so parallel collections share the same execution context backend as the rest of the scala.concurrent package.

Attributes

See also
Supertypes
trait TaskSupport
trait Tasks
class Object
trait Matchable
class Any
Show all

This tasks implementation uses execution contexts to spawn a parallel computation.

This tasks implementation uses execution contexts to spawn a parallel computation.

As an optimization, it internally checks whether the execution context is the standard implementation based on fork/join pools, and if it is, creates a ForkJoinTaskSupport that shares the same pool to forward its request to it.

Otherwise, it uses an execution context exclusive Tasks implementation to divide the tasks into smaller chunks and execute operations on it.

Attributes

Supertypes
trait Tasks
class Object
trait Matchable
class Any
Known subtypes
class ForkJoinTaskSupport(val environment: ForkJoinPool) extends TaskSupport, AdaptiveWorkStealingForkJoinTasks

A task support that uses a fork join pool to schedule tasks.

A task support that uses a fork join pool to schedule tasks.

Attributes

See also
Supertypes
trait TaskSupport
trait Tasks
class Object
trait Matchable
class Any
Show all

An implementation trait for parallel tasks based on the fork/join framework.

An implementation trait for parallel tasks based on the fork/join framework.

Attributes

Companion
object
Supertypes
trait Tasks
class Object
trait Matchable
class Any
Known subtypes
object ForkJoinTasks

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type

A trait describing objects that provide a fork/join pool.

A trait describing objects that provide a fork/join pool.

Attributes

Supertypes
class Object
trait Matchable
class Any
Known subtypes

Parallel iterators allow splitting and provide a remaining method to obtain the number of elements remaining in the iterator.

Parallel iterators allow splitting and provide a remaining method to obtain the number of elements remaining in the iterator.

Type parameters

T

type of the elements iterated.

Attributes

Supertypes
trait Signalling
trait Splitter[T]
trait Iterator[T]
trait IterableOnce[T]
class Object
trait Matchable
class Any
Show all
Known subtypes
class Appended[U, PI]
class Mapped[S]
class Taken
class Zipped[S]
class ZippedAll[U, S]
class EntryIterator[T, IterRepr]
trait SeqSplitter[T]
class Elements
class Patched[U]
Show all
Self type

A template trait for parallel iterable collections.

A template trait for parallel iterable collections.

$paralleliterableinfo

$sideeffects

Type parameters

T

the element type of the collection

Attributes

Companion
object
Supertypes
trait Parallel
trait IterableOnce[T]
trait HasNewBuilder[T, ParIterable[T]]
class Object
trait Matchable
class Any
Show all
Known subtypes
trait ParIterable[T]
trait ParMap[K, V]
class ParHashMap[K, V]
class WithDefault[K, V]
trait ParSeq[T]
class ParRange
class ParVector[T]
trait ParSet[T]
class ParHashSet[T]
trait ParIterable[T]
trait ParMap[K, V]
class ParHashMap[K, V]
class WithDefault[K, V]
class ParTrieMap[K, V]
trait ParSeq[T]
class ParArray[T]
trait ParSet[T]
class ParHashSet[T]
trait ParMap[K, V]
class WithDefault[A, B]
trait ParSeq[T]
trait ParSet[T]
Show all

This object provides a set of operations needed to create ParIterable values.

This object provides a set of operations needed to create ParIterable values.

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
trait ParIterableLike[+T, +CC <: (ParIterable), +Repr <: ParIterable[T], +Sequential <: Iterable[T] & IterableOps[T, Iterable, Sequential]] extends IterableOnce[T], CustomParallelizable[T, Repr], Parallel, HasNewCombiner[T, Repr]

A template trait for parallel collections of type ParIterable[T].

A template trait for parallel collections of type ParIterable[T].

This is a base trait for Scala parallel collections. It defines behaviour common to all parallel collections. Concrete parallel collections should inherit this trait and ParIterable if they want to define specific combiner factories.

Parallel operations are implemented with divide and conquer style algorithms that parallelize well. The basic idea is to split the collection into smaller parts until they are small enough to be operated on sequentially.

All of the parallel operations are implemented as tasks within this trait. Tasks rely on the concept of splitters, which extend iterators. Every parallel collection defines:

   def splitter: IterableSplitter[T]

which returns an instance of IterableSplitter[T], which is a subtype of Splitter[T]. Splitters have a method remaining to check the remaining number of elements, and method split which is defined by splitters. Method split divides the splitters iterate over into disjunct subsets:

   def split: Seq[Splitter]

which splits the splitter into a sequence of disjunct subsplitters. This is typically a very fast operation which simply creates wrappers around the receiver collection. This can be repeated recursively.

Tasks are scheduled for execution through a scala.collection.parallel.TaskSupport object, which can be changed through the tasksupport setter of the collection.

Method newCombiner produces a new combiner. Combiners are an extension of builders. They provide a method combine which combines two combiners and returns a combiner containing elements of both combiners. This method can be implemented by aggressively copying all the elements into the new combiner or by lazily binding their results. It is recommended to avoid copying all of the elements for performance reasons, although that cost might be negligible depending on the use case. Standard parallel collection combiners avoid copying when merging results, relying either on a two-step lazy construction or specific data-structure properties.

Methods:

   def seq: Sequential
   def par: Repr

produce the sequential or parallel implementation of the collection, respectively. Method par just returns a reference to this parallel collection. Method seq is efficient - it will not copy the elements. Instead, it will create a sequential version of the collection using the same underlying data structure. Note that this is not the case for sequential collections in general - they may copy the elements and produce a different underlying data structure.

The combination of methods toMap, toSeq or toSet along with par and seq is a flexible way to change between different collection types.

Since this trait extends the GenIterable trait, methods like size must also be implemented in concrete collections, while iterator forwards to splitter by default.

Each parallel collection is bound to a specific fork/join pool, on which dormant worker threads are kept. The fork/join pool contains other information such as the parallelism level, that is, the number of processors used. When a collection is created, it is assigned the default fork/join pool found in the scala.parallel package object.

Parallel collections are not necessarily ordered in terms of the foreach operation (see Traversable). Parallel sequences have a well defined order for iterators - creating an iterator and traversing the elements linearly will always yield the same order. However, bulk operations such as foreach, map or filter always occur in undefined orders for all parallel collections.

Existing parallel collection implementations provide strict parallel iterators. Strict parallel iterators are aware of the number of elements they have yet to traverse. It's also possible to provide non-strict parallel iterators, which do not know the number of elements remaining. To do this, the new collection implementation must override isStrictSplitterCollection to false. This will make some operations unavailable.

To create a new parallel collection, extend the ParIterable trait, and implement size, splitter, newCombiner and seq. Having an implicit combiner factory requires extending this trait in addition, as well as providing a companion object, as with regular collections.

Method size is implemented as a constant time operation for parallel collections, and parallel collection operations rely on this assumption.

The higher-order functions passed to certain operations may contain side-effects. Since implementations of bulk operations may not be sequential, this means that side-effects may not be predictable and may produce data-races, deadlocks or invalidation of state if care is not taken. It is up to the programmer to either avoid using side-effects or to use some form of synchronization when accessing mutable data.

Type parameters

Repr

the type of the actual collection containing the elements

T

the element type of the collection

Attributes

Supertypes
trait HasNewCombiner[T, Repr]
trait Parallel
trait CustomParallelizable[T, Repr]
trait Parallelizable[T, Repr]
trait IterableOnce[T]
class Object
trait Matchable
class Any
Show all
Known subtypes
trait ParIterable[T]
trait ParMap[K, V]
class ParHashMap[K, V]
class WithDefault[K, V]
trait ParSeq[T]
class ParRange
class ParVector[T]
trait ParSet[T]
class ParHashSet[T]
trait ParMapLike[K, V, CC, Repr, Sequential]
trait ParIterable[T]
trait ParMap[K, V]
class ParHashMap[K, V]
class WithDefault[K, V]
class ParTrieMap[K, V]
trait ParSeq[T]
class ParArray[T]
trait ParSet[T]
class ParHashSet[T]
trait ParMapLike[K, V, CC, Repr, Sequential]
trait ParSetLike[T, CC, Repr, Sequential]
trait ParIterable[T]
trait ParMap[K, V]
class WithDefault[A, B]
trait ParSeq[T]
trait ParSet[T]
trait ParMapLike[K, V, CC, Repr, Sequential]
trait ParSeqLike[T, CC, Repr, Sequential]
trait ParSetLike[T, CC, Repr, Sequential]
Show all
Self type
ParIterableLike[T, CC, Repr, Sequential]
trait ParMap[K, +V] extends GenericParMapTemplate[K, V, ParMap], ParIterable[(K, V)], ParMapLike[K, V, ParMap, ParMap[K, V], Map[K, V]]

A template trait for parallel maps.

A template trait for parallel maps.

The higher-order functions passed to certain operations may contain side-effects. Since implementations of bulk operations may not be sequential, this means that side-effects may not be predictable and may produce data-races, deadlocks or invalidation of state if care is not taken. It is up to the programmer to either avoid using side-effects or to use some form of synchronization when accessing mutable data.

Type parameters

K

the key type of the map

V

the value type of the map

Attributes

Companion
object
Supertypes
trait ParMapLike[K, V, ParMap, ParMap[K, V], Map[K, V]]
trait Equals
trait ParIterable[(K, V)]
trait ParIterableLike[(K, V), ParIterable, ParMap[K, V], Map[K, V]]
trait Parallel
trait CustomParallelizable[(K, V), ParMap[K, V]]
trait Parallelizable[(K, V), ParMap[K, V]]
trait IterableOnce[(K, V)]
trait HasNewCombiner[(K, V), ParMap[K, V]]
trait HasNewBuilder[(K, V), ParIterable[(K, V)]]
class Object
trait Matchable
class Any
Show all
Known subtypes
trait ParMap[K, V]
class ParHashMap[K, V]
class WithDefault[K, V]
trait ParMap[K, V]
class ParHashMap[K, V]
class WithDefault[K, V]
class ParTrieMap[K, V]
class WithDefault[A, B]
Show all
Self type
ParMap[K, V]
object ParMap extends ParMapFactory[ParMap, Map]

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
ParMap.type
trait ParMapLike[K, +V, +CC <: (ParMap), +Repr <: ParMapLike[K, V, ParMap, Repr, Sequential] & ParMap[K, V], +Sequential <: Map[K, V] & MapOps[K, V, Map, Sequential]] extends ParIterableLike[(K, V), ParIterable, Repr, Sequential], Equals

A template trait for mutable parallel maps. This trait is to be mixed in with concrete parallel maps to override the representation type.

A template trait for mutable parallel maps. This trait is to be mixed in with concrete parallel maps to override the representation type.

The higher-order functions passed to certain operations may contain side-effects. Since implementations of bulk operations may not be sequential, this means that side-effects may not be predictable and may produce data-races, deadlocks or invalidation of state if care is not taken. It is up to the programmer to either avoid using side-effects or to use some form of synchronization when accessing mutable data.

Type parameters

K

the key type of the map

V

the value type of the map

Attributes

Supertypes
trait Equals
trait ParIterableLike[(K, V), ParIterable, Repr, Sequential]
trait HasNewCombiner[(K, V), Repr]
trait Parallel
trait CustomParallelizable[(K, V), Repr]
trait Parallelizable[(K, V), Repr]
trait IterableOnce[(K, V)]
class Object
trait Matchable
class Any
Show all
Known subtypes
trait ParMapLike[K, V, CC, Repr, Sequential]
class ParHashMap[K, V]
trait ParMap[K, V]
class WithDefault[K, V]
trait ParMapLike[K, V, CC, Repr, Sequential]
class ParHashMap[K, V]
trait ParMap[K, V]
class WithDefault[K, V]
class ParTrieMap[K, V]
trait ParMap[K, V]
class WithDefault[A, B]
Show all
Self type
ParMapLike[K, V, CC, Repr, Sequential]
trait ParSeq[+T] extends ParIterable[T], GenericParTemplate[T, ParSeq], ParSeqLike[T, ParSeq, ParSeq[T], Seq[T]]

A template trait for parallel sequences.

A template trait for parallel sequences.

$parallelseqinfo

The higher-order functions passed to certain operations may contain side-effects. Since implementations of bulk operations may not be sequential, this means that side-effects may not be predictable and may produce data-races, deadlocks or invalidation of state if care is not taken. It is up to the programmer to either avoid using side-effects or to use some form of synchronization when accessing mutable data.

Type parameters

T

the type of the elements in this parallel sequence

Attributes

Companion
object
Supertypes
trait ParSeqLike[T, ParSeq, ParSeq[T], Seq[T]]
trait Equals
trait ParIterable[T]
trait ParIterableLike[T, ParSeq, ParSeq[T], Seq[T]]
trait Parallel
trait Parallelizable[T, ParSeq[T]]
trait IterableOnce[T]
trait HasNewCombiner[T, ParSeq[T]]
trait HasNewBuilder[T, ParSeq[T]]
class Object
trait Matchable
class Any
Show all
Known subtypes
trait ParSeq[T]
class ParRange
class ParVector[T]
trait ParSeq[T]
class ParArray[T]
object ParSeq extends ParFactory[ParSeq]

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
ParSeq.type
trait ParSeqLike[+T, +CC <: (ParSeq), +Repr <: ParSeq[T], +Sequential <: Seq[T] & SeqOps[T, AnyConstr, Sequential]] extends ParIterableLike[T, CC, Repr, Sequential], Equals

A template trait for sequences of type ParSeq[T], representing parallel sequences with element type T.

A template trait for sequences of type ParSeq[T], representing parallel sequences with element type T.

Parallel sequences inherit the Seq trait. Their indexing and length computations are defined to be efficient. Like their sequential counterparts they always have a defined order of elements. This means they will produce resulting parallel sequences in the same way sequential sequences do. However, the order in which they perform bulk operations on elements to produce results is not defined and is generally nondeterministic. If the higher-order functions given to them produce no sideeffects, then this won't be noticeable.

Type parameters

Repr

the type of the actual collection containing the elements

Sequential

the type of the sequential version of this parallel collection

T

the type of the elements contained in this collection

Attributes

Supertypes
trait Equals
trait ParIterableLike[T, CC, Repr, Sequential]
trait HasNewCombiner[T, Repr]
trait Parallel
trait CustomParallelizable[T, Repr]
trait Parallelizable[T, Repr]
trait IterableOnce[T]
class Object
trait Matchable
class Any
Show all
Known subtypes
trait ParSeq[T]
class ParRange
class ParVector[T]
class ParArray[T]
trait ParSeq[T]
trait ParSeq[T]
Show all
Self type
ParSeqLike[T, CC, Repr, Sequential]
trait ParSet[T] extends GenericParTemplate[T, ParSet], ParIterable[T], ParSetLike[T, ParSet, ParSet[T], Set[T]]

A template trait for parallel sets.

A template trait for parallel sets.

The higher-order functions passed to certain operations may contain side-effects. Since implementations of bulk operations may not be sequential, this means that side-effects may not be predictable and may produce data-races, deadlocks or invalidation of state if care is not taken. It is up to the programmer to either avoid using side-effects or to use some form of synchronization when accessing mutable data.

Type parameters

T

the element type of the set

Attributes

Companion
object
Supertypes
trait ParSetLike[T, ParSet, ParSet[T], Set[T]]
trait Equals
trait T => Boolean
trait ParIterable[T]
trait ParIterableLike[T, ParSet, ParSet[T], Set[T]]
trait Parallel
trait Parallelizable[T, ParSet[T]]
trait IterableOnce[T]
trait HasNewCombiner[T, ParSet[T]]
trait HasNewBuilder[T, ParSet[T]]
class Object
trait Matchable
class Any
Show all
Known subtypes
trait ParSet[T]
class ParHashSet[T]
trait ParSet[T]
class ParHashSet[T]
Self type
ParSet[T]
object ParSet extends ParSetFactory[ParSet]

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
ParSet.type
trait ParSetLike[T, +CC <: (ParIterable), +Repr <: ParSet[T], +Sequential <: Set[T] & SetOps[T, Set, Sequential]] extends ParIterableLike[T, CC, Repr, Sequential], T => Boolean, Equals

A template trait for parallel sets. This trait is mixed in with concrete parallel sets to override the representation type.

A template trait for parallel sets. This trait is mixed in with concrete parallel sets to override the representation type.

The higher-order functions passed to certain operations may contain side-effects. Since implementations of bulk operations may not be sequential, this means that side-effects may not be predictable and may produce data-races, deadlocks or invalidation of state if care is not taken. It is up to the programmer to either avoid using side-effects or to use some form of synchronization when accessing mutable data.

Type parameters

T

the element type of the set

Attributes

Supertypes
trait Equals
trait T => Boolean
trait ParIterableLike[T, CC, Repr, Sequential]
trait HasNewCombiner[T, Repr]
trait Parallel
trait CustomParallelizable[T, Repr]
trait Parallelizable[T, Repr]
trait IterableOnce[T]
class Object
trait Matchable
class Any
Show all
Known subtypes
class ParHashSet[T]
trait ParSet[T]
trait ParSetLike[T, CC, Repr, Sequential]
class ParHashSet[T]
trait ParSet[T]
trait ParSet[T]
Show all
Self type
ParSetLike[T, CC, Repr, Sequential]
trait PreciseSplitter[+T] extends Splitter[T]

A precise splitter (or a precise split iterator) can be split into arbitrary number of splitters that traverse disjoint subsets of arbitrary sizes.

A precise splitter (or a precise split iterator) can be split into arbitrary number of splitters that traverse disjoint subsets of arbitrary sizes.

Implementors might want to override the parameterless split method for efficiency.

Type parameters

T

type of the elements this splitter traverses

Attributes

Supertypes
trait Splitter[T]
trait Iterator[T]
trait IterableOnce[T]
class Object
trait Matchable
class Any
Show all
Known subtypes
trait SeqSplitter[+T] extends IterableSplitter[T], PreciseSplitter[T]

Parallel sequence iterators allow splitting into arbitrary subsets.

Parallel sequence iterators allow splitting into arbitrary subsets.

Type parameters

T

type of the elements iterated.

Attributes

Supertypes
trait PreciseSplitter[T]
trait IterableSplitter[T]
trait Signalling
trait Splitter[T]
trait Iterator[T]
trait IterableOnce[T]
class Object
trait Matchable
class Any
Show all
Known subtypes
Self type
trait Splitter[+T] extends Iterator[T]

A splitter (or a split iterator) can be split into more splitters that traverse over disjoint subsets of elements.

A splitter (or a split iterator) can be split into more splitters that traverse over disjoint subsets of elements.

Type parameters

T

type of the elements this splitter traverses

Attributes

Companion
object
Supertypes
trait Iterator[T]
trait IterableOnce[T]
class Object
trait Matchable
class Any
Show all
Known subtypes
trait IterableSplitter[T]
class Appended[U, PI]
class Mapped[S]
class Taken
class Zipped[S]
class ZippedAll[U, S]
class EntryIterator[T, IterRepr]
trait SeqSplitter[T]
class Elements
class Patched[U]
trait PreciseSplitter[T]
Show all
object Splitter

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
Splitter.type
trait Task[R, +Tp]

Attributes

Supertypes
class Object
trait Matchable
class Any
Known subtypes
trait Accessor[R, Tp]
trait Transformer[R, Tp]
class Copy[U, That]
trait ParSeqLikeTransformer[R, Tp]
trait ParSeqLikeAccessor[R, Tp]
class ParArrayMap[S]
class ScanToArray[U]
class Size
Show all
trait TaskSupport extends Tasks

A trait implementing the scheduling of a parallel collection operation.

A trait implementing the scheduling of a parallel collection operation.

Parallel collections are modular in the way operations are scheduled. Each parallel collection is parameterized with a task support object which is responsible for scheduling and load-balancing tasks to processors.

A task support object can be changed in a parallel collection after it has been created, but only during a quiescent period, i.e. while there are no concurrent invocations to parallel collection methods.

There are currently a few task support implementations available for parallel collections. The scala.collection.parallel.ForkJoinTaskSupport uses a fork-join pool internally.

The scala.collection.parallel.ExecutionContextTaskSupport uses the default execution context implementation found in scala.concurrent, and it reuses the thread pool used in scala.concurrent.

The execution context task support is set to each parallel collection by default, so parallel collections reuse the same fork-join pool as the future API.

Here is a way to change the task support of a parallel collection:

import scala.collection.parallel._
val pc = mutable.ParArray(1, 2, 3)
pc.tasksupport = new ForkJoinTaskSupport(
  new java.util.concurrent.ForkJoinPool(2))

Attributes

See also

Configuring Parallel Collections section on the parallel collection's guide for more information.

Supertypes
trait Tasks
class Object
trait Matchable
class Any
Known subtypes
trait Tasks

A trait that declares task execution capabilities used by parallel collections.

A trait that declares task execution capabilities used by parallel collections.

Attributes

Supertypes
class Object
trait Matchable
class Any
Known subtypes
trait TraversableOps[T]

Attributes

Supertypes
class Object
trait Matchable
class Any

Value members

Concrete methods

def setTaskSupport[Coll](c: Coll, t: TaskSupport): Coll
def thresholdFromSize(sz: Int, parallelismLevel: Int): Int

Computes threshold from the size of the collection and the parallelism level.

Computes threshold from the size of the collection and the parallelism level.

Attributes

Implicits

Implicits

final implicit def CollectionsHaveToParArray[C, T](c: C)(implicit asGto: C => IterableOnce[T]): CollectionsHaveToParArray[C, T]

Adds toParArray method to collection classes.

Adds toParArray method to collection classes.

Attributes