Immutable parallel hash set, based on hash tries.
Immutable parallel hash set, based on hash tries.
This is a base trait for Scala parallel collections. It defines behaviour
common to all parallel collections. Concrete parallel collections should
inherit this trait and ParIterable
if they want to define specific combiner
factories.
Parallel operations are implemented with divide and conquer style algorithms that parallelize well. The basic idea is to split the collection into smaller parts until they are small enough to be operated on sequentially.
All of the parallel operations are implemented as tasks within this trait. Tasks rely on the concept of splitters, which extend iterators. Every parallel collection defines:
def splitter: IterableSplitter[T]
which returns an instance of IterableSplitter[T]
, which is a subtype of Splitter[T]
.
Splitters have a method remaining
to check the remaining number of elements,
and method split
which is defined by splitters. Method split
divides the splitters
iterate over into disjunct subsets:
def split: Seq[Splitter]
which splits the splitter into a sequence of disjunct subsplitters. This is typically a very fast operation which simply creates wrappers around the receiver collection. This can be repeated recursively.
Tasks are scheduled for execution through a
scala.collection.parallel.TaskSupport object, which can be changed
through the tasksupport
setter of the collection.
Method newCombiner
produces a new combiner. Combiners are an extension of builders.
They provide a method combine
which combines two combiners and returns a combiner
containing elements of both combiners.
This method can be implemented by aggressively copying all the elements into the new combiner
or by lazily binding their results. It is recommended to avoid copying all of
the elements for performance reasons, although that cost might be negligible depending on
the use case. Standard parallel collection combiners avoid copying when merging results,
relying either on a two-step lazy construction or specific data-structure properties.
Methods:
def seq: Sequential
def par: Repr
produce the sequential or parallel implementation of the collection, respectively.
Method par
just returns a reference to this parallel collection.
Method seq
is efficient - it will not copy the elements. Instead,
it will create a sequential version of the collection using the same underlying data structure.
Note that this is not the case for sequential collections in general - they may copy the elements
and produce a different underlying data structure.
The combination of methods toMap
, toSeq
or toSet
along with par
and seq
is a flexible
way to change between different collection types.
Since this trait extends the GenIterable
trait, methods like size
must also
be implemented in concrete collections, while iterator
forwards to splitter
by
default.
Each parallel collection is bound to a specific fork/join pool, on which dormant worker
threads are kept. The fork/join pool contains other information such as the parallelism
level, that is, the number of processors used. When a collection is created, it is assigned the
default fork/join pool found in the scala.parallel
package object.
Parallel collections are not necessarily ordered in terms of the foreach
operation (see Traversable
). Parallel sequences have a well defined order for iterators - creating
an iterator and traversing the elements linearly will always yield the same order.
However, bulk operations such as foreach
, map
or filter
always occur in undefined orders for all
parallel collections.
Existing parallel collection implementations provide strict parallel iterators. Strict parallel iterators are aware
of the number of elements they have yet to traverse. It's also possible to provide non-strict parallel iterators,
which do not know the number of elements remaining. To do this, the new collection implementation must override
isStrictSplitterCollection
to false
. This will make some operations unavailable.
To create a new parallel collection, extend the ParIterable
trait, and implement size
, splitter
,
newCombiner
and seq
. Having an implicit combiner factory requires extending this trait in addition, as
well as providing a companion object, as with regular collections.
Method size
is implemented as a constant time operation for parallel collections, and parallel collection
operations rely on this assumption.
The higher-order functions passed to certain operations may contain side-effects. Since implementations of bulk operations may not be sequential, this means that side-effects may not be predictable and may produce data-races, deadlocks or invalidation of state if care is not taken. It is up to the programmer to either avoid using side-effects or to use some form of synchronization when accessing mutable data.
- Type Params
- T
the element type of the set
- See also
Scala's Parallel Collections Library overview section on Parallel Hash Tries for more information.
- Companion
- object
Type members
Classlikes
Inherited classlikes
Standard accessor task that iterates over the elements of the collection.
Standard accessor task that iterates over the elements of the collection.
- Type Params
- R
type of the result of this method (
R
for result).- Tp
the representation type of the task at hand.
- Inherited from
- ParIterableLike
- Inherited from
- ParIterableLike
Inherited types
Value members
Constructors
Concrete methods
- Definition Classes
- ParSet -> ParIterable -> ParSet -> ParIterable -> ParIterableLike -> GenericParTemplate -> GenericTraversableTemplate
Inherited methods
Computes the intersection between this set and another set.
Computes the intersection between this set and another set.
'''Note:''' Same as intersect
.
- Value Params
- that
the set to intersect with.
- Returns
a new set consisting of all elements that are both in this set and in the given set
that
.- Inherited from
- ParSetLike
The difference of this set and another set.
The difference of this set and another set.
'''Note:''' Same as diff
.
- Value Params
- that
the set of elements to exclude.
- Returns
a set containing those elements of this set that are not also contained in the given set
that
.- Inherited from
- ParSetLike
Aggregates the results of applying an operator to subsequent elements.
Aggregates the results of applying an operator to subsequent elements.
This is a more general form of fold
and reduce
. It has similar semantics, but does
not require the result to be a supertype of the element type. It traverses the elements in
different partitions sequentially, using seqop
to update the result, and then
applies combop
to results from different partitions. The implementation of this
operation may operate on an arbitrary number of collection partitions, so combop
may be invoked arbitrary number of times.
For example, one might want to process some elements and then produce a Set
. In this
case, seqop
would process an element and append it to the set, while combop
would concatenate two sets from different partitions together. The initial value
z
would be an empty set.
pc.aggregate(Set[Int]())(_ += process(_), _ ++ _)
Another example is calculating geometric mean from a collection of doubles (one would typically require big doubles for this).
- Type Params
- S
the type of accumulated results
- Value Params
- combop
an associative operator used to combine results from different partitions
- seqop
an operator used to accumulate results within a partition
- z
the initial value for the accumulated result of the partition - this will typically be the neutral element for the
seqop
operator (e.g.Nil
for list concatenation or0
for summation) and may be evaluated more than once
- Inherited from
- ParIterableLike
Drops all elements in the longest prefix of elements that satisfy the predicate, and returns a collection composed of the remaining elements.
Drops all elements in the longest prefix of elements that satisfy the predicate, and returns a collection composed of the remaining elements.
This method will use indexFlag
signalling capabilities. This means
that splitters may set and read the indexFlag
state.
The index flag is initially set to maximum integer value.
- Value Params
- pred
the predicate used to test the elements
- Returns
a collection composed of all the elements after the longest prefix of elements in this parallel iterable that satisfy the predicate
pred
- Inherited from
- ParIterableLike
Compares this set with another object for equality.
Compares this set with another object for equality.
'''Note:''' This operation contains an unchecked cast: if that
is a set, it will assume with an unchecked cast
that it has the same element type as this set.
Any subsequent ClassCastException is treated as a false
result.
- Value Params
- that
the other object
- Returns
true
ifthat
is a set which contains the same elements as this set.- Definition Classes
- Inherited from
- ParSetLike
Tests whether a predicate holds for some element of this parallel iterable.
Tests whether a predicate holds for some element of this parallel iterable.
This method will use abort
signalling capabilities. This means
that splitters may send and read abort
signals.
- Value Params
- p
a predicate used to test elements
- Returns
true if
p
holds for some element, false otherwise- Inherited from
- ParIterableLike
Finds some element in the collection for which the predicate holds, if such an element exists. The element may not necessarily be the first such element in the iteration order.
Finds some element in the collection for which the predicate holds, if such an element exists. The element may not necessarily be the first such element in the iteration order.
If there are multiple elements obeying the predicate, the choice is nondeterministic.
This method will use abort
signalling capabilities. This means
that splitters may send and read abort
signals.
- Value Params
- p
predicate used to test the elements
- Returns
an option value with the element if such an element exists, or
None
otherwise- Inherited from
- ParIterableLike
Converts this collection of traversable collections into a collection formed by the elements of these traversable collections.
Converts this collection of traversable collections into a collection formed by the elements of these traversable collections.
- Type Params
- B
the type of the elements of each traversable collection.
- Value Params
- asTraversable
an implicit conversion which asserts that the element type of this collection is a
GenTraversable
.
- Returns
a new collection resulting from concatenating all element collections.
- Inherited from
- GenericTraversableTemplate
Folds the elements of this sequence using the specified associative binary operator. The order in which the elements are reduced is unspecified and may be nondeterministic.
Folds the elements of this sequence using the specified associative binary operator. The order in which the elements are reduced is unspecified and may be nondeterministic.
Note this method has a different signature than the foldLeft
and foldRight
methods of the trait Traversable
.
The result of folding may only be a supertype of this parallel collection's
type parameter T
.
- Type Params
- U
a type parameter for the binary operator, a supertype of
T
.
- Value Params
- op
a binary operator that must be associative
- z
a neutral element for the fold operation, it may be added to the result an arbitrary number of times, not changing the result (e.g.
Nil
for list concatenation, 0 for addition, or 1 for multiplication)
- Returns
the result of applying fold operator
op
between all the elements andz
- Inherited from
- ParIterableLike
Tests whether a predicate holds for all elements of this parallel iterable.
Tests whether a predicate holds for all elements of this parallel iterable.
This method will use abort
signalling capabilities. This means
that splitters may send and read abort
signals.
- Value Params
- p
a predicate used to test elements
- Returns
true if
p
holds for all elements, false otherwise- Inherited from
- ParIterableLike
Applies a function f
to all the elements of parallel iterable in an undefined order.
Applies a function f
to all the elements of parallel iterable in an undefined order.
- Type Params
- U
the result type of the function applied to each element, which is always discarded
- Value Params
- f
function applied to each element
- Inherited from
- ParIterableLike
Computes the intersection between this set and another set.
Computes the intersection between this set and another set.
- Value Params
- that
the set to intersect with.
- Returns
a new set consisting of all elements that are both in this set and in the given set
that
.- Inherited from
- ParSetLike
Denotes whether this parallel collection has strict splitters.
Denotes whether this parallel collection has strict splitters.
This is true in general, and specific collection instances may choose to
override this method. Such collections will fail to execute methods
which rely on splitters being strict, i.e. returning a correct value
in the remaining
method.
This method helps ensure that such failures occur on method invocations, rather than later on and in unpredictable ways.
- Inherited from
- ParIterableLike
Creates a new split iterator used to traverse the elements of this collection.
Creates a new split iterator used to traverse the elements of this collection.
By default, this method is implemented in terms of the protected splitter
method.
- Returns
a split iterator
- Inherited from
- ParIterableLike
Reduces the elements of this sequence using the specified associative binary operator.
Reduces the elements of this sequence using the specified associative binary operator.
The order in which operations are performed on elements is unspecified and may be nondeterministic.
Note this method has a different signature than the reduceLeft
and reduceRight
methods of the trait Traversable
.
The result of reducing may only be a supertype of this parallel collection's
type parameter T
.
- Type Params
- U
A type parameter for the binary operator, a supertype of
T
.
- Value Params
- op
A binary operator that must be associative.
- Returns
The result of applying reduce operator
op
between all the elements if the collection is nonempty.- Throws
- UnsupportedOperationException
if this parallel iterable is empty.
- Inherited from
- ParIterableLike
Optionally reduces the elements of this sequence using the specified associative binary operator.
Optionally reduces the elements of this sequence using the specified associative binary operator.
The order in which operations are performed on elements is unspecified and may be nondeterministic.
Note this method has a different signature than the reduceLeftOption
and reduceRightOption
methods of the trait Traversable
.
The result of reducing may only be a supertype of this parallel collection's
type parameter T
.
- Type Params
- U
A type parameter for the binary operator, a supertype of
T
.
- Value Params
- op
A binary operator that must be associative.
- Returns
An option value containing result of applying reduce operator
op
between all the elements if the collection is nonempty, andNone
otherwise.- Inherited from
- ParIterableLike
Computes a prefix scan of the elements of the collection.
Computes a prefix scan of the elements of the collection.
Note: The neutral element z
may be applied more than once.
- Type Params
- U
element type of the resulting collection
- Value Params
- op
the associative operator for the scan
- z
neutral element for the operator
op
- Returns
a new parallel iterable containing the prefix scan of the elements in this parallel iterable
- Inherited from
- ParIterableLike
Splits this parallel iterable into a prefix/suffix pair according to a predicate.
Splits this parallel iterable into a prefix/suffix pair according to a predicate.
This method will use indexFlag
signalling capabilities. This means
that splitters may set and read the indexFlag
state.
The index flag is initially set to maximum integer value.
- Value Params
- pred
the predicate used to test the elements
- Returns
a pair consisting of the longest prefix of the collection for which all the elements satisfy
pred
, and the rest of the collection- Inherited from
- ParIterableLike
Tests whether this set is a subset of another set.
Tests whether this set is a subset of another set.
- Value Params
- that
the set to test.
- Returns
true
if this set is a subset ofthat
, i.e. if every element of this set is also an element ofthat
.- Inherited from
- ParSetLike
Takes the longest prefix of elements that satisfy the predicate.
Takes the longest prefix of elements that satisfy the predicate.
This method will use indexFlag
signalling capabilities. This means
that splitters may set and read the indexFlag
state.
The index flag is initially set to maximum integer value.
- Value Params
- pred
the predicate used to test the elements
- Returns
the longest prefix of this parallel iterable of elements that satisfy the predicate
pred
- Inherited from
- ParIterableLike
The task support object which is responsible for scheduling and load-balancing tasks to processors.
The task support object which is responsible for scheduling and load-balancing tasks to processors.
- See also
- Inherited from
- ParIterableLike
Changes the task support object which is responsible for scheduling and load-balancing tasks to processors.
Changes the task support object which is responsible for scheduling and load-balancing tasks to processors.
A task support object can be changed in a parallel collection after it has been created, but only during a quiescent period, i.e. while there are no concurrent invocations to parallel collection methods.
Here is a way to change the task support of a parallel collection:
import scala.collection.parallel._
val pc = mutable.ParArray(1, 2, 3)
pc.tasksupport = new ForkJoinTaskSupport(
new java.util.concurrent.ForkJoinPool(2))
- See also
- Inherited from
- ParIterableLike
Transposes this collection of traversable collections into a collection of collections.
Transposes this collection of traversable collections into a collection of collections.
The resulting collection's type will be guided by the static type of collection. For example:
val xs = List(
Set(1, 2, 3),
Set(4, 5, 6)).transpose
// xs == List(
// List(1, 4),
// List(2, 5),
// List(3, 6))
val ys = Vector(
List(1, 2, 3),
List(4, 5, 6)).transpose
// ys == Vector(
// Vector(1, 4),
// Vector(2, 5),
// Vector(3, 6))
- Type Params
- B
the type of the elements of each traversable collection.
- Value Params
- asTraversable
an implicit conversion which asserts that the element type of this collection is a
Traversable
.
- Returns
a two-dimensional collection of collections which has as ''n''th row the ''n''th column of this collection.
- Throws
- IllegalArgumentException
if all collections in this collection are not of the same size.
- Inherited from
- GenericTraversableTemplate
Converts this collection of pairs into two collections of the first and second half of each pair.
Converts this collection of pairs into two collections of the first and second half of each pair.
val xs = $Coll(
(1, "one"),
(2, "two"),
(3, "three")).unzip
// xs == ($Coll(1, 2, 3),
// $Coll(one, two, three))
- Type Params
- A1
the type of the first half of the element pairs
- A2
the type of the second half of the element pairs
- Value Params
- asPair
an implicit conversion which asserts that the element type of this collection is a pair.
- Returns
a pair of collections, containing the first, respectively second half of each element pair of this collection.
- Inherited from
- GenericTraversableTemplate
Converts this collection of triples into three collections of the first, second, and third element of each triple.
Converts this collection of triples into three collections of the first, second, and third element of each triple.
val xs = $Coll(
(1, "one", '1'),
(2, "two", '2'),
(3, "three", '3')).unzip3
// xs == ($Coll(1, 2, 3),
// $Coll(one, two, three),
// $Coll(1, 2, 3))
- Type Params
- A1
the type of the first member of the element triples
- A2
the type of the second member of the element triples
- A3
the type of the third member of the element triples
- Value Params
- asTriple
an implicit conversion which asserts that the element type of this collection is a triple.
- Returns
a triple of collections, containing the first, second, respectively third member of each element triple of this collection.
- Inherited from
- GenericTraversableTemplate
Zips this parallel iterable with its indices.
Zips this parallel iterable with its indices.
- Type Params
- U
the type of the first half of the returned pairs (this is always a supertype of the collection's element type
T
).
- Returns
A new collection of type
ParIterable
containing pairs consisting of all elements of this parallel iterable paired with their index. Indices start at 0.- Inherited from
- ParIterableLike
Computes the union between this set and another set.
Computes the union between this set and another set.
'''Note:''' Same as union
.
- Value Params
- that
the set to form the union with.
- Returns
a new set consisting of all elements that are in this set or in the given set
that
.- Inherited from
- ParSetLike
Deprecated and Inherited methods
Implicits
Inherited implicits
- Inherited from
- ParIterableLike