Accumulate elements with the given Accumulators.
Accumulate elements with the given Accumulators.
For example:
val max = sc.maxAccumulator[Int]("max") val min = sc.maxAccumulator[Int]("min") sc.parallelize(1 to 100).accumulate(max, min)
Accumulate elements with the given Accumulators by
applying f
.
Accumulate elements with the given Accumulators by
applying f
.
For example:
val max = sc.maxAccumulator[Int]("max") val min = sc.maxAccumulator[Int]("min") sc.textFile("input.txt").accumulateBy(max, min)(_.length)
Count elements with an automatically created Accumulator.
Count elements with an Accumulator.
Count positive and negative results in a SCollection.filter with automatically created Accumulators.
Count positive and negative results in a SCollection.filter with the given Accumulators.
Count positive and negative results in a SCollection.filter with the given Accumulators.
For example:
val pos = sc.sumAccumulator[Long]("positive") val neg = sc.sumAccumulator[Long]("negative") sc.parallelize(1 to 100).accumulateCountFilter(pos, neg)(_ > 95)
Enhanced version of SCollection with Accumulator helpers.