RDD wrapper supporting methods that compute partial-sums (from right to left) across the RDD.
Callers should be aware of one implementation detail: by default, scan-rights proceed by reversing the RDD,
performing a scan-left, then reversing the result, which involves 3 Spark jobs.
An alternative implementation delegates to scala.collection.Iterator.scanRight, which is likely less expensive,
but materializes whole partitions into memory, which is generally a severe anti-pattern in Spark computations.
RDD wrapper supporting methods that compute partial-sums (from right to left) across the RDD.
Callers should be aware of one implementation detail: by default, scan-rights proceed by reversing the RDD, performing a scan-left, then reversing the result, which involves 3 Spark jobs.
An alternative implementation delegates to scala.collection.Iterator.scanRight, which is likely less expensive, but materializes whole partitions into memory, which is generally a severe anti-pattern in Spark computations.