partitions

Type Members

1. case class AppendEmptyPartition[T](rdd: RDD[T])(implicit evidence$2: ClassTag[T]) extends Product with Serializable

   

2. case class AppendEmptyPartitionRDD[T](rdd: RDD[T])(implicit evidence$1: ClassTag[T]) extends RDD[T] with Product with Serializable

   

3. class CollectPartitionsRDD[T] extends Serializable

   

   Wrap an RDD and expose a collectPartitions method that is similar to RDD.collect, but returns an Array of per-partition Arrays.

4. case class FilterPartitionIdxs[T](rdd: RDD[T]) extends Product with Serializable

   

5. case class OrderedRepartitionRDD[T](rdd: RDD[T])(implicit evidence$1: ClassTag[T]) extends Product with Serializable

   

   Some helpers for repartitioning an RDD while retaining the order of its elements.

6. case class PartitionByKeyRDD[K, V](rdd: RDD[((PartitionIndex, K), V)])(implicit evidence$1: Ordering[K], evidence$2: ClassTag[V]) extends Product with Serializable

   

   Add partitionByKey method to paired RDDs whose key is a tuple of (partition idx, elem idx), which sends elements to the partition indicated by partition idx, and sorts them within each partition according to elem idx.

7. class PartitionFirstElemsRDD[T] extends AnyRef

   

   Helpers for fetching the first element from each partition of an RDD.

8. class PartitionSizesRDD[T] extends AnyRef

   

   Helper for determining the size of each partition of an RDD.

9. case class PrependOrderedIDsRDD[T](rdd: RDD[T])(implicit evidence$1: ClassTag[T]) extends Product with Serializable

   

   Lazily key each element by its partition number and intra-partition idx.

   Lazily key each element by its partition number and intra-partition idx.

   Useful in tandem with PartitionByKeyRDD.

10. sealed abstract class RDDStats[T] extends AnyRef

    

11. case class RangePartition(index: PartitionIndex, parents: Seq[Partition]) extends Partition with Product with Serializable

    

12. case class RangePartitionRDD[T](parentRDD: RDD[T], partitionParentsMap: IndexedSeq[Option[Seq[PartitionIndex]]], bounds: Bounds[T])(implicit evidence$1: Ordering[T], evidence$2: ClassTag[T]) extends RDD[T] with SortedRDD[T] with Product with Serializable

    

13. case class ReducePartitionsRDD[T](rdd: RDD[T]) extends Product with Serializable

    

    Helper APIs for reducing RDD partitions to single elements, either with a combiner function (reducePartitions) or by directly mapping a partition Iterator to a single element (collapsePartitions).

14. case class SlicePartitionsRDD[T](prev: RDD[T], start: Int, end: Int)(implicit evidence$1: ClassTag[T]) extends PartitionPruningRDD[T] with Product with Serializable

    

15. trait SortedRDD[T] extends AnyRef

    

16. case class SortedRDDStats[T] extends RDDStats[T] with Serializable with Product

    

17. case class UnsortedRDDStats[T] extends RDDStats[T] with Serializable with Product