partitioner

Type Members

1. class BroadcastPartitioner extends MulticastPartitioner

   

2. class CoLocationPartitioner extends UnicastPartitioner

   

   Will have the same parallelism with last processor And each task in current processor will co-locate with task of last processor

3. class HashPartitioner extends UnicastPartitioner

   

   Only make sense when the message has implemented the hashCode() Otherwise, it will use Object.hashCode(), which will not return same hash code after serialization and deserialization.

4. trait MulticastPartitioner extends Partitioner

   

5. sealed trait Partitioner extends Serializable

   

6. class PartitionerByClassName extends PartitionerFactory with Serializable

   

7. case class PartitionerDescription(partitionerFactory: PartitionerFactory) extends Product with Serializable

   
   partitionerFactory

8. sealed trait PartitionerFactory extends AnyRef

   

9. class PartitionerObject extends PartitionerFactory with Serializable

   

10. class ShuffleGroupingPartitioner extends UnicastPartitioner

    

    The idea of ShuffleGroupingPartitioner is derived from Storm.

    The idea of ShuffleGroupingPartitioner is derived from Storm. Messages are randomly distributed across the downstream's tasks in a way such that each task is guaranteed to get an equal number of messages.

11. class ShufflePartitioner extends UnicastPartitioner

    

    Round Robin partition the data.

12. trait UnicastPartitioner extends Partitioner