partitioner

Type Members

1. class CoLocationPartitioner extends Partitioner

   

   Will have the same parallism with last processor And each task in current processor will co-locate with task of last processor

2. class HashPartitioner extends Partitioner

   

   Only make sense when the message has implemented the hashCode() Otherwise, it will use Object.hashCode(), which will not return same hash code after serialization and deserialization.

3. trait Partitioner extends Serializable

   

4. case class PartitionerByClassName(partitionerClass: String) extends PartitionerFactory with Product with Serializable

   

5. case class PartitionerDescription(partitionerFactory: PartitionerFactory) extends Product with Serializable

   
   partitionerFactory

6. sealed trait PartitionerFactory extends AnyRef

   

7. case class PartitionerObject(partitioner: Partitioner) extends PartitionerFactory with Product with Serializable

   

8. class ShuffleGroupingPartitioner extends Partitioner

   

   The idea of ShuffleGroupingPartitioner is derived from Storm.

   The idea of ShuffleGroupingPartitioner is derived from Storm. Messages are randomly distributed across the downstream's tasks in a way such that each task is guaranteed to get an equal number of messages.

9. class ShufflePartitioner extends Partitioner

   

   Round Robin partition the data.