exchange

Type Members

1. case class BroadcastExchangeExec(mode: BroadcastMode, child: SparkPlan) extends Exchange with Product with Serializable

   

   A BroadcastExchangeExec collects, transforms and finally broadcasts the result of a transformed SparkPlan.

2. case class EnsureRequirements(conf: SQLConf) extends Rule[SparkPlan] with Product with Serializable

   

   Ensures that the Partitioning of input data meets the Distribution requirements for each operator by inserting ShuffleExchange Operators where required.

   Ensures that the Partitioning of input data meets the Distribution requirements for each operator by inserting ShuffleExchange Operators where required. Also ensure that the input partition ordering requirements are met.

3. abstract class Exchange extends SparkPlan with UnaryExecNode

   

   Base class for operators that exchange data among multiple threads or processes.

   Base class for operators that exchange data among multiple threads or processes.

   Exchanges are the key class of operators that enable parallelism. Although the implementation differs significantly, the concept is similar to the exchange operator described in "Volcano -- An Extensible and Parallel Query Evaluation System" by Goetz Graefe.

4. case class ReuseExchange(conf: SQLConf) extends Rule[SparkPlan] with Product with Serializable

   

   Find out duplicated exchanges in the spark plan, then use the same exchange for all the references.

5. case class ReusedExchangeExec(output: Seq[Attribute], child: Exchange) extends SparkPlan with LeafExecNode with Product with Serializable

   

   A wrapper for reused exchange to have different output, because two exchanges which produce logically identical output will have distinct sets of output attribute ids, so we need to preserve the original ids because they're what downstream operators are expecting.

6. case class ShuffleExchange(newPartitioning: Partitioning, child: SparkPlan, coordinator: Option[ExchangeCoordinator]) extends Exchange with Product with Serializable

   

   Performs a shuffle that will result in the desired newPartitioning.