Packages

package exchange

Ordering
  1. Alphabetic
Visibility
  1. Public
  2. All

Type Members

  1. case class BroadcastExchangeExec(mode: BroadcastMode, child: SparkPlan) extends Exchange with Product with Serializable

    A BroadcastExchangeExec collects, transforms and finally broadcasts the result of a transformed SparkPlan.

  2. case class EnsureRequirements(conf: SQLConf) extends Rule[SparkPlan] with Product with Serializable

    Ensures that the Partitioning of input data meets the Distribution requirements for each operator by inserting ShuffleExchangeExec Operators where required.

    Ensures that the Partitioning of input data meets the Distribution requirements for each operator by inserting ShuffleExchangeExec Operators where required. Also ensure that the input partition ordering requirements are met.

  3. abstract class Exchange extends SparkPlan with UnaryExecNode

    Base class for operators that exchange data among multiple threads or processes.

    Base class for operators that exchange data among multiple threads or processes.

    Exchanges are the key class of operators that enable parallelism. Although the implementation differs significantly, the concept is similar to the exchange operator described in "Volcano -- An Extensible and Parallel Query Evaluation System" by Goetz Graefe.

  4. case class ReuseExchange(conf: SQLConf) extends Rule[SparkPlan] with Product with Serializable

    Find out duplicated exchanges in the spark plan, then use the same exchange for all the references.

  5. case class ReusedExchangeExec(output: Seq[Attribute], child: Exchange) extends SparkPlan with LeafExecNode with Product with Serializable

    A wrapper for reused exchange to have different output, because two exchanges which produce logically identical output will have distinct sets of output attribute ids, so we need to preserve the original ids because they're what downstream operators are expecting.

  6. case class ShuffleExchangeExec(outputPartitioning: Partitioning, child: SparkPlan, canChangeNumPartitions: Boolean = true) extends Exchange with Product with Serializable

    Performs a shuffle that will result in the desired partitioning.

Value Members

  1. object BroadcastExchangeExec extends Serializable
  2. object ShuffleExchangeExec extends Serializable

Ungrouped