Returns true if this node and its children have already been gone through analysis and verification.
Returns true if this node and its children have already been gone through analysis and verification. Note that this is only an optimization used to avoid analyzing trees that have already been analyzed, and can be reset by transformations.
Returns the tree node at the specified number.
Returns the tree node at the specified number. Numbers for each node can be found in the numberedTreeString.
Returns a string representing the arguments to this node, minus any children
Returns a string representing the arguments to this node, minus any children
Returns a 'scala code' representation of this TreeNode
and its children.
Returns a 'scala code' representation of this TreeNode
and its children. Intended for use
when debugging where the prettier toString function is obfuscating the actual structure. In the
case of 'pure' TreeNodes
that only contain primitives and other TreeNodes, the result can be
pasted in the REPL to build an equivalent Tree.
Returns a Seq of the children of this node.
Returns true if all its children of this query plan have been resolved.
Returns true if all its children of this query plan have been resolved.
Args that have cleaned such that differences in expression id should not affect equality
Args that have cleaned such that differences in expression id should not affect equality
Returns a Seq containing the result of applying a partial function to all elements in this tree on which the function is defined.
Returns a Seq containing the result of applying a partial function to all elements in this tree on which the function is defined.
Finds and returns the first TreeNode of the tree for which the given partial function is defined (pre-order), and applies the partial function to it.
Returns all of the expressions present in this query plan operator.
Returns all of the expressions present in this query plan operator.
Faster version of equality which short-circuits when two treeNodes are the same instance.
Faster version of equality which short-circuits when two treeNodes are the same instance.
We don't just override Object.equals, as doing so prevents the scala compiler from
generating case class equals
methods
Find the first TreeNode that satisfies the condition specified by f
.
Returns a Seq by applying a function to all nodes in this tree and using the elements of the resulting collections.
Returns a Seq by applying a function to all nodes in this tree and using the elements of the resulting collections.
Runs the given function on this node and then recursively on children.
Runs the given function recursively on children then on this node.
Appends the string represent of this node and its children to the given StringBuilder.
Appends the string represent of this node and its children to the given StringBuilder.
The set of all attributes that are input to this operator by its children.
The set of all attributes that are input to this operator by its children.
Creates a copy of this type of tree node after a transformation.
Creates a copy of this type of tree node after a transformation. Must be overridden by child classes that have constructor arguments that are not present in the productIterator.
the new product arguments.
Returns a Seq containing the result of applying the given function to each node in this tree in a preorder traversal.
Returns a Seq containing the result of applying the given function to each node in this tree in a preorder traversal.
the function to be applied.
Returns a copy of this node where f
has been applied to all the nodes children.
Returns a copy of this node where f
has been applied to all the nodes children.
Attributes that are referenced by expressions but not provided by this nodes children.
Attributes that are referenced by expressions but not provided by this nodes children. Subclasses should override this method if they produce attributes internally as it is used by assertions designed to prevent the construction of invalid plans.
Note that virtual columns should be excluded. Currently, we only support the grouping ID virtual column.
Returns the name of this type of TreeNode.
Returns the name of this type of TreeNode. Defaults to the class name.
Returns a string representation of the nodes in this tree, where each operator is numbered.
Args to the constructor that should be copied, but not transformed.
Args to the constructor that should be copied, but not transformed. These are appended to the transformed args automatically by makeCopy
Returns the set of attributes that are output by this node.
Returns the set of attributes that are output by this node.
Prints out the schema in the tree format
Prints out the schema in the tree format
All Attributes that appear in expressions from this operator.
All Attributes that appear in expressions from this operator. Note that this set does not include attributes that are implicitly referenced by being passed through to the output tuple.
Performs attribute resolution given a name and a sequence of possible attributes.
Performs attribute resolution given a name and a sequence of possible attributes.
Optionally resolves the given strings to a NamedExpression based on the output of this LogicalPlan.
Optionally resolves the given strings to a NamedExpression based on the output of this
LogicalPlan. The attribute is expressed as string in the following form:
[scope].AttributeName.[nested].[fields]...
.
Optionally resolves the given strings to a NamedExpression using the input from all child nodes of this LogicalPlan.
Optionally resolves the given strings to a NamedExpression using the input from all child
nodes of this LogicalPlan. The attribute is expressed as
as string in the following form: [scope].AttributeName.[nested].[fields]...
.
Recursively transforms the expressions of a tree, skipping nodes that have already been analyzed.
Recursively transforms the expressions of a tree, skipping nodes that have already been analyzed.
Returns a copy of this node where rule
has been recursively applied first to all of its
children and then itself (post-order).
Returns a copy of this node where rule
has been recursively applied first to all of its
children and then itself (post-order). When rule
does not apply to a given node, it is left
unchanged. This function is similar to transformUp
, but skips sub-trees that have already
been marked as analyzed.
the function use to transform this nodes children
Given an attribute name, split it to name parts by dot, but
don't split the name parts quoted by backticks, for example,
ab.cd
.efg
should be split into two parts "ab.cd" and "efg".
Given an attribute name, split it to name parts by dot, but
don't split the name parts quoted by backticks, for example,
ab.cd
.efg
should be split into two parts "ab.cd" and "efg".
Returns true if this expression and all its children have been resolved to a specific schema and false if it still contains any unresolved placeholders.
Returns true if this expression and all its children have been resolved to a specific schema
and false if it still contains any unresolved placeholders. Implementations of LogicalPlan
can override this (e.g.
UnresolvedRelation
should return false
).
Returns true when the given logical plan will return the same results as this logical plan.
Returns true when the given logical plan will return the same results as this logical plan.
Since its likely undecidable to generally determine if two given plans will produce the same results, it is okay for this function to return false, even if the results are actually the same. Such behavior will not affect correctness, only the application of performance enhancements like caching. However, it is not acceptable to return true if the results could possibly be different.
By default this function performs a modified version of equality that is tolerant of cosmetic differences like attribute naming and or expression id differences. Logical operators that can do better should override this function.
Returns the output schema in the tree format.
Returns the output schema in the tree format.
String representation of this node without any children
A prefix string used when printing the plan.
A prefix string used when printing the plan.
We use "!" to indicate an invalid plan, and "'" to indicate an unresolved plan.
Computes Statistics for this plan.
Computes Statistics for this plan. The default implementation assumes the output cardinality is the product of of all child plan's cardinality, i.e. applies in the case of cartesian joins.
LeafNodes must override this.
The arguments that should be included in the arg string.
The arguments that should be included in the arg string. Defaults to the productIterator
.
Returns a copy of this node where rule
has been recursively applied to the tree.
Returns a copy of this node where rule
has been recursively applied to the tree.
When rule
does not apply to a given node it is left unchanged.
Users should not expect a specific directionality. If a specific directionality is needed,
transformDown or transformUp should be used.
the function use to transform this nodes children
Returns the result of running transformExpressions on this node and all its children.
Returns the result of running transformExpressions on this node and all its children.
Returns a copy of this node where rule
has been recursively applied to all the children of
this node.
Returns a copy of this node where rule
has been recursively applied to all the children of
this node. When rule
does not apply to a given node it is left unchanged.
the function used to transform this nodes children
Returns a copy of this node where rule
has been recursively applied to it and all of its
children (pre-order).
Returns a copy of this node where rule
has been recursively applied to it and all of its
children (pre-order). When rule
does not apply to a given node it is left unchanged.
the function used to transform this nodes children
Runs transform with rule
on all expressions present in this query operator.
Runs transform with rule
on all expressions present in this query operator.
Users should not expect a specific directionality. If a specific directionality is needed,
transformExpressionsDown or transformExpressionsUp should be used.
the rule to be applied to every expression in this operator.
Runs transformDown with rule
on all expressions present in this query operator.
Runs transformDown with rule
on all expressions present in this query operator.
the rule to be applied to every expression in this operator.
Runs transformUp with rule
on all expressions present in this query operator.
Runs transformUp with rule
on all expressions present in this query operator.
the rule to be applied to every expression in this operator.
Returns a copy of this node where rule
has been recursively applied first to all of its
children and then itself (post-order).
Returns a copy of this node where rule
has been recursively applied first to all of its
children and then itself (post-order). When rule
does not apply to a given node, it is left
unchanged.
the function use to transform this nodes children
Returns a string representation of the nodes in this tree
Returns a string representation of the nodes in this tree
Returns a copy of this node with the children replaced.
Returns a copy of this node with the children replaced. TODO: Validate somewhere (in debug mode?) that children are ordered correctly.
Return a new RDD that has exactly
numPartitions
partitions. Differs from RepartitionByExpression as this method is called directly by DataFrame's, because the user asked forcoalesce
orrepartition
. RepartitionByExpression is used when the consumer of the output requires some specific ordering or distribution of the data.