org.apache.spark.sql.catalyst.plans.logical

Statistics

case class Statistics(sizeInBytes: BigInt, rowCount: Option[BigInt] = scala.None, colStats: Map[String, ColumnStat] = ..., isBroadcastable: Boolean = false) extends Product with Serializable

Estimates of various statistics. The default estimation logic simply lazily multiplies the corresponding statistic produced by the children. To override this behavior, override statistics and assign it an overridden version of Statistics.

NOTE: concrete and/or overridden versions of statistics fields should pay attention to the performance of the implementations. The reason is that estimations might get triggered in performance-critical processes, such as query plan planning.

Note that we are using a BigInt here since it is easy to overflow a 64-bit integer in cardinality estimation (e.g. cartesian joins).

sizeInBytes

Physical size in bytes. For leaf operators this defaults to 1, otherwise it defaults to the product of children's sizeInBytes.

rowCount

Estimated number of rows.

colStats

Column-level statistics.

isBroadcastable

If true, output is small enough to be used in a broadcast join.

Linear Supertypes
Serializable, Serializable, Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. Statistics
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. AnyRef
  7. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new Statistics(sizeInBytes: BigInt, rowCount: Option[BigInt] = scala.None, colStats: Map[String, ColumnStat] = ..., isBroadcastable: Boolean = false)

    sizeInBytes

    Physical size in bytes. For leaf operators this defaults to 1, otherwise it defaults to the product of children's sizeInBytes.

    rowCount

    Estimated number of rows.

    colStats

    Column-level statistics.

    isBroadcastable

    If true, output is small enough to be used in a broadcast join.

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any

  3. final def ##(): Int

    Definition Classes
    AnyRef → Any

  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any

  6. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  7. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  8. val colStats: Map[String, ColumnStat]

    Column-level statistics.

  9. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  10. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  11. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  12. val isBroadcastable: Boolean

    If true, output is small enough to be used in a broadcast join.

  13. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  14. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  15. final def notify(): Unit

    Definition Classes
    AnyRef

  16. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  17. val rowCount: Option[BigInt]

    Estimated number of rows.

  18. def simpleString: String

    Readable string representation for the Statistics.

  19. val sizeInBytes: BigInt

    Physical size in bytes.

    Physical size in bytes. For leaf operators this defaults to 1, otherwise it defaults to the product of children's sizeInBytes.

  20. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  21. def toString(): String

    Definition Classes
    Statistics → AnyRef → Any

  22. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  23. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  24. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped