com.ignition.frame

BasicStats

case class BasicStats(dataFields: Iterable[(String, BasicAggregator)], groupFields: Iterable[String] = immutable.this.Nil) extends FrameTransformer with Product with Serializable

Calculates basic statistics about the specified fields.

Linear Supertypes
Product, Equals, FrameTransformer, FrameStep, Transformer[DataFrame, SparkRuntime], SingleInputStep[DataFrame, SparkRuntime], ConnectionTarget[DataFrame, SparkRuntime], SingleOutputStep[DataFrame, SparkRuntime], ConnectionSource[DataFrame, SparkRuntime], Step[DataFrame, SparkRuntime], JsonExport, XmlExport, AbstractStep, Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. BasicStats
  2. Product
  3. Equals
  4. FrameTransformer
  5. FrameStep
  6. Transformer
  7. SingleInputStep
  8. ConnectionTarget
  9. SingleOutputStep
  10. ConnectionSource
  11. Step
  12. JsonExport
  13. XmlExport
  14. AbstractStep
  15. Serializable
  16. Serializable
  17. AnyRef
  18. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new BasicStats(dataFields: Iterable[(String, BasicAggregator)], groupFields: Iterable[String] = immutable.this.Nil)

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any

  3. final def ##(): Int

    Definition Classes
    AnyRef → Any

  4. def %(field: String, functions: BasicAggregator*): BasicStats

  5. def %(tuple: (String, BasicAggregator)): BasicStats

  6. def -->(targets: ConnectionTarget[DataFrame, SparkRuntime]*): Unit

    Definition Classes
    ConnectionSource

  7. def -->(tgt: MultiInputStep[DataFrame, SparkRuntime]): tgt.type

    Definition Classes
    ConnectionSource

  8. def -->(tgt: ConnectionTarget[DataFrame, SparkRuntime]): tgt.type

    Definition Classes
    ConnectionSource

  9. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  10. final def ==(arg0: Any): Boolean

    Definition Classes
    Any

  11. def add(field: String, functions: BasicAggregator*): BasicStats

  12. def add(tuple: (String, BasicAggregator)): BasicStats

  13. def addStepListener(listener: StepListener[DataFrame, SparkRuntime]): Unit

    Registers a new listener.

    Registers a new listener.

    Definition Classes
    Step

  14. def allInputsRequired: Boolean

    Specifies if the step should throw an error if one of the inputs is not connected.

    Specifies if the step should throw an error if one of the inputs is not connected.

    Definition Classes
    Step

  15. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  16. def buildSchema(index: Int)(implicit runtime: SparkRuntime): StructType

    Returns the output schema of the step at a given index by calling output() in preview mode and retrieving the schema from the DataFrame.

    Returns the output schema of the step at a given index by calling output() in preview mode and retrieving the schema from the DataFrame. Since the default implementation of this method calls output(), invoking it from output() directly or indirectly will result in infinite cycle.

    Attributes
    protected
    Definition Classes
    FrameStep

  17. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  18. def compute(arg: DataFrame)(implicit runtime: SparkRuntime): DataFrame

    Attributes
    protected
    Definition Classes
    BasicStats → Transformer

  19. def compute(index: Int)(implicit runtime: SparkRuntime): DataFrame

    Computes a step output value at the specified index.

    Computes a step output value at the specified index. This method is invoked from output() and can safely throw any exception, which will be wrapped into ExecutionException.

    index

    the output value index.

    Attributes
    protected
    Definition Classes
    Transformer → Step

  20. def ctx(implicit runtime: SparkRuntime): SQLContext

    Returns the implicit SQLContext.

    Returns the implicit SQLContext.

    Attributes
    protected
    Definition Classes
    FrameStep

  21. val dataFields: Iterable[(String, BasicAggregator)]

  22. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  23. final def evaluate(implicit runtime: SparkRuntime): IndexedSeq[DataFrame]

    Evaluates all step's outputs and returns a list of results.

    Evaluates all step's outputs and returns a list of results.

    Definition Classes
    Step
    Annotations
    @throws( classOf[ExecutionException] )

  24. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  25. def from(src: ConnectionSource[DataFrame, SparkRuntime]): BasicStats.this.type

    Definition Classes
    ConnectionTarget

  26. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  27. def groupBy(fields: String*): BasicStats

  28. val groupFields: Iterable[String]

  29. val index: Int

    Definition Classes
    Transformer → SingleInputStep → ConnectionTarget → SingleOutputStep → ConnectionSource

  30. def input(implicit runtime: SparkRuntime): DataFrame

    Definition Classes
    SingleInputStep

  31. val inputCount: Int

    The maximum number of input ports.

    The maximum number of input ports.

    Definition Classes
    SingleInputStep → Step

  32. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  33. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  34. final def notify(): Unit

    Definition Classes
    AnyRef

  35. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  36. def optLimit(df: DataFrame, preview: Boolean): DataFrame

    Takes one row from the data frame is preview is true.

    Takes one row from the data frame is preview is true.

    Attributes
    protected
    Definition Classes
    FrameStep

  37. final def outSchema(implicit runtime: SparkRuntime): StructType

    Returns the output schema of the step at index 0.

    Returns the output schema of the step at index 0.

    Definition Classes
    FrameStep

  38. final def outSchema(index: Int)(implicit runtime: SparkRuntime): StructType

    Returns the output schema of the step at a given index.

    Returns the output schema of the step at a given index.

    Definition Classes
    FrameStep

  39. def outbounds(index: Int): SingleOutputStep[DataFrame, SparkRuntime]

    Definition Classes
    SingleOutputStep

  40. final def output(implicit runtime: SparkRuntime): DataFrame

    Shortcut for output(0).

    Shortcut for output(0). Computes a step output at index 0.

    Definition Classes
    Step
    Annotations
    @throws( classOf[ExecutionException] )
    Exceptions thrown
    ExecutionException

    in case of an error, or if the step is not connected.

  41. final def output(index: Int)(implicit runtime: SparkRuntime): DataFrame

    Computes a step output value at the specified index.

    Computes a step output value at the specified index.

    index

    the output value index.

    Definition Classes
    Step
    Annotations
    @throws( classOf[ExecutionException] )
    Exceptions thrown
    ExecutionException

    in case of an error, or if the step is not connected.

  42. val outputCount: Int

    The number of output ports.

    The number of output ports.

    Definition Classes
    SingleOutputStep → Step

  43. def removeStepListener(listener: StepListener[DataFrame, SparkRuntime]): Unit

    Unregisters a listener.

    Unregisters a listener.

    Definition Classes
    Step

  44. def resetCache(predecessors: Boolean, descendants: Boolean): Unit

    Clears the cache of this step and optionally that of its predecessors and descendants.

    Clears the cache of this step and optionally that of its predecessors and descendants.

    Definition Classes
    Step

  45. def sc(implicit runtime: SparkRuntime): SparkContext

    Returns the implicit SparkContext.

    Returns the implicit SparkContext.

    Attributes
    protected
    Definition Classes
    FrameStep

  46. val step: Transformer[DataFrame, SparkRuntime]

    Definition Classes
    Transformer → SingleInputStep → ConnectionTarget → SingleOutputStep → ConnectionSource

  47. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  48. def to(targets: ConnectionTarget[DataFrame, SparkRuntime]*): Unit

    Definition Classes
    ConnectionSource

  49. def to(tgt: MultiInputStep[DataFrame, SparkRuntime]): tgt.type

    Definition Classes
    ConnectionSource

  50. def to(tgt: ConnectionTarget[DataFrame, SparkRuntime]): tgt.type

    Definition Classes
    ConnectionSource

  51. def toJson: JValue

    Definition Classes
    BasicStats → JsonExport

  52. def toXml: Elem

    Definition Classes
    BasicStats → XmlExport

  53. def value(implicit runtime: SparkRuntime): DataFrame

    Definition Classes
    SingleOutputStep → ConnectionSource

  54. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  55. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  56. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  57. final def wrap[U](body: ⇒ U): U

    Wraps exceptions into ExecutionException instances.

    Wraps exceptions into ExecutionException instances.

    Attributes
    protected
    Definition Classes
    AbstractStep

Inherited from Product

Inherited from Equals

Inherited from FrameTransformer

Inherited from FrameStep

Inherited from Transformer[DataFrame, SparkRuntime]

Inherited from SingleInputStep[DataFrame, SparkRuntime]

Inherited from ConnectionTarget[DataFrame, SparkRuntime]

Inherited from SingleOutputStep[DataFrame, SparkRuntime]

Inherited from ConnectionSource[DataFrame, SparkRuntime]

Inherited from Step[DataFrame, SparkRuntime]

Inherited from JsonExport

Inherited from XmlExport

Inherited from AbstractStep

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped