Package

com.github.timgent.sparkdataquality

metrics

Permalink

package metrics

Visibility
  1. Public
  2. All

Type Members

  1. case class ComplianceFn(definition: Column, description: String) extends Product with Serializable

    Permalink

    Describes a compliance check for a dataset

    Describes a compliance check for a dataset

    definition

    - a column definition which should return true or false. true if the dataset row meets the compliance check you want, otherwise false.

    description

    - a description of the compliance function which will be used when persisting corresponding metrics

  2. case class DatasetDescription(value: String) extends Product with Serializable

    Permalink

    Wrapper for a string describing a Dataset

    Wrapper for a string describing a Dataset

    value

    - the description of the Dataset

  3. case class MetricComparator[MV <: MetricValue](description: String, fn: (MV, MV) ⇒ Boolean) extends Product with Serializable

    Permalink
  4. sealed trait MetricDescriptor extends AnyRef

    Permalink

    Describes the metric being calculated

  5. case class MetricFilter(filter: Column, filterDescription: String) extends Product with Serializable

    Permalink

    Defines a filter to be applied before a metric is calculated

    Defines a filter to be applied before a metric is calculated

    filter

    - a column containing true or false depending if the row should be included in the metrics calculation

    filterDescription

    - a readable description of the filter (used for persistence)

  6. sealed trait MetricValue extends AnyRef

    Permalink

    Represents the value of a metric

Value Members

  1. object MetricComparator extends Serializable

    Permalink

    Object for some helper functions to help with metric comparisons

  2. object MetricDescriptor

    Permalink
  3. object MetricFilter extends Serializable

    Permalink
  4. object MetricValue

    Permalink

Ungrouped