Package

com.github.sadikovi.spark.netflow

index

Permalink

package index

Visibility
  1. Public
  2. All

Type Members

  1. class Attribute[T] extends AnyRef

    Permalink

    Attribute interface to collect and check statistics.

    Attribute interface to collect and check statistics. Included support of different combinations of collected parameters: count, min/max, and set of values through bit vector. Here are some common flags: 7 - enable all parameters, 1 - enable count, 6 - enable min/max and set. In order to create attribute comparison function is required similar the lt function used in sortWith method. Name must unique to the attribute.

  2. case class StatisticsPathResolver(maybeRoot: Option[String]) extends Product with Serializable

    Permalink

    StatisticsPathResolver is a simple class to find the statistics path based on a file path.

    StatisticsPathResolver is a simple class to find the statistics path based on a file path. Also takes into account possible root to store/read statistics. Note that root may not necessarily exist.

  3. case class StatisticsPathStatus(path: String, exists: Boolean) extends Product with Serializable

    Permalink

    StatisticsPathStatus is a holder of the parameters for statistics file.

    StatisticsPathStatus is a holder of the parameters for statistics file. path is a fully-qualified file path, either HDFS or local file system, and exists indicates if file for that path exists.

Value Members

  1. object Attribute

    Permalink
  2. object AttributeMap

    Permalink

    Interface to create or load attribute map.

    Interface to create or load attribute map. When creating new attribute map returns map with predefined set of statistics. Note that number, name and type of attributes should be in sync with resolved interface columns, meaning that columns with attribute names should have statistics enabled, otherwise it will not collect anything, resulting in incorrect filtering.

  3. object StatisticsUtils

    Permalink

Ungrouped