com.stripe.brushfire.scalding

Trainer

Related Docs: object Trainer | package scalding

case class Trainer[K, V, T](trainingDataExecution: Execution[TypedPipe[Instance[K, V, T]]], samplerExecution: Execution[Sampler[K]], treeExecution: Execution[TypedPipe[(Int, Tree[K, V, T])]], unitExecution: Execution[Unit], reducers: Int)(implicit evidence$1: Ordering[K], evidence$2: Ordering[V], evidence$3: Monoid[T]) extends Product with Serializable

Linear Supertypes
Serializable, Serializable, Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. Trainer
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. AnyRef
  7. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new Trainer(trainingDataExecution: Execution[TypedPipe[Instance[K, V, T]]], samplerExecution: Execution[Sampler[K]], treeExecution: Execution[TypedPipe[(Int, Tree[K, V, T])]], unitExecution: Execution[Unit], reducers: Int)(implicit arg0: Ordering[K], arg1: Ordering[V], arg2: Monoid[T])

Value Members

  1. final def !=(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  5. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  6. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  7. def execution: Execution[Unit]

  8. def expand[S](path: String)(implicit splitter: Splitter[V, T], evaluator: Evaluator[V, T], stopper: Stopper[T], inj: Injection[Tree[K, V, T], String]): Trainer[K, V, T]

    expand each tree by one level, by attempting to split every leaf.

    expand each tree by one level, by attempting to split every leaf.

    path

    where to save the new tree

    splitter

    the splitter to use to generate candidate splits for each leaf

    evaluator

    the evaluator to use to decide which split to use for each leaf

  9. def expandFrom(base: String, step: Int, to: Int)(implicit splitter: Splitter[V, T], evaluator: Evaluator[V, T], stopper: Stopper[T], inj: Injection[Tree[K, V, T], String]): Trainer[K, V, T]

  10. def expandInMemory(path: String, times: Int)(implicit splitter: Splitter[V, T], evaluator: Evaluator[V, T], stopper: Stopper[T], inj: Injection[Tree[K, V, T], String]): Trainer[K, V, T]

  11. def expandTimes(base: String, times: Int)(implicit splitter: Splitter[V, T], evaluator: Evaluator[V, T], stopper: Stopper[T], inj: Injection[Tree[K, V, T], String]): Trainer[K, V, T]

    recursively expand multiple times, writing out the new tree at each step

  12. def featureImportance[P, E](error: Error[T, P, E])(fn: (TypedPipe[(K, E)]) ⇒ Execution[_])(implicit voter: Voter[T, P]): Trainer[K, V, T]

    featureImportance should: shuffle data randomly (group on something random then sort on something random?), then stream through and have each instance pick one feature value at random to pass on to the following instance.

    featureImportance should: shuffle data randomly (group on something random then sort on something random?), then stream through and have each instance pick one feature value at random to pass on to the following instance. then group by permuted feature and compare error.

    E
    error
    returns

  13. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  14. def flatMapSampler(fn: ((TypedPipe[Instance[K, V, T]], Sampler[K])) ⇒ Execution[Sampler[K]]): Trainer[K, V, T]

  15. def flatMapTrees(fn: ((TypedPipe[Instance[K, V, T]], Sampler[K], Iterable[(Int, Tree[K, V, T])])) ⇒ Execution[TypedPipe[(Int, Tree[K, V, T])]]): Trainer[K, V, T]

  16. def forceTrainingDataToDisk: Trainer[K, V, T]

  17. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  18. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  19. def load(path: String)(implicit inj: Injection[Tree[K, V, T], String]): Trainer[K, V, T]

  20. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  21. final def notify(): Unit

    Definition Classes
    AnyRef
  22. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  23. def outOfTime(quantile: Double = 0.8): Trainer[K, V, T]

    add out of time validation

  24. def prune[P, E](path: String, error: Error[T, P, E])(implicit voter: Voter[T, P], inj: Injection[Tree[K, V, T], String], ord: Ordering[E]): Trainer[K, V, T]

    prune a tree to minimize validation error

    prune a tree to minimize validation error

    Construct a Map[Int,T] from the trainingData for each tree, and then transform the trees using the prune method.

  25. val reducers: Int

  26. val samplerExecution: Execution[Sampler[K]]

  27. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  28. def tee[A](fn: ((TypedPipe[Instance[K, V, T]], Sampler[K], Iterable[(Int, Tree[K, V, T])])) ⇒ Execution[A]): Trainer[K, V, T]

  29. val trainingDataExecution: Execution[TypedPipe[Instance[K, V, T]]]

  30. val treeExecution: Execution[TypedPipe[(Int, Tree[K, V, T])]]

  31. val unitExecution: Execution[Unit]

  32. def updateTargets(path: String)(implicit inj: Injection[Tree[K, V, T], String]): Trainer[K, V, T]

    Update the leaves of the current trees from the training set.

    Update the leaves of the current trees from the training set.

    The leaves target distributions will be set to the summed distributions of the instances in the training set that would get classified to them. Often used to initialize an empty tree.

  33. def validate[P, E](error: Error[T, P, E])(fn: (ValuePipe[E]) ⇒ Execution[_])(implicit voter: Voter[T, P]): Trainer[K, V, T]

    produce an error object from the current trees and the validation set

  34. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  35. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  36. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped