Class/Object

com.stripe.brushfire.scalding

Trainer

Related Docs: object Trainer | package scalding

Permalink

case class Trainer[K, V, T](trainingDataExecution: Execution[TypedPipe[Instance[K, V, T]]], samplerExecution: Execution[Sampler[K]], treeExecution: Execution[TypedPipe[(Int, Tree[K, V, T])]], unitExecution: Execution[Unit], reducers: Int)(implicit evidence$1: Ordering[K], evidence$2: Ordering[V], evidence$3: Monoid[T]) extends Product with Serializable

Linear Supertypes
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. Trainer
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new Trainer(trainingDataExecution: Execution[TypedPipe[Instance[K, V, T]]], samplerExecution: Execution[Sampler[K]], treeExecution: Execution[TypedPipe[(Int, Tree[K, V, T])]], unitExecution: Execution[Unit], reducers: Int)(implicit arg0: Ordering[K], arg1: Ordering[V], arg2: Monoid[T])

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  6. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  7. def execution: Execution[Unit]

    Permalink
  8. def expand[S](path: String)(implicit splitter: Splitter[V, T], evaluator: Evaluator[V, T], stopper: Stopper[T], inj: Injection[Tree[K, V, T], String]): Trainer[K, V, T]

    Permalink

    expand each tree by one level, by attempting to split every leaf.

    expand each tree by one level, by attempting to split every leaf.

    path

    where to save the new tree

    splitter

    the splitter to use to generate candidate splits for each leaf

    evaluator

    the evaluator to use to decide which split to use for each leaf

  9. def expandFrom(base: String, step: Int, to: Int)(implicit splitter: Splitter[V, T], evaluator: Evaluator[V, T], stopper: Stopper[T], inj: Injection[Tree[K, V, T], String]): Trainer[K, V, T]

    Permalink
  10. def expandInMemory(path: String, times: Int)(implicit splitter: Splitter[V, T], evaluator: Evaluator[V, T], stopper: Stopper[T], inj: Injection[Tree[K, V, T], String]): Trainer[K, V, T]

    Permalink
  11. def expandTimes(base: String, times: Int)(implicit splitter: Splitter[V, T], evaluator: Evaluator[V, T], stopper: Stopper[T], inj: Injection[Tree[K, V, T], String]): Trainer[K, V, T]

    Permalink

    recursively expand multiple times, writing out the new tree at each step

  12. def featureImportance[P, E](error: Error[T, P, E])(fn: (TypedPipe[(K, E)]) ⇒ Execution[_])(implicit voter: Voter[T, P]): Trainer[K, V, T]

    Permalink

    featureImportance should: shuffle data randomly (group on something random then sort on something random?), then stream through and have each instance pick one feature value at random to pass on to the following instance.

    featureImportance should: shuffle data randomly (group on something random then sort on something random?), then stream through and have each instance pick one feature value at random to pass on to the following instance. then group by permuted feature and compare error.

  13. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  14. def flatMapSampler(fn: ((TypedPipe[Instance[K, V, T]], Sampler[K])) ⇒ Execution[Sampler[K]]): Trainer[K, V, T]

    Permalink
  15. def flatMapTrees(fn: ((TypedPipe[Instance[K, V, T]], Sampler[K], Iterable[(Int, Tree[K, V, T])])) ⇒ Execution[TypedPipe[(Int, Tree[K, V, T])]]): Trainer[K, V, T]

    Permalink
  16. def forceTrainingDataToDisk: Trainer[K, V, T]

    Permalink
  17. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  18. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  19. def load(path: String)(implicit inj: Injection[Tree[K, V, T], String]): Trainer[K, V, T]

    Permalink
  20. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  21. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  22. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  23. def outOfTime(quantile: Double = 0.8): Trainer[K, V, T]

    Permalink

    add out of time validation

  24. def prune[P, E](path: String, error: Error[T, P, E])(implicit voter: Voter[T, P], inj: Injection[Tree[K, V, T], String], ord: Ordering[E]): Trainer[K, V, T]

    Permalink

    prune a tree to minimize validation error

    prune a tree to minimize validation error

    Construct a Map[Int,T] from the trainingData for each tree, and then transform the trees using the prune method.

  25. val reducers: Int

    Permalink
  26. val samplerExecution: Execution[Sampler[K]]

    Permalink
  27. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  28. def tee[A](fn: ((TypedPipe[Instance[K, V, T]], Sampler[K], Iterable[(Int, Tree[K, V, T])])) ⇒ Execution[A]): Trainer[K, V, T]

    Permalink
  29. val trainingDataExecution: Execution[TypedPipe[Instance[K, V, T]]]

    Permalink
  30. val treeExecution: Execution[TypedPipe[(Int, Tree[K, V, T])]]

    Permalink
  31. val unitExecution: Execution[Unit]

    Permalink
  32. def updateTargets(path: String)(implicit inj: Injection[Tree[K, V, T], String]): Trainer[K, V, T]

    Permalink

    Update the leaves of the current trees from the training set.

    Update the leaves of the current trees from the training set.

    The leaves target distributions will be set to the summed distributions of the instances in the training set that would get classified to them. Often used to initialize an empty tree.

  33. def validate[P, E](error: Error[T, P, E])(fn: (ValuePipe[E]) ⇒ Execution[_])(implicit voter: Voter[T, P]): Trainer[K, V, T]

    Permalink

    produce an error object from the current trees and the validation set

  34. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  35. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  36. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped