OpWorkflowRunner

Value Members

1. final def !=(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

2. final def ##(): Int

   

   Definition Classes

   AnyRef → Any

3. final def ==(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

4. final def addApplicationEndHandler(h: (AppMetrics) ⇒ Unit): OpWorkflowRunner.this.type

   

   Add handle to collect the app metrics when the application ends

   Add handle to collect the app metrics when the application ends

   h

   a handle to collect the app metrics when the application ends

5. final def asInstanceOf[T0]: T0

   

   Definition Classes

   Any

6. def clone(): AnyRef

   

   Attributes

   protected[java.lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( ... )

7. def computeFeatures(params: OpParams)(implicit spark: SparkSession): OpWorkflowRunnerResult

   

   This method can be used to generate features from part of your workflow for exploration outside the app

   This method can be used to generate features from part of your workflow for exploration outside the app

   params

   parameters injected at runtime

   spark

   spark context which runs the workflow

   returns

   FeaturesResult

   Attributes

   protected

8. final def eq(arg0: AnyRef): Boolean

   

   Definition Classes

   AnyRef

9. def equals(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

10. def evaluate(params: OpParams)(implicit spark: SparkSession): OpWorkflowRunnerResult

    

    This method is used to call the evaluator on a test set of data scored by the trained workflow

    This method is used to call the evaluator on a test set of data scored by the trained workflow

    params

    parameters injected at runtime

    spark

    spark context which runs the workflow

    returns

    EvaluateResult

    Attributes

    protected

11. val evaluationReader: Option[Reader[_]]

    

    reader to use to load up data for evaluation

12. val evaluator: Option[OpEvaluatorBase[_ <: EvaluationMetrics]]

    

    evaluator that you wish to use to evaluate the results of your workflow on a test dataset

13. val featureToComputeUpTo: Option[OPFeature]

    

    feature to generate data upto if calling the 'Features' run type

14. def finalize(): Unit

    

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

15. final def getClass(): Class[_]

    

    Definition Classes

    AnyRef → Any

16. def hashCode(): Int

    

    Definition Classes

    AnyRef → Any

17. final def isInstanceOf[T0]: Boolean

    

    Definition Classes

    Any

18. lazy val log: Logger

    

    Attributes

    protected

19. final def ne(arg0: AnyRef): Boolean

    

    Definition Classes

    AnyRef

20. final def notify(): Unit

    

    Definition Classes

    AnyRef

21. final def notifyAll(): Unit

    

    Definition Classes

    AnyRef

22. def run(runType: OpWorkflowRunType, opParams: OpParams)(implicit spark: SparkSession, streaming: StreamingContext): OpWorkflowRunnerResult

    

    Run using a specified runner config

    Run using a specified runner config

    runType

    workflow run type

    opParams

    parameters injected at runtime

    spark

    spark context which runs the workflow

    streaming

    spark streaming context which runs the workflow

    returns

    runner result

23. def score(params: OpParams)(implicit spark: SparkSession): OpWorkflowRunnerResult

    

    This method is used to load up a previously trained workflow and use it to score a new dataset

    This method is used to load up a previously trained workflow and use it to score a new dataset

    params

    parameters injected at runtime

    spark

    spark context which runs the workflow

    returns

    ScoreResult

    Attributes

    protected

24. val scoringEvaluator: Option[OpEvaluatorBase[_ <: EvaluationMetrics]]

    

    optional scoring evaluator that you wish to use when scoring

25. val scoringReader: Reader[_]

    

    reader to use to load up data for scoring

26. def streamingScore(params: OpParams)(implicit spark: SparkSession, streaming: StreamingContext): OpWorkflowRunnerResult

    

    This method is used to load up a previously trained workflow and use it to stream scores to a write location

    This method is used to load up a previously trained workflow and use it to stream scores to a write location

    params

    parameters injected at runtime

    spark

    spark context which runs the workflow

    streaming

    spark streaming context which runs the workflow

    returns

    StreamingScoreResult

    Attributes

    protected

27. val streamingScoreReader: Option[StreamingReader[_]]

    

    reader to use to load up data for streaming score

28. final def synchronized[T0](arg0: ⇒ T0): T0

    

    Definition Classes

    AnyRef

29. def toString(): String

    

    Definition Classes

    AnyRef → Any

30. def train(params: OpParams)(implicit spark: SparkSession): OpWorkflowRunnerResult

    

    This method is called to train your workflow

    This method is called to train your workflow

    params

    parameters injected at runtime

    spark

    spark context which runs the workflow

    returns

    TrainResult

    Attributes

    protected

31. val trainingReader: Reader[_]

    

    reader to use to load up data for training

32. final def wait(): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

33. final def wait(arg0: Long, arg1: Int): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

34. final def wait(arg0: Long): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

35. val workflow: OpWorkflow

    

    the workflow that you want to run (Note: the workflow should have the resultFeatures set)