Class

com.salesforce.op.stages.sparkwrappers.specific

OpPredictorWrapper

Related Doc: package specific

Permalink

class OpPredictorWrapper[E <: Predictor[Vector, E, M], M <: PredictionModel[Vector, M]] extends Estimator[OpPredictorWrapperModel[M]] with OpPipelineStage2[RealNN, OPVector, Prediction] with SparkWrapperParams[E]

Wraps a spark ML predictor. Predictors represent supervised learning algorithms (regression and classification) in spark ML that inherit from Predictor, supported models are: org.apache.spark.ml.classification.LogisticRegression org.apache.spark.ml.regression.LinearRegression, org.apache.spark.ml.classification.RandomForestClassifier, org.apache.spark.ml.regression.RandomForestRegressor, org.apache.spark.ml.classification.NaiveBayesModel, org.apache.spark.ml.classification.GBTClassifier, org.apache.spark.ml.regression.GBTRegressor, org.apache.spark.ml.classification.DecisionTreeClassifier org.apache.spark.ml.regression.DecisionTreeRegressor, org.apache.spark.ml.classification.LinearSVC org.apache.spark.ml.classification.MultilayerPerceptronClassifier, org.apache.spark.ml.regression.GeneralizedLinearRegression. Their defining characteristic is that they output a model which takes in 2 columns as input (labels and features) and output one to three column as result.

E

spark estimator to wrap

M

spark model returned

Linear Supertypes
SparkWrapperParams[E], OpPipelineStage2[RealNN, OPVector, Prediction], HasIn2, HasIn1, OpPipelineStage[Prediction], OpPipelineStageBase, MLWritable, OpPipelineStageParams, InputParams, Estimator[OpPredictorWrapperModel[M]], PipelineStage, Logging, Params, Serializable, Serializable, Identifiable, AnyRef, Any
Known Subclasses
OpDecisionTreeClassifier, OpDecisionTreeRegressor, OpGBTClassifier, OpGBTRegressor, OpGeneralizedLinearRegression, OpLinearRegression, OpLinearSVC, OpLogisticRegression, OpMultilayerPerceptronClassifier, OpNaiveBayes, OpRandomForestClassifier, OpRandomForestRegressor, OpXGBoostClassifier, OpXGBoostRegressor
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. OpPredictorWrapper
  2. SparkWrapperParams
  3. OpPipelineStage2
  4. HasIn2
  5. HasIn1
  6. OpPipelineStage
  7. OpPipelineStageBase
  8. MLWritable
  9. OpPipelineStageParams
  10. InputParams
  11. Estimator
  12. PipelineStage
  13. Logging
  14. Params
  15. Serializable
  16. Serializable
  17. Identifiable
  18. AnyRef
  19. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new OpPredictorWrapper(predictor: E, uid: String = UID[OpPredictorWrapper[_, _]])(implicit tti1: scala.reflect.api.JavaUniverse.TypeTag[RealNN], tti2: scala.reflect.api.JavaUniverse.TypeTag[OPVector], tto: scala.reflect.api.JavaUniverse.TypeTag[Prediction], ttov: scala.reflect.api.JavaUniverse.TypeTag[Map[String, Double]])

    Permalink

    predictor

    the predictor to wrap

    uid

    stage uid

Type Members

  1. final type InputFeatures = (FeatureLike[RealNN], FeatureLike[OPVector])

    Permalink
    Definition Classes
    OpPipelineStage2 → OpPipelineStage → InputParams

  2. final type OutputFeatures = FeatureLike[Prediction]

    Permalink
    Definition Classes
    OpPipelineStage → OpPipelineStageBase

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def $[T](param: Param[T]): T

    Permalink
    Attributes
    protected
    Definition Classes
    Params

  4. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  5. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  6. final def checkInputLength(features: Array[_]): Boolean

    Permalink
    Definition Classes
    OpPipelineStage2 → InputParams

  7. def checkSerializable: Try[Unit]

    Permalink
    Definition Classes
    OpPipelineStageBase

  8. final def clear(param: Param[_]): OpPredictorWrapper.this.type

    Permalink
    Definition Classes
    Params

  9. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  10. final def copy(extra: ParamMap): OpPredictorWrapper.this.type

    Permalink
    Definition Classes
    OpPipelineStageBase → Params

  11. def copyValues[T <: Params](to: T, extra: ParamMap): T

    Permalink
    Attributes
    protected
    Definition Classes
    Params

  12. final def defaultCopy[T <: Params](extra: ParamMap): T

    Permalink
    Attributes
    protected
    Definition Classes
    Params

  13. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  14. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  15. def explainParam(param: Param[_]): String

    Permalink
    Definition Classes
    Params

  16. def explainParams(): String

    Permalink
    Definition Classes
    Params

  17. final def extractParamMap(): ParamMap

    Permalink
    Definition Classes
    Params

  18. final def extractParamMap(extra: ParamMap): ParamMap

    Permalink
    Definition Classes
    Params

  19. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  20. def fit(dataset: Dataset[_]): OpPredictorWrapperModel[M]

    Permalink

    Function that fits the binary model

    Function that fits the binary model

    Definition Classes
    OpPredictorWrapper → Estimator

  21. def fit(dataset: Dataset[_], paramMaps: Array[ParamMap]): Seq[OpPredictorWrapperModel[M]]

    Permalink
    Definition Classes
    Estimator
    Annotations
    @Since( "2.0.0" )

  22. def fit(dataset: Dataset[_], paramMap: ParamMap): OpPredictorWrapperModel[M]

    Permalink
    Definition Classes
    Estimator
    Annotations
    @Since( "2.0.0" )

  23. def fit(dataset: Dataset[_], firstParamPair: ParamPair[_], otherParamPairs: ParamPair[_]*): OpPredictorWrapperModel[M]

    Permalink
    Definition Classes
    Estimator
    Annotations
    @Since( "2.0.0" ) @varargs()

  24. final def get[T](param: Param[T]): Option[T]

    Permalink
    Definition Classes
    Params

  25. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  26. final def getDefault[T](param: Param[T]): Option[T]

    Permalink
    Definition Classes
    Params

  27. final def getInputFeature[T <: FeatureType](i: Int): Option[FeatureLike[T]]

    Permalink
    Definition Classes
    InputParams

  28. final def getInputFeatures(): Array[OPFeature]

    Permalink
    Definition Classes
    InputParams

  29. final def getInputSchema(): StructType

    Permalink
    Definition Classes
    OpPipelineStageParams

  30. final def getMetadata(): Metadata

    Permalink
    Definition Classes
    OpPipelineStageParams

  31. final def getOrDefault[T](param: Param[T]): T

    Permalink
    Definition Classes
    Params

  32. def getOutput(): FeatureLike[Prediction]

    Permalink
    Definition Classes
    OpPipelineStage2 → OpPipelineStageBase

  33. final def getOutputFeatureName: String

    Permalink
    Definition Classes
    OpPipelineStage

  34. def getParam(paramName: String): Param[Any]

    Permalink
    Definition Classes
    Params

  35. def getSparkMlStage(): Option[E]

    Permalink
    Definition Classes
    SparkWrapperParams

  36. def getStageSavePath(): Option[String]

    Permalink
    Definition Classes
    SparkWrapperParams

  37. final def getTransientFeature(i: Int): Option[TransientFeature]

    Permalink
    Definition Classes
    InputParams

  38. final def getTransientFeatures(): Array[TransientFeature]

    Permalink
    Definition Classes
    InputParams

  39. final def hasDefault[T](param: Param[T]): Boolean

    Permalink
    Definition Classes
    Params

  40. def hasParam(paramName: String): Boolean

    Permalink
    Definition Classes
    Params

  41. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  42. final def in1: TransientFeature

    Permalink
    Attributes
    protected
    Definition Classes
    HasIn1

  43. final def in2: TransientFeature

    Permalink
    Attributes
    protected
    Definition Classes
    HasIn2

  44. def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  45. def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  46. final def inputAsArray(in: InputFeatures): Array[OPFeature]

    Permalink
    Definition Classes
    OpPipelineStage2 → InputParams

  47. val inputParam1Name: String

    Permalink

  48. val inputParam2Name: String

    Permalink

  49. final def isDefined(param: Param[_]): Boolean

    Permalink
    Definition Classes
    Params

  50. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  51. final def isSet(param: Param[_]): Boolean

    Permalink
    Definition Classes
    Params

  52. def isTraceEnabled(): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  53. def log: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  54. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  55. def logDebug(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  56. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  57. def logError(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  58. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  59. def logInfo(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  60. def logName: String

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  61. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  62. def logTrace(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  63. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  64. def logWarning(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  65. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  66. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  67. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  68. def onGetMetadata(): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    OpPipelineStageParams

  69. def onSetInput(): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    OpPipelineStageBase

  70. val operationName: String

    Permalink
    Definition Classes
    OpPredictorWrapper → OpPipelineStageBase

  71. final def outputAsArray(out: OutputFeatures): Array[OPFeature]

    Permalink
    Definition Classes
    OpPipelineStage → OpPipelineStageBase

  72. def outputFeatureUid: String

    Permalink
    Attributes
    protected[com.salesforce.op]
    Definition Classes
    OpPipelineStage2 → OpPipelineStage

  73. def outputIsResponse: Boolean

    Permalink
    Definition Classes
    OpPipelineStage

  74. val outputParamName: String

    Permalink

  75. lazy val params: Array[Param[_]]

    Permalink
    Definition Classes
    Params

  76. val predictor: E

    Permalink

    the predictor to wrap

  77. def save(path: String): Unit

    Permalink
    Definition Classes
    MLWritable
    Annotations
    @Since( "1.6.0" ) @throws( ... )

  78. final def set(paramPair: ParamPair[_]): OpPredictorWrapper.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    Params

  79. final def set(param: String, value: Any): OpPredictorWrapper.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    Params

  80. final def set[T](param: Param[T], value: T): OpPredictorWrapper.this.type

    Permalink
    Definition Classes
    Params

  81. final def setDefault(paramPairs: ParamPair[_]*): OpPredictorWrapper.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    Params

  82. final def setDefault[T](param: Param[T], value: T): OpPredictorWrapper.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    Params

  83. final def setInput(features: InputFeatures): OpPredictorWrapper.this.type

    Permalink
    Definition Classes
    OpPipelineStageBase

  84. final def setInputFeatures[S <: OPFeature](features: Array[S]): OpPredictorWrapper.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    InputParams

  85. final def setMetadata(m: Metadata): OpPredictorWrapper.this.type

    Permalink
    Definition Classes
    OpPipelineStageParams

  86. def setOutputFeatureName(name: String): OpPredictorWrapper.this.type

    Permalink
    Definition Classes
    OpPipelineStage

  87. def setSparkMlStage(stage: Option[E]): OpPredictorWrapper.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    SparkWrapperParams

  88. def setStageSavePath(path: String): OpPredictorWrapper.this.type

    Permalink
    Definition Classes
    SparkWrapperParams

  89. final val sparkInputColParamNames: StringArrayParam

    Permalink
    Definition Classes
    SparkWrapperParams

  90. final val sparkMlStage: SparkStageParam[E]

    Permalink
    Definition Classes
    SparkWrapperParams

  91. final val sparkOutputColParamNames: StringArrayParam

    Permalink
    Definition Classes
    SparkWrapperParams

  92. final def stageName: String

    Permalink
    Definition Classes
    OpPipelineStageBase

  93. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  94. def toString(): String

    Permalink
    Definition Classes
    Identifiable → AnyRef → Any

  95. final def transformSchema(schema: StructType): StructType

    Permalink
    Definition Classes
    OpPipelineStageBase

  96. def transformSchema(schema: StructType, logging: Boolean): StructType

    Permalink
    Attributes
    protected
    Definition Classes
    PipelineStage
    Annotations
    @DeveloperApi()

  97. implicit val tti1: scala.reflect.api.JavaUniverse.TypeTag[RealNN]

    Permalink

  98. implicit val tti2: scala.reflect.api.JavaUniverse.TypeTag[OPVector]

    Permalink

  99. implicit val tto: scala.reflect.api.JavaUniverse.TypeTag[Prediction]

    Permalink
    Definition Classes
    OpPredictorWrapper → OpPipelineStage2

  100. implicit val ttov: scala.reflect.api.JavaUniverse.TypeTag[Map[String, Double]]

    Permalink
    Definition Classes
    OpPredictorWrapper → OpPipelineStage2

  101. val uid: String

    Permalink

    stage uid

    stage uid

    Definition Classes
    OpPredictorWrapper → Identifiable

  102. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  103. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  104. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  105. final def write: MLWriter

    Permalink
    Definition Classes
    OpPipelineStageBase → MLWritable

Inherited from SparkWrapperParams[E]

Inherited from OpPipelineStage2[RealNN, OPVector, Prediction]

Inherited from HasIn2

Inherited from HasIn1

Inherited from OpPipelineStage[Prediction]

Inherited from OpPipelineStageBase

Inherited from MLWritable

Inherited from OpPipelineStageParams

Inherited from InputParams

Inherited from Estimator[OpPredictorWrapperModel[M]]

Inherited from PipelineStage

Inherited from Logging

Inherited from Params

Inherited from Serializable

Inherited from Serializable

Inherited from Identifiable

Inherited from AnyRef

Inherited from Any

Ungrouped