Trait

ai.deepsense.deeplang.params.wrappers.spark

SparkParamWrapper

Related Doc: package spark

Permalink

trait SparkParamWrapper[P <: org.apache.spark.ml.param.Params, T, U] extends Param[U]

This trait should be mixed in by deeplang parameters derived from Spark parameters. Wrapped Spark parameters should be able to convert deeplang parameter values to Spark parameter values.

T

Type of wrapped Spark parameter value

U

Type of deeplang parameter value

Linear Supertypes
Param[U], AnyRef, Any
Known Subclasses
BooleanParamWrapper, ChoiceParamWrapper, ColumnSelectorParamWrapper, DocConcentrationParam, DoubleArrayParamWrapper, DoubleParamWrapper, FloatParamWrapper, ForwardSparkParamWrapper, IntArrayParamWrapper, IntParamWrapper, LongParamWrapper, MultipleChoiceParamWrapper, MultipleColumnCreatorParamWrapper, PrefixBasedColumnCreatorParamWrapper, SingleColumnCreatorParamWrapper, SingleColumnSelectorParamWrapper, StringParamWrapper, TopicConcentrationParam
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SparkParamWrapper
  2. Param
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def convert(value: U)(schema: StructType): T

    Permalink

    Convert deeplang parameter value to wrapped Spark parameter value.

    Convert deeplang parameter value to wrapped Spark parameter value.

    value

    deeplang parameter value

    schema

    DataFrame schema, used in column selectors to extract column names

    returns

    Spark parameter value

  2. abstract val description: Option[String]

    Permalink
    Definition Classes
    Param

  3. abstract val name: String

    Permalink
    Definition Classes
    Param

  4. abstract val parameterType: ParameterType

    Permalink
    Definition Classes
    Param

  5. abstract def replicate(name: String): Param[U]

    Permalink

    Used to extract public parameters in custom transformer.

    Used to extract public parameters in custom transformer.

    name

    name of replicated parameter

    returns

    replicated parameter

    Definition Classes
    Param

  6. abstract val sparkParamGetter: (P) ⇒ org.apache.spark.ml.param.Param[T]

    Permalink

    This function should extract wrapped parameter from Spark params of type P.

  7. abstract def valueFromJson(jsValue: JsValue, graphReader: GraphReader): U

    Permalink
    Definition Classes
    Param

  8. abstract def valueToJson(value: U): JsValue

    Permalink
    Definition Classes
    Param

Concrete Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. def ->(value: U): ParamPair[U]

    Permalink
    Definition Classes
    Param

  4. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  5. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  6. def canEqual(other: Any): Boolean

    Permalink
    Definition Classes
    Param

  7. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  8. def constraints: String

    Permalink
    Definition Classes
    Param

  9. def convertAny(value: Any)(schema: StructType): T

    Permalink

  10. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  11. def equals(other: Any): Boolean

    Permalink
    Definition Classes
    Param → AnyRef → Any

  12. def extraJsFields: Map[String, JsValue]

    Permalink

    Subclasses should overwrite this method if they want to add custom fields to json description.

    Subclasses should overwrite this method if they want to add custom fields to json description.

    Attributes
    protected
    Definition Classes
    Param

  13. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  14. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  15. def hashCode(): Int

    Permalink
    Definition Classes
    Param → AnyRef → Any

  16. val isGriddable: Boolean

    Permalink
    Definition Classes
    Param

  17. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  18. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  19. def nestedWrappers: Seq[SparkParamWrapper[_, _, _]]

    Permalink

    returns

    Wrappers for nested parameters

  20. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  21. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  22. def serializeDefault(default: U): JsValue

    Permalink

    Describes default serialization of default values.

    Describes default serialization of default values.

    default

    Default value of parameter

    Attributes
    protected
    Definition Classes
    Param

  23. def sparkParam(sparkEntity: org.apache.spark.ml.param.Params): org.apache.spark.ml.param.Param[T]

    Permalink

    This method extracts wrapped Spark parameter from Params, using a function defined as sparkParamGetter.

    This method extracts wrapped Spark parameter from Params, using a function defined as sparkParamGetter. The method is used in ParamsWithSparkWrappers to get parameter values.

    sparkEntity

    Spark params

    returns

    Wrapped Spark parameter

  24. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  25. final def toJson(maybeDefault: Option[Any]): JsObject

    Permalink

    Describes json representation of this parameter.

    Describes json representation of this parameter.

    maybeDefault

    Optional default value of parameter. Should be of type Option[T], but we need to receive Any because Params have to use this method without knowing T.

    Definition Classes
    Param

  26. def toString(): String

    Permalink
    Definition Classes
    Param → AnyRef → Any

  27. def validate(value: U): Vector[DeepLangException]

    Permalink
    Definition Classes
    Param

  28. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  29. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  30. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Param[U]

Inherited from AnyRef

Inherited from Any

Ungrouped