ai.deepsense.deeplang.doperables
SparkModelWrapper
Related Doc:
package doperables
Creates a transformed DataFrame based on input DataFrame.
Creates a transformed DataFrame based on input DataFrame.
Should be implemented in subclasses.
Should be implemented in subclasses. For known schema of input DataFrame, infers schema of output DataFrame. If it is not able to do it for some reasons, it returns None.
Can be implemented in a subclass, if access to infer context is required.
Can be implemented in a subclass, if access to infer context is required. For known schema of input DataFrame, infers schema of output DataFrame. If it is not able to do it for some reasons, it returns None.
Sequence of params without values for this class, parsed from Json.
Sequence of params without values for this class, parsed from Json. If a name of a parameter is unknown, it's ignored JsNull is treated as empty object. JsNull as a value of a parameter unsets param's value.
Sequence of paramPairs for this class, parsed from Json.
Sequence of paramPairs for this class, parsed from Json. If a name of a parameter is unknown, it's ignored JsNull is treated as empty object. JsNull as value of parameter is ignored.
Json describing values associated to parameters.
Json describing values associated to parameters.
Parent EstimatorWrapper has to be set before _transformSchema() execution - estimators are responsible for schema inference in models.
Parent EstimatorWrapper has to be set before _transformSchema() execution - estimators are responsible for schema inference in models. We use a mutable field because estimator instances that contain fresh parameters are created dynamically. Passing an estimator to _transformSchema by parameter would require a change of Transformer.transformSchema signature.
Compares 'this' and 'other' params.
Compares 'this' and 'other' params. Objects are equal when they are of the same class and their parameters have the same values set.
True, if 'this' and 'other' are the same.
Model has to be set before _transform() execution.
Model has to be set before _transform() execution. We use a mutable field because model is created by estimator dynamically. Passing it to _transform by parameter is not possible without changing Transformer.transform signature.
Spark model wrappers mustn't have default parameters.
Spark model wrappers mustn't have default parameters. All initial parameter values are inherited from parent estimator.
Sets param values based on provided json.
Sets param values based on provided json. If a name of a parameter is unknown, it's ignored JsNull is treated as empty object.
When ignoreNulls = false, JsNull as a value of a parameter unsets param's value. When ignoreNulls = true, parameters with JsNull values are ignored.
This method extracts Spark parameters from SparkParamWrappers that are: - declared directly in class which mixes this trait in - declared in values of parameters (i.e.
This method extracts Spark parameters from SparkParamWrappers that are: - declared directly in class which mixes this trait in - declared in values of parameters (i.e. ChoiceParam, MultipleChoiceParam)
Validates Params entities that contain dynamic parameters' values.
Validates Params entities that contain dynamic parameters' values. Validation errors are wrapped in DeepLangMultiException.
Validates params' values by: 1.
Validates params' values by: 1. testing whether the params have values set (or default values), 2. testing whether the values meet the constraints, 3. testing custom validations, possibly spanning over multiple params.
Wrapper for creating deeplang Transformers from spark.ml Models. It is parametrized by model and estimator types, because these entities are tightly coupled.
Every SparkModelWrapper should have a no-arg constructor.
type of wrapped ml.Model
type of wrapped ml.Estimator