ai.deepsense.deeplang.doperables.spark.wrappers.transformers
StopWordsRemover
Related Doc:
package transformers
ai.deepsense.deeplang.doperables.spark.wrappers.transformers
Creates a transformed DataFrame based on input DataFrame.
Creates a transformed DataFrame based on input DataFrame.
Should be implemented in subclasses.
Should be implemented in subclasses. For known schema of input DataFrame, infers schema of output DataFrame. If it is not able to do it for some reasons, it returns None.
Can be implemented in a subclass, if access to infer context is required.
Can be implemented in a subclass, if access to infer context is required. For known schema of input DataFrame, infers schema of output DataFrame. If it is not able to do it for some reasons, it returns None.
Determines whether to perform automatic conversion of numeric input column to one-element vector column.
Determines whether to perform automatic conversion of numeric input column
to one-element vector column. Can be overridden in Transformer implementation.
Sequence of params without values for this class, parsed from Json.
Sequence of params without values for this class, parsed from Json. If a name of a parameter is unknown, it's ignored JsNull is treated as empty object. JsNull as a value of a parameter unsets param's value.
Sequence of paramPairs for this class, parsed from Json.
Sequence of paramPairs for this class, parsed from Json. If a name of a parameter is unknown, it's ignored JsNull is treated as empty object. JsNull as value of parameter is ignored.
Json describing values associated to parameters.
Json describing values associated to parameters.
Compares 'this' and 'other' params.
Compares 'this' and 'other' params. Objects are equal when they are of the same class and their parameters have the same values set.
True, if 'this' and 'other' are the same.
Sets param values based on provided json.
Sets param values based on provided json. If a name of a parameter is unknown, it's ignored JsNull is treated as empty object.
When ignoreNulls = false, JsNull as a value of a parameter unsets param's value. When ignoreNulls = true, parameters with JsNull values are ignored.
This method extracts Spark parameters from SparkParamWrappers that are: - declared directly in class which mixes this trait in - declared in values of parameters (i.e.
This method extracts Spark parameters from SparkParamWrappers that are: - declared directly in class which mixes this trait in - declared in values of parameters (i.e. ChoiceParam, MultipleChoiceParam)
Transforms 'inputColumn' and stores the results in 'outputColumn'.
Transforms 'inputColumn' and stores the results in 'outputColumn'. This method should throw an exception when the outputColumn already exists (especially when outputColumn equals inputColumn). The has the same contract as transform() Spark http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.ml.UnaryTransformer's one. If the contract in UnaryTransformer change then this method will have to change, too. Eg. when UnaryTransformer cease to throw on duplicated columns, the contract of this method should be changed accordingly.
Transforms schema to reflect changes that transformSingleColumn(..) would do (including throwing exceptions).
Transforms schema to reflect changes that transformSingleColumn(..) would do (including throwing exceptions).
Validates Params entities that contain dynamic parameters' values.
Validates Params entities that contain dynamic parameters' values. Validation errors are wrapped in DeepLangMultiException.
Validates params' values by: 1.
Validates params' values by: 1. testing whether the params have values set (or default values), 2. testing whether the values meet the constraints, 3. testing custom validations, possibly spanning over multiple params.