pipeline

Type Members

1. abstract class AbstractTransformer extends Transformer with HasAutoMlIdColumn with HasDebug with HasPipelineId

   

2. class AutoMlOutputDatasetTransformer extends AbstractTransformer with DefaultParamsWritable with HasLabelColumn with HasFeaturesColumns

   

3. class CardinalityLimitColumnPrunerTransformer extends AbstractTransformer with DefaultParamsWritable with HasLabelColumn with HasTransformCalculated

   

4. class ColumnNameTransformer extends Transformer with DefaultParamsWritable with HasInputCols with HasOutputCols with HasDebug with HasPipelineId

   

5. class CovarianceFilterTransformer extends AbstractTransformer with DefaultParamsWritable with HasLabelColumn with HasFeaturesColumns with HasFieldsRemoved with HasTransformCalculated with HasFeatureColumn

   

6. class DataSanitizerTransformer extends AbstractTransformer with DefaultParamsWritable with HasLabelColumn with HasFeatureColumn

   

7. class DatasetsUnionTransformer extends AbstractTransformer with DefaultParamsWritable

   

8. class DateFieldTransformer extends AbstractTransformer with DefaultParamsWritable with DataValidation with HasLabelColumn

   

9. class DropColumnsTransformer extends AbstractTransformer with DefaultParamsWritable with HasInputCols

   

10. class DropTempTableTransformer extends AbstractTransformer with DefaultParamsWritable with WithNoopsStage

    

11. final case class FeatureEngineeringOutput(pipelineModel: PipelineModel, originalDfViewName: String, decidedModel: String, transformedForTrainingDf: DataFrame) extends Product with Serializable

    

12. class FeaturePipeline extends DataValidation

    

13. trait HasAutoMlIdColumn extends Params

    

14. trait HasDebug extends Params

    

    Base trait for setting/accessing debug flags.

    Base trait for setting/accessing debug flags. Meant to be extended by all pipeline stages, which inherit pipeline stage logging by default

15. trait HasFeatureColumn extends Params

    

16. trait HasFeaturesColumns extends Params

    

17. trait HasFieldsRemoved extends Params

    

18. trait HasFieldsToIgnore extends Params

    

19. trait HasInteractionColumns extends Params

    

    Trait for defining whether interaction columns have been set for the application of Feature Interactions

    Trait for defining whether interaction columns have been set for the application of Feature Interactions

    Since

    0.6.2

20. trait HasLabelColumn extends Params

    

21. trait HasPipelineId extends Params

    
    Since

    0.6.1 trait for decorating all pipeline stages with pipeline ID. Helpful when troubleshooting logs with a given pipeline ID (eg, fetched from MLflow)

22. trait HasTransformCalculated extends Params

    

23. class InteractionTransformer extends AbstractTransformer with DefaultParamsWritable with HasInteractionColumns

    

    Transformer for creating interacted feature fields based on FeatureInteraction module

    Transformer for creating interacted feature fields based on FeatureInteraction module

    Since

    0.6.2

24. trait IsTrainingStage extends AnyRef

    

25. class MlFlowLoggingValidationStageTransformer extends AbstractTransformer with DefaultParamsWritable with WithNoopsStage

    

26. class OutlierFilterTransformer extends AbstractTransformer with DefaultParamsWritable with HasLabelColumn with HasFieldsToIgnore with IsTrainingStage

    

27. class PearsonFilterTransformer extends AbstractTransformer with DefaultParamsWritable with HasLabelColumn with HasFeatureColumn with HasFeaturesColumns with HasFieldsRemoved with HasTransformCalculated

    

28. class RegisterTempTableTransformer extends AbstractTransformer with DefaultParamsWritable

    

29. class RepartitionTransformer extends AbstractTransformer with WithNoopsStage with DefaultParamsWritable with HasDebug

    

    A WithNoopsStage transformer stage that is helpful to repartition a DataFrame coming out of any pipeline stage

30. class RoundUpDoubleTransformer extends AbstractTransformer with DefaultParamsWritable with HasTransformCalculated with HasInputCols

    

31. class SQLWrapperTransformer extends AbstractTransformer with DefaultParamsWritable

    

32. class SyntheticFeatureGenTransformer extends AbstractTransformer with HasLabelColumn with HasFeatureColumn with HasFieldsToIgnore with DefaultParamsWritable with IsTrainingStage

    

33. class VarianceFilterTransformer extends AbstractTransformer with DefaultParamsWritable with HasLabelColumn with HasFeatureColumn with HasTransformCalculated

    

34. final case class VectorizationOutput(pipelineModel: PipelineModel, vectorizedCols: Array[String]) extends Product with Serializable

    

35. trait WithNoopsStage extends AnyRef

    

    Marker interface to signify any transformer extending this trait will not alter an input dataset.

    Marker interface to signify any transformer extending this trait will not alter an input dataset. This is only for the edge cases where it is required to do an external Ops before pipeline execution can continue. An example would be to do Mlflow params Validation before training continues. Helpful in scenarios where fail-fast feature is needed Example transformers are DropTempTableTransformer, MlFlowLoggingValidationStageTransformer.

    NOTE: Noops implies no changes to the input Dataset, but the implementation can result in a change to an external state

36. class ZipRegisterTempTransformer extends AbstractTransformer with DefaultParamsWritable with HasLabelColumn with HasFeaturesColumns