nlp

Type Members

1. case class Annotation(annotatorType: String, start: Int, end: Int, result: String, metadata: Map[String, String]) extends Product with Serializable

   

   represents annotator's output parts and their details

   represents annotator's output parts and their details

   annotatorType

   the type of annotation

   start

   the index of the first character under this annotation

   end

   the index after the last character under this annotation

   metadata

   associated metadata for this annotation

2. abstract class AnnotatorApproach[M <: Model[M]] extends Estimator[M] with HasInputAnnotationCols with HasOutputAnnotationCol with HasAnnotatorType with DefaultParamsWritable

   

   This class should grow once we start training on datasets and share params For now it stands as a dummy placeholder for future reference

3. abstract class AnnotatorModel[M <: Model[M]] extends Model[M] with ParamsAndFeaturesWritable with HasAnnotatorType with HasInputAnnotationCols with HasOutputAnnotationCol with TransformModelSchema

   

   This trait implements logic that applies nlp using Spark ML Pipeline transformers Should strongly change once UsedDefinedTypes are allowed https://issues.apache.org/jira/browse/SPARK-7768

4. class DocumentAssembler extends Transformer with DefaultParamsWritable with HasAnnotatorType with HasOutputAnnotationCol

   

   Created by saif on 06/07/17.

5. class FeaturesReader[T <: HasFeatures] extends MLReader[T]

   

6. class FeaturesWriter[T] extends MLWriter with HasFeatures

   

7. class Finisher extends Transformer with DefaultParamsWritable

   

8. trait HasAnnotatorType extends AnyRef

   

9. trait HasFeatures extends AnyRef

   

10. trait HasInputAnnotationCols extends Params

    

11. trait HasOutputAnnotationCol extends Params

    

12. trait HasRecursiveFit[M <: Model[M]] extends AnyRef

    

    AnnotatorApproach'es may extend this trait in order to allow RecursivePipelines to include intermediate steps trained PipelineModel's

13. trait HasWordEmbeddings extends AutoCloseable with ParamsAndFeaturesWritable

    

    Base class for models that uses Word Embeddings.

    Base class for models that uses Word Embeddings. This implementation is based on RocksDB so it has a compact RAM usage

    Corresponding Approach have to implement AnnotatorWithWordEmbeddings

14. trait ParamsAndFeaturesReadable[T <: HasFeatures] extends DefaultParamsReadable[T]

    

15. trait ParamsAndFeaturesWritable extends DefaultParamsWritable with Params with HasFeatures

    

16. abstract class RawAnnotator[M <: Model[M]] extends Model[M] with ParamsAndFeaturesWritable with HasAnnotatorType with HasInputAnnotationCols with HasOutputAnnotationCol

    

    Created by jose on 25/01/18.

17. class RecursivePipeline extends Pipeline

    

18. class TokenAssembler extends AnnotatorModel[TokenAssembler]

    

    Created by saif on 06/07/17.

19. trait TransformModelSchema extends AnyRef

    

    Created by jose on 25/01/18.