com.johnsnowlabs.nlp.annotators.ner.crf
NerCrfApproach
Related Docs:
object NerCrfApproach
| package crf
com.johnsnowlabs.nlp.annotators.ner.crf
c0 params defining decay speed for gradient (Default: 2250000)
CRF based Named Entity Recognition Tagger
CRF based Named Entity Recognition Tagger
Entities to recognize
Entities to recognize
Additional dictionary to use for features
c0 params defining decay speed for gradient
Whether or not to calculate prediction confidence by token, includes in metadata
input annotations columns currently used
L2 regularization coefficient
If Epoch relative improvement less than eps then training is stopped
Maximum number of epochs to train
Maximum number of epochs to train
Minimum number of epochs to train
Minimum number of epochs to train
Features with less weights then this param value will be filtered
Gets annotation column name going to generate
Gets annotation column name going to generate
Random seed
Random seed
Level of verbosity during training
Level of verbosity during training
Whether or not to calculate prediction confidence by token, included in metadata (Default: false)
Input annotator types : DOCUMENT, TOKEN, POS, WORD_EMBEDDINGS
Input annotator types : DOCUMENT, TOKEN, POS, WORD_EMBEDDINGS
columns that contain annotations necessary to run this annotator AnnotatorType is used both as input and output columns if not specified
columns that contain annotations necessary to run this annotator AnnotatorType is used both as input and output columns if not specified
L2 regularization coefficient (Default: 1f)
Column with label per each token
Column with label per each token
If Epoch relative improvement is less than lossEps then training is stopped (Default: 1e-3f)
Maximum number of epochs to train
Maximum number of epochs to train
Minimum number of epochs to train
Minimum number of epochs to train
Features with less weights then this param value will be filtered
Output annotator types : NAMED_ENTITY
Output annotator types : NAMED_ENTITY
Random seed
Random seed
c0 params defining decay speed for gradient
Entities to recognize
Entities to recognize
Additional dictionary to use for features
Additional dictionary to use for features
Whether or not to calculate prediction confidence by token, includes in metadata
Overrides required annotators column if different than default
Overrides required annotators column if different than default
L2 regularization coefficient
Column with label per each token
Column with label per each token
If Epoch relative improvement less than eps then training is stopped
Maximum number of epochs to train
Maximum number of epochs to train
Minimum number of epochs to train
Minimum number of epochs to train
Features with less weights then this param value will be filtered
Overrides annotation column name when transforming
Overrides annotation column name when transforming
Random seed
Random seed
Level of verbosity during training
Level of verbosity during training
Level of verbosity during training
Level of verbosity during training
requirement for pipeline transformation validation.
requirement for pipeline transformation validation. It is called on fit()
required uid for storing annotator to disk
required uid for storing annotator to disk
takes a Dataset and checks to see if all the required annotation types are present.
takes a Dataset and checks to see if all the required annotation types are present.
to be validated
True if all the required types are present, else false
Level of verbosity during training (Default: Verbose.Silent.id)
Level of verbosity during training (Default: Verbose.Silent.id)
A list of (hyper-)parameter keys this annotator can take. Users can set and get the parameter values through setters and getters, respectively.
Required input and expected output annotator types
Algorithm for training a Named Entity Recognition Model
For instantiated/pretrained models, see NerCrfModel.
This Named Entity recognition annotator allows for a generic model to be trained by utilizing a CRF machine learning algorithm. The training data should be a labeled Spark Dataset, e.g. CoNLL 2003 IOB with
Annotationtype columns. The data should have columns of typeDOCUMENT, TOKEN, POS, WORD_EMBEDDINGSand an additional label column of annotator typeNAMED_ENTITY. Excluding the label, this can be done with for exampleOptionally the user can provide an entity dictionary file with setExternalFeatures for better accuracy.
For extended examples of usage, see the Spark NLP Workshop and the NerCrfApproachTestSpec.
Example
NerConverter to further process the results
NerDLApproach for a deep learning based approach