Checks whether a suitable NerDL graph is available for the given training dataset, before any computations/training is done. This annotator is useful for custom training cases, where specialized graphs might not be available and we want to check before embeddings are evaluated.

Important: This annotator should be used or positioned before any embedding or NerDLApproach annotators in the pipeline and will process the whole dataset to extract the required graph parameters.

This annotator requires a dataset with at least two columns: one with tokens and one with the labels. In addition, it requires the used embedding annotator in the pipeline to extract the suitable embedding dimension.

For extended examples of usage, see the Examples and the NerDLGraphCheckerTestSpec.

Example

import com.johnsnowlabs.nlp.annotator._
import com.johnsnowlabs.nlp.training.CoNLL
import org.apache.spark.ml.Pipeline

// This CoNLL dataset already includes a sentence, token and label
// column with their respective annotator types. If a custom dataset is used,
// these need to be defined with for example:
val conll = CoNLL()
val trainingData = conll.readDataset(spark, "src/test/resources/conll2003/eng.train")

val embeddings = BertEmbeddings
  .pretrained()
  .setInputCols("sentence", "token")
  .setOutputCol("embeddings")

// Requires the data for NerDLApproach graphs: text, tokens, labels and the embedding model
val nerDLGraphChecker = new NerDLGraphChecker()
  .setInputCols("sentence", "token")
  .setLabelColumn("label")
  .setEmbeddingsModel(embeddings)

val nerTagger = new NerDLApproach()
  .setInputCols("sentence", "token", "embeddings")
  .setLabelColumn("label")
  .setOutputCol("ner")
  .setMaxEpochs(1)
  .setRandomSeed(0)
  .setVerbose(0)

val pipeline = new Pipeline().setStages(
  Array(nerDLGraphChecker, embeddings, nerTagger))

// Will throw an exception if no suitable graph is found
val pipelineModel = pipeline.fit(trainingData)