XLNet Model wrapper with TensorFlow Wrapper
XLNet SentencePiece model with SentencePieceWrapper
Configuration for TensorFlow session
labels which model was trained with in order
TF v2 signatures in Spark NLP
Calcuate sigmoid from returned logits
Calcuate sigmoid from returned logits
logits output from output layer
Calcuate softmax from retruned logits
Calcuate softmax from retruned logits
logits output from output layer
Encode the input sequence to indexes IDs adding padding where necessary
Encode the input sequence to indexes IDs adding padding where necessary
XLNet SentencePiece model with SentencePieceWrapper
XLNet Model wrapper with TensorFlow Wrapper
Word-level and span-level alignment with Tokenizer https://github.com/google-research/bert#tokenization
Word-level and span-level alignment with Tokenizer https://github.com/google-research/bert#tokenization
### Input orig_tokens = ["John", "Johanson", "'s", "house"] labels = ["NNP", "NNP", "POS", "NN"]
# bert_tokens == ["[CLS]", "john", "johan", "##son", "'", "s", "house", "[SEP]"] # orig_to_tok_map == [1, 2, 4, 6]