BertPretrainOutput

Output of BERT

• encoded: float tensor of size (batch, sequence length, embedding dimension ) holds per token embeddings
• languageModelScores: float tensor of size (batch, sequence length, vocabulary size) holds per token log probability distributions (from logSoftMax)
• wholeSentenceBinaryClassifierScore: float tensor of size (batch) holds the output score of the whole sentence prediction task suitable for BCELogitLoss