lamp.nn.bert.BertLoss
See theBertLoss companion class
object BertLoss
Attributes
Members list
Type members
Inherited types
The names of the product elements
The names of the product elements
Attributes
- Inherited from:
- Mirror
The name of the type
The name of the type
Attributes
- Inherited from:
- Mirror
Value members
Concrete methods
def apply[S : Sc](maxLength: Int, vocabularySize: Int, segmentVocabularySize: Int, mlmHiddenDim: Int, wholeStentenceHiddenDim: Int, numBlocks: Int, embeddingDim: Int, attentionHiddenPerHeadDim: Int, attentionNumHeads: Int, bertEncoderMlpHiddenDim: Int, dropout: Double, padToken: Long, tOpt: STenOptions, linearized: Boolean, positionEmbedding: Option[STen]): BertLoss
Allocate Bert module
Allocate Bert module
Value parameters
- bertEncoderMlpHiddenDim
-
Hidden dimension within transformer blocks
- embeddingDim
-
Width of the initial embedding dimension, as well as the output width of the feed forward network in each transformer block
- linearized
-
Whether to use linearized self attention
- maxLength
-
Total sequence length including cls, sep and potential pad tokens
- mlmHiddenDim
-
Hidden dimension of the masked language model decoder
- numBlocks
-
Number of transformer blocks
- padToken
-
pad will be ignored
- segmentVocabularySize
-
Vocabulary size of the segment features
- vocabularySize
-
Total vocabulary size including cls, sep, pad, mask tokens
- wholeStentenceHiddenDim
-
Hidden dimension of the whole sentence task decoder
Attributes
Implicits
In this article