object BertEncoder
- Companion:
- class
Type members
Classlikes
Value members
Concrete methods
def apply[S : Sc](maxLength: Int, vocabularySize: Int, segmentVocabularySize: Int, numBlocks: Int, embeddingDim: Int, attentionHiddenPerHeadDim: Int, attentionNumHeads: Int, mlpHiddenDim: Int, dropout: Double, padToken: Long, tOpt: STenOptions, linearized: Boolean): BertEncoder
Factory for the encoder module of Bert
Factory for the encoder module of Bert
Input is (tokens, segments) where tokens and segments are both
(batch,num tokens) long tensor.
- Value parameters:
- attentionHiddenPerHeadDim
size of hidden attention dimension of each attention head
- attentionNumHeads
number of attention heads
- dropout
dropout rate
- embeddingDim
input embedding dimension
- maxLength
maximum num token length
- mlpHiddenDim
size of hidden dimension of the two layer perceptron
- numBlocks
number of transformer blocks to create
- out
output dimension
- padToken
pad token, (batch, seq) positions where
tokens==padTokenare ignored, padding is not the same as masking- tOpt
tensor options
- vocabularySize
vocabulary size
- Returns:
a module