Attributes
- Companion
- class
- Graph
-
- Supertypes
-
trait Producttrait Mirrorclass Objecttrait Matchableclass Any
- Self type
-
BertEncoder.type
Members list
Type members
Classlikes
Attributes
- Supertypes
- Self type
Inherited types
The names of the product elements
The names of the product elements
Attributes
- Inherited from:
- Mirror
The name of the type
The name of the type
Attributes
- Inherited from:
- Mirror
Value members
Concrete methods
Factory for the encoder module of Bert
Factory for the encoder module of Bert
Input is (tokens, segments) where tokens and segments are both (batch,num tokens) long tensor.
Value parameters
- attentionHiddenPerHeadDim
-
size of hidden attention dimension of each attention head
- attentionNumHeads
-
number of attention heads
- dropout
-
dropout rate
- embeddingDim
-
input embedding dimension
- maxLength
-
maximum num token length
- mlpHiddenDim
-
size of hidden dimension of the two layer perceptron
- numBlocks
-
number of transformer blocks to create
- out
-
output dimension
- positionEmbedding
-
optional float tensor of size (sequence length, embedding dimension) if missing the absolute positional embeddings from Vaswani et al 2017 is used Following the Bert paper the position embeddings are summed
- tOpt
-
tensor options
- vocabularySize
-
vocabulary size
Attributes
- Returns
-
a module