lamp.nn.TransformerEncoder$
See theTransformerEncoder companion class
object TransformerEncoder
Attributes
- Companion
- class
- Graph
-
- Supertypes
-
trait Producttrait Mirrorclass Objecttrait Matchableclass Any
- Self type
-
TransformerEncoder.type
Members list
Type members
Inherited types
The names of the product elements
The names of the product elements
Attributes
- Inherited from:
- Mirror
The name of the type
The name of the type
Attributes
- Inherited from:
- Mirror
Value members
Concrete methods
def apply[S : Sc](numBlocks: Int, in: Int, attentionHiddenPerHeadDim: Int, attentionNumHeads: Int, mlpHiddenDim: Int, dropout: Double, tOpt: STenOptions, linearized: Boolean, gptOrder: Boolean, causalMask: Boolean): TransformerEncoder
Factory for the encoder module of transformer. Does not include initial embedding or position encoding.
Factory for the encoder module of transformer. Does not include initial embedding or position encoding.
Value parameters
- attentionHiddenPerHeadDim
-
size of hidden attention dimension of each attention head
- attentionNumHeads
-
number of attention heads
- dropout
-
dropout rate
- in
-
input dimension
- mlpHiddenDim
-
size of hidden dimension of the two layer perceptron
- numBlocks
-
number of transformer blocks to create (layers)
- tOpt
-
tensor options
Attributes
- Returns
-
a module
Implicits
Implicits
In this article