TransformerDecoder

lamp.nn.TransformerDecoder$
See theTransformerDecoder companion class

Attributes

Companion
class
Graph
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type

Members list

Type members

Inherited types

type MirroredElemLabels <: Tuple

The names of the product elements

The names of the product elements

Attributes

Inherited from:
Mirror
type MirroredLabel <: String

The name of the type

The name of the type

Attributes

Inherited from:
Mirror

Value members

Concrete methods

def apply[S : Sc](numBlocks: Int, in: Int, attentionHiddenPerHeadDim: Int, attentionNumHeads: Int, mlpHiddenDim: Int, dropout: Double, tOpt: STenOptions, linearized: Boolean, decoderDecoderCausalMask: Boolean, encoderDecoderCausalMask: Boolean): TransformerDecoder

Factory for the decoder module of transformer. Does not include initial embedding or position encoding.

Factory for the decoder module of transformer. Does not include initial embedding or position encoding.

Value parameters

attentionHiddenPerHeadDim

size of hidden attention dimension of each attention head

attentionNumHeads

number of attention heads

dropout

dropout rate

in

input dimension

mlpHiddenDim

size of hidden dimension of the two layer perceptron

numBlocks

number of transformer blocks to create (layers)

tOpt

tensor options

Attributes

Returns

a module

Implicits

Implicits

implicit val load: Load[TransformerDecoder]