Attributes
- Companion
- class
- Graph
-
- Supertypes
-
trait Producttrait Mirrorclass Objecttrait Matchableclass Any
- Self type
-
LanguageModelLoss.type
Members list
Type members
Inherited types
The names of the product elements
The names of the product elements
Attributes
- Inherited from:
- Mirror
The name of the type
The name of the type
Attributes
- Inherited from:
- Mirror
Value members
Concrete methods
Allocate language model module with negative log likelihood loss
Allocate language model module with negative log likelihood loss
Value parameters
- attentionHiddenPerHeadDim
-
Per head hidden dimension in the multihead attention
- attentionNumHeads
-
Number of attention heads in the multihead attention
- embeddingDim
-
Width of the initial embedding dimension, as well as the output width of each transformer block
- encoderMlpHiddenDim
-
Hidden dimension within transformer blocks
- linearized
-
Whether to use linearized self attention
- maxLength
-
Total sequence length including padding if used. Sometimes called block length or context length.
- numBlocks
-
Number of transformer blocks (layers).
- padToken
-
This token is ignored during loss computation. Not used otherwise.
- tOpt
-
TensorOption to set device and data type
- vocabularySize
-
Total vocabulary size.