lamp.nn.TransformerEmbedding
See theTransformerEmbedding companion object
case class TransformerEmbedding(embedding: Embedding, addPositionalEmbedding: Boolean, positionalEmbedding: Constant) extends GenericModule[Variable, Variable]
A module with positional and token embeddings
Token embeddings are lookup embeddings. Positional embeddings are supplied as a constant. They are supposed to come from a fixed unlearned derivation of the positions.
Token and positional embeddings are summed.
Gradients are not computed for positionalEmbedding
Attributes
- Companion
- object
- Graph
-
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass Any
Members list
In this article