lamp-core/lamp.nn/Attention

Attention

object Attention
class Object
trait Matchable
class Any

Value members

Concrete methods

def dotProductAttention[S : Sc](query: Variable, keyvalue: Variable, tokens: Variable, padToken: Long): Variable

Dot product attention

Dot product attention

Value parameters:
key

num keys x batch x d

query

batch x d

Returns:

batch x d

def forward[T, M <: StatefulModule[Variable, Variable, T], S : Sc](decoder: M & StatefulModule[Variable, Variable, T], x: Variable, keyValue: Variable, state: T, tokens: Variable, padToken: Long)(stateToKey: T => Variable): (Variable, T)
def sequenceMask[S : Sc](tokens: Variable, maskable: Variable, maskedToken: Long, fill: Double): Variable
Value parameters:
maskable

batch x seq

tokens

seq x batch (long)

Returns:

batch x seq where (seq,batch,:) is set to fill if tokens(seq,batch)== maskedToken