Attention
object Attention
Value members
Concrete methods
def dotProductAttention[S : Sc](query: Variable, keyvalue: Variable, tokens: Variable, padToken: Long): Variable
Dot product attention
Dot product attention
- Value parameters:
- key
num keys x batch x d
- query
batch x d
- Returns:
batch x d
def sequenceMask[S : Sc](tokens: Variable, maskable: Variable, maskedToken: Long, fill: Double): Variable
- Value parameters:
- maskable
batch x seq
- tokens
seq x batch (long)
- Returns:
batch x seq where (seq,batch,:) is set to fill if tokens(seq,batch)== maskedToken