Base type of modules
Modules are functions of type (Seq[lamp.autograd.Constant],A) => B
, where
the Seq[lamp.autograd.Constant]
arguments are optimizable parameters and
A
is a non-optimizable input.
Modules provide a way to build composite functions while also keep track of the parameter list of the composite function.
===Example===
case object Weights extends LeafTag
case object Bias extends LeafTag
case class Linear(weights: Constant, bias: Option[Constant]) extends Module {
override val state = List(
weights -> Weights
) ++ bias.toList.map(b => (b, Bias))
def forward[S: Sc](x: Variable): Variable = {
val v = x.mm(weights)
bias.map(_ + v).getOrElse(v)
}
}
Some other attributes of modules are attached by type classes e.g. with the nn.TrainingMode, nn.Load type classes.
class Object
trait Matchable
class Any
class MultiheadAttention
class TransformerEmbedding
class TransformerEncoder
class TransformerEncoderBlock
class BertEncoder
class BertLoss
class BertPretrainModule
class GraphAttention
Value members
Abstract methods
Concrete methods
Computes the gradient of loss with respect to the parameters.
Computes the gradient of loss with respect to the parameters.
Returns the total number of optimizable parameters.
Returns the total number of optimizable parameters.