Symbolic

Provides @Symbolic annotation to create symbolic methods, in which you can create Layers from mathematical formulas.

Symbolic is a dependent type class that calculates a specific Layer type according to NativeOutput. Combining with implicit-dependent-type compiler plugin, it can be treated as a type annotation in the form of NativeOutput @Symbolic, converting NativeOutput to a specific Layer type.

Three usages of @Symbolic

Implicit parameter types used for symbol methods

In case that the implicit parameter of an method is marked with @Symbolic, then this method is symbol method. The implicit parameter type marked with @Symbolic is the input type of this symbol method. In this case, NativeOutput @Symbolic will be expanded as:

Identity[NativeOutput, Derivative type of NativeOutput]

For example:

def sumNetwork(implicit scores: INDArray @Symbolic): Double = {
exp(scores).sum
}

In the above code, because the derivative type of INDArray is also INDArray, the input type INDArray @Symbolic of sumNetwork, once expanded, is Identity[INDArray, INDArray]

Used for symbol method internal variable and return value

A NativeOutput @Symbolic inside a symbol method, or at the return position of a symbol method, will be expanded as:

Layer.Aux[Tape.Aux[value type of input type, derivative type of input type], Tape.Aux[NativeOutput, derivative type of NativeOutput]]

For example:

def sumNetwork(implicit scores: INDArray @Symbolic): Double @Symbolic = {
val expScores: INDArray @Symbolic = exp(scores)
val result: Double @Symbolic = expScores.sum
result
}

In the above code, the type INDArray @Symbolic of expScores is expanded as:

Layer.Aux[Tape.Aux[INDArray, INDArray], Tape.Aux[INDArray, INDArray]]

The type Double @Symbolic of result is expanded as:

Layer.Aux[Tape.Aux[INDArray, INDArray], Tape.Aux[Double, Double]]

Used for cases excluding symbol method

(NativeInput => NativeOutput) @Symbolic outside a symbol method, will be expanded as:

Layer.Aux[Tape.Aux[NativeInput, derivative type of NativeInput], Tape.Aux[NativeOutput, derivative type of NativeOutput]]

For example:

val predictor: (INDArray => Double) @Symbolic = sumNetwork

In the above code, type (INDArray => Double) @Symbolic of predictor is expanded as:

Layer.Aux[Tape.Aux[INDArray, INDArray], Tape.Aux[Double, Double]]

Custom symbol type

The @Symbolic determines the mapping relation between the primitive type and derivative by checking Symbolic.ToLiteral implicit value. Therefore, @Symbolic can be a custom symbol type once you define your own the implicit Symbolic.ToLiteral.

For example, if you want to support Short @Symbolic, using Float as the derivative type of Short, then you can conduct the follows:

implicit object ShortToLiteral extends ToLiteral[Short] {
  override type Data = Short
  override type Delta = Float
  override def apply(data: Short) = Literal(data)
}

def makeShortNetwork(implicit input: Short @Symbolic): Short @Symbolic = {
  input
}

val shortNetwork: (Short => Short) @Symbolic = makeShortNetwork

Thus, type of shortNetwork is expanded as:

Layer.Aux[Tape.Aux[Short, Float], Tape.Aux[Short, Float]]