Package org.nd4j.linalg.api.ops.impl.transforms.custom

Class LayerNorm

java.lang.Object

org.nd4j.autodiff.functions.DifferentialFunction

org.nd4j.linalg.api.ops.DynamicCustomOp

org.nd4j.linalg.api.ops.impl.transforms.custom.LayerNorm

All Implemented Interfaces:

CustomOp

public class LayerNorm
extends DynamicCustomOp

Nested Class Summary

Nested classes/interfaces inherited from class org.nd4j.linalg.api.ops.DynamicCustomOp

DynamicCustomOp.DynamicCustomOpsBuilder

Field Summary

Fields inherited from class org.nd4j.linalg.api.ops.DynamicCustomOp

axis, bArguments, dArguments, iArguments, inplaceCall, inputArguments, outputArguments, outputVariables, sArguments, tArguments

Fields inherited from class org.nd4j.autodiff.functions.DifferentialFunction

dimensions, extraArgs, inPlace, ownName, ownNameSetWithDefault, sameDiff, scalarValue

Constructor Summary

Constructors

Constructor	Description
LayerNorm(@NonNull SameDiff sameDiff, @NonNull SDVariable input, @NonNull SDVariable gain, SDVariable bias, boolean channelsFirst, int... dimensions)
LayerNorm(@NonNull INDArray input, @NonNull INDArray gain, boolean channelsFirst, int... dimensions)
LayerNorm(SameDiff sameDiff, SDVariable input, SDVariable gain, boolean channelsFirst, int... dimensions)
LayerNorm(INDArray input, INDArray gain, INDArray result, boolean channelsFirst, int... dimensions)
LayerNorm(INDArray input, INDArray gain, INDArray bias, INDArray result, boolean channelsFirst, int... dimensions)

Method Summary

Modifier and Type	Method	Description
void	addBArgument(boolean... arg)
List<DataType>	calculateOutputDataTypes(List<DataType> dataTypes)	Calculate the data types for the output arrays.
void	configureFromArguments()	This allows a custom op to configure relevant fields from its arguments.
List<SDVariable>	doDiff(List<SDVariable> gradient)	The actual implementation for automatic differentiation.
int	numOutputArguments()
String	onnxName()	The opName of this function in onnx
String	opName()	This method returns op opName as string
Map<String,Object>	propertiesForFunction()	Returns the properties for a given function
void	setDimensions(int[] dimensions)
void	setPropertiesForFunction(Map<String,Object> properties)
String	tensorflowName()	The opName of this function tensorflow

Methods inherited from class org.nd4j.linalg.api.ops.DynamicCustomOp

addDArgument, addIArgument, addIArgument, addInputArgument, addOutputArgument, addOutputsToOp, addSArgument, addTArgument, assertValidForExecution, bArgs, builder, calculateOutputShape, calculateOutputShape, clearArrays, computeArrays, dArgs, generateFake, generateFake, getBArgument, getDescriptor, getIArgument, getInputArgument, getOutputArgument, getSArgument, getTArgument, getValue, iArgs, initFromOnnx, initFromTensorFlow, inputArguments, mappingsForFunction, numBArguments, numDArguments, numIArguments, numInputArguments, numSArguments, numTArguments, opHash, opNum, opType, outputArguments, outputVariables, outputVariables, removeIArgument, removeInputArgument, removeOutputArgument, removeSArgument, removeTArgument, sArgs, setInputArgument, setInputArguments, setOutputArgument, setValueFor, tArgs, toString, wrapFilterNull, wrapOrNull, wrapOrNull

Methods inherited from class org.nd4j.autodiff.functions.DifferentialFunction

arg, arg, argNames, args, attributeAdaptersForFunction, configFieldName, configureWithSameDiff, diff, dup, equals, getBooleanFromProperty, getDoubleValueFromProperty, getIntValueFromProperty, getLongValueFromProperty, getNumOutputs, getStringFromProperty, hashCode, isConfigProperties, larg, onnxNames, outputs, outputVariable, outputVariablesNames, rarg, replaceArg, setInstanceId, tensorflowNames

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.nd4j.linalg.api.ops.CustomOp

isInplaceCall

Constructor Detail

LayerNorm

public LayerNorm(@NonNull
                 @NonNull SameDiff sameDiff,
                 @NonNull
                 @NonNull SDVariable input,
                 @NonNull
                 @NonNull SDVariable gain,
                 SDVariable bias,
                 boolean channelsFirst,
                 int... dimensions)

LayerNorm

public LayerNorm(SameDiff sameDiff,
                 SDVariable input,
                 SDVariable gain,
                 boolean channelsFirst,
                 int... dimensions)

LayerNorm

public LayerNorm(INDArray input,
                 INDArray gain,
                 INDArray bias,
                 INDArray result,
                 boolean channelsFirst,
                 int... dimensions)

LayerNorm

public LayerNorm(@NonNull
                 @NonNull INDArray input,
                 @NonNull
                 @NonNull INDArray gain,
                 boolean channelsFirst,
                 int... dimensions)

LayerNorm

public LayerNorm(INDArray input,
                 INDArray gain,
                 INDArray result,
                 boolean channelsFirst,
                 int... dimensions)

Method Detail

setDimensions

public void setDimensions(int[] dimensions)

addBArgument

public void addBArgument(boolean... arg)

Specified by:

addBArgument in interface CustomOp

Overrides:

addBArgument in class DynamicCustomOp

propertiesForFunction

public Map<String,Object> propertiesForFunction()

Description copied from class: DifferentialFunction

Returns the properties for a given function

Overrides:

propertiesForFunction in class DynamicCustomOp

Returns:

configureFromArguments

public void configureFromArguments()

Description copied from interface: CustomOp

This allows a custom op to configure relevant fields from its arguments. This is needed when ops are created via reflection for things like model import.

Specified by:

configureFromArguments in interface CustomOp

Overrides:

configureFromArguments in class DynamicCustomOp

setPropertiesForFunction

public void setPropertiesForFunction(Map<String,Object> properties)

Overrides:

setPropertiesForFunction in class DynamicCustomOp

opName

public String opName()

Description copied from class: DynamicCustomOp

This method returns op opName as string

Specified by:

opName in interface CustomOp

Overrides:

opName in class DynamicCustomOp

Returns:

tensorflowName

public String tensorflowName()

Description copied from class: DifferentialFunction

The opName of this function tensorflow

Overrides:

tensorflowName in class DynamicCustomOp

Returns:

onnxName

public String onnxName()

Description copied from class: DifferentialFunction

The opName of this function in onnx

Overrides:

onnxName in class DynamicCustomOp

Returns:

doDiff

public List<SDVariable> doDiff(List<SDVariable> gradient)

Description copied from class: DifferentialFunction

The actual implementation for automatic differentiation.

Overrides:

doDiff in class DynamicCustomOp

Returns:

calculateOutputDataTypes

public List<DataType> calculateOutputDataTypes(List<DataType> dataTypes)

Description copied from class: DifferentialFunction

Calculate the data types for the output arrays. Though datatypes can also be inferred from DifferentialFunction.calculateOutputShape(), this method differs in that it does not require the input arrays to be populated. This is important as it allows us to do greedy datatype inference for the entire net - even if arrays are not available.

Overrides:

calculateOutputDataTypes in class DifferentialFunction

Parameters:

dataTypes - The data types of the inputs

Returns:

The data types of the outputs

numOutputArguments

public int numOutputArguments()

Specified by:

numOutputArguments in interface CustomOp

Overrides:

numOutputArguments in class DynamicCustomOp