org.deeplearning4j.nn.layers.feedforward.embedding

Class EmbeddingLayer

java.lang.Object

org.deeplearning4j.nn.layers.AbstractLayer<LayerConfT>

org.deeplearning4j.nn.layers.BaseLayer<EmbeddingLayer>

org.deeplearning4j.nn.layers.feedforward.embedding.EmbeddingLayer

All Implemented Interfaces:

Serializable, Cloneable, Layer, Model

public class EmbeddingLayer
extends BaseLayer<EmbeddingLayer>

Embedding layer: feed-forward layer that expects single integers per example as input (class numbers, in range 0 to numClass-1) as input. This input has shape [numExamples,1] instead of [numExamples,numClasses] for the equivalent one-hot representation. Mathematically, EmbeddingLayer is equivalent to using a DenseLayer with a one-hot representation for the input; however, it can be much more efficient with a large number of classes (as a dense layer + one-hot input does a matrix multiply with all but one value being zero).
Note: can only be used as the first layer for a network
Note 2: For a given example index i, the output is activationFunction(weights.getRow(i) + bias), hence the weight rows can be considered a vector/embedding for each example.

Author:

Alex Black

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.deeplearning4j.nn.api.Layer

Layer.TrainingMode, Layer.Type

Field Summary

Fields inherited from class org.deeplearning4j.nn.layers.BaseLayer

gradient, gradientsFlattened, gradientViews, optimizer, params, paramsFlattened, score, solver

Fields inherited from class org.deeplearning4j.nn.layers.AbstractLayer

cacheMode, conf, dropoutApplied, dropoutMask, index, input, iterationListeners, maskArray, maskState, preOutput

Constructor Summary

Constructors

Constructor and Description
EmbeddingLayer(NeuralNetConfiguration conf)

Method Summary

Modifier and Type	Method and Description
org.nd4j.linalg.api.ndarray.INDArray	activate(boolean training) Trigger an activation with the last specified input
protected void	applyDropOutIfNecessary(boolean training)
org.nd4j.linalg.primitives.Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray>	backpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon) Calculate the gradient relative to the error in the next layer
boolean	isPretrainLayer() Returns true if the layer can be trained in an unsupervised/pretrain manner (VAE, RBMs etc)
org.nd4j.linalg.api.ndarray.INDArray	preOutput(boolean training)

Methods inherited from class org.deeplearning4j.nn.layers.BaseLayer

accumulateScore, activate, activate, activationMean, applyLearningRateScoreDecay, calcGradient, calcL1, calcL2, clone, computeGradientAndScore, error, fit, fit, getGradientsViewArray, getOptimizer, getParam, gradient, initParams, iterate, layerConf, merge, numParams, params, paramTable, paramTable, preOutput, score, setBackpropGradientsViewArray, setParam, setParams, setParams, setParamsViewArray, setParamTable, setScoreWithZ, toString, transpose, update, update

Methods inherited from class org.deeplearning4j.nn.layers.AbstractLayer

activate, activate, activate, addListeners, applyMask, batchSize, clear, conf, derivativeActivation, feedForwardMaskArray, getIndex, getInput, getInputMiniBatchSize, getListeners, getMaskArray, gradientAndScore, init, input, layerId, numParams, preOutput, preOutput, setCacheMode, setConf, setIndex, setInput, setInputMiniBatchSize, setListeners, setListeners, setMaskArray, type, validateInput

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

EmbeddingLayer

public EmbeddingLayer(NeuralNetConfiguration conf)

Method Detail

backpropGradient

public org.nd4j.linalg.primitives.Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> backpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon)

Description copied from interface: Layer

Calculate the gradient relative to the error in the next layer

Specified by:

backpropGradient in interface Layer

Overrides:

backpropGradient in class BaseLayer<EmbeddingLayer>

Parameters:

epsilon - w^(L+1)*delta^(L+1). Or, equiv: dC/da, i.e., (dC/dz)*(dz/da) = dC/da, where C is cost function a=sigma(z) is activation.

Returns:

Pair where Gradient is gradient for this layer, INDArray is epsilon needed by next layer, but before element-wise multiply by sigmaPrime(z). So for standard feed-forward layer, if this layer is L, then return.getSecond() == (w^(L)*(delta^(L))^T)^T

preOutput

public org.nd4j.linalg.api.ndarray.INDArray preOutput(boolean training)

Overrides:

preOutput in class BaseLayer<EmbeddingLayer>

activate

public org.nd4j.linalg.api.ndarray.INDArray activate(boolean training)

Description copied from interface: Layer

Trigger an activation with the last specified input

Specified by:

activate in interface Layer

Overrides:

activate in class BaseLayer<EmbeddingLayer>

Parameters:

training - training or test mode

Returns:

the activation of the last specified input

isPretrainLayer

public boolean isPretrainLayer()

Description copied from interface: Layer

Returns true if the layer can be trained in an unsupervised/pretrain manner (VAE, RBMs etc)

Returns:

true if the layer can be pretrained (using fit(INDArray), false otherwise

applyDropOutIfNecessary

protected void applyDropOutIfNecessary(boolean training)

Overrides:

applyDropOutIfNecessary in class AbstractLayer<EmbeddingLayer>