org.deeplearning4j.nn.layers.recurrent

Class LSTMHelpers

java.lang.Object

org.deeplearning4j.nn.layers.recurrent.LSTMHelpers

public class LSTMHelpers
extends Object

RNN tutorial: http://deeplearning4j.org/usingrnns.html READ THIS FIRST if you want to understand what the heck is happening here. Shared code for the standard "forwards" LSTM RNN and the bidirectional LSTM RNN This was extracted from GravesLSTM and refactored into static helper functions. The general reasoning for this was so we only have math in one place, instead of two. Based on Graves: Supervised Sequence Labelling with Recurrent Neural Networks http://www.cs.toronto.edu/~graves/phd.pdf See also for full/vectorized equations (and a comparison to other LSTM variants): Greff et al. 2015, "LSTM: A Search Space Odyssey", pg11. This is the "vanilla" variant in said paper http://arxiv.org/pdf/1503.04069.pdf Please note that truncated backpropagation through time (BPTT) will not work with the bidirectional layer as-is. Additionally, variable length data sets will also not work with the bidirectional layer.

Author:

Alex Black (LSTM implementation), Benjamin Joseph (refactoring for bidirectional LSTM)

Field Summary

Fields

Modifier and Type	Field and Description
static String	SIGMOID

Method Summary

Modifier and Type	Method and Description
static FwdPassReturn	activateHelper(Layer layer, NeuralNetConfiguration conf, org.nd4j.linalg.api.ndarray.INDArray input, org.nd4j.linalg.api.ndarray.INDArray recurrentWeights, org.nd4j.linalg.api.ndarray.INDArray originalInputWeights, org.nd4j.linalg.api.ndarray.INDArray biases, boolean training, org.nd4j.linalg.api.ndarray.INDArray originalPrevOutputActivations, org.nd4j.linalg.api.ndarray.INDArray originalPrevMemCellState, boolean forBackprop, boolean forwards, String inputWeightKey) Returns FwdPassReturn object with activations/INDArrays.
static Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray>	backpropGradientHelper(NeuralNetConfiguration conf, org.nd4j.linalg.api.ndarray.INDArray input, org.nd4j.linalg.api.ndarray.INDArray recurrentWeights, org.nd4j.linalg.api.ndarray.INDArray inputWeights, org.nd4j.linalg.api.ndarray.INDArray epsilon, boolean truncatedBPTT, int tbpttBackwardLength, FwdPassReturn fwdPass, boolean forwards, String inputWeightKey, String recurrentWeightKey, String biasWeightKey, Map<String,org.nd4j.linalg.api.ndarray.INDArray> gradientViews)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SIGMOID

public static final String SIGMOID

See Also:

Constant Field Values

Method Detail

activateHelper

public static FwdPassReturn activateHelper(Layer layer,
                                           NeuralNetConfiguration conf,
                                           org.nd4j.linalg.api.ndarray.INDArray input,
                                           org.nd4j.linalg.api.ndarray.INDArray recurrentWeights,
                                           org.nd4j.linalg.api.ndarray.INDArray originalInputWeights,
                                           org.nd4j.linalg.api.ndarray.INDArray biases,
                                           boolean training,
                                           org.nd4j.linalg.api.ndarray.INDArray originalPrevOutputActivations,
                                           org.nd4j.linalg.api.ndarray.INDArray originalPrevMemCellState,
                                           boolean forBackprop,
                                           boolean forwards,
                                           String inputWeightKey)

Returns FwdPassReturn object with activations/INDArrays. Allows activateHelper to be used for forward pass, backward pass and rnnTimeStep whilst being reasonably efficient for all

backpropGradientHelper

public static Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> backpropGradientHelper(NeuralNetConfiguration conf,
                                                                                         org.nd4j.linalg.api.ndarray.INDArray input,
                                                                                         org.nd4j.linalg.api.ndarray.INDArray recurrentWeights,
                                                                                         org.nd4j.linalg.api.ndarray.INDArray inputWeights,
                                                                                         org.nd4j.linalg.api.ndarray.INDArray epsilon,
                                                                                         boolean truncatedBPTT,
                                                                                         int tbpttBackwardLength,
                                                                                         FwdPassReturn fwdPass,
                                                                                         boolean forwards,
                                                                                         String inputWeightKey,
                                                                                         String recurrentWeightKey,
                                                                                         String biasWeightKey,
                                                                                         Map<String,org.nd4j.linalg.api.ndarray.INDArray> gradientViews)