Class/Object

com.intel.analytics.bigdl.nn

LSTM

Related Docs: object LSTM | package nn

Permalink

class LSTM[T] extends Cell[T]

Long Short Term Memory architecture. Ref. A.: http://arxiv.org/pdf/1303.5778v1 (blueprint for this module) B. http://web.eecs.utk.edu/~itamar/courses/ECE-692/Bobby_paper1.pdf C. http://arxiv.org/pdf/1503.04069v1.pdf D. https://github.com/wojzaremba/lstm

Annotations
@SerialVersionUID()
Linear Supertypes
Cell[T], AbstractModule[Table, Table, T], Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. LSTM
  2. Cell
  3. AbstractModule
  4. Serializable
  5. Serializable
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new LSTM(inputSize: Int, hiddenSize: Int, p: Double = 0, wRegularizer: Regularizer[T] = null, uRegularizer: Regularizer[T] = null, bRegularizer: Regularizer[T] = null)(implicit arg0: ClassTag[T], ev: TensorNumeric[T])

    Permalink

    inputSize

    the size of each input vector

    hiddenSize

    Hidden unit size in the LSTM

    p

    is used for Dropout probability. For more details about RNN dropouts, please refer to [RnnDrop: A Novel Dropout for RNNs in ASR] (http://www.stat.berkeley.edu/~tsmoon/files/Conference/asru2015.pdf) [A Theoretically Grounded Application of Dropout in Recurrent Neural Networks] (https://arxiv.org/pdf/1512.05287.pdf)

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. def accGradParameters(input: Table, gradOutput: Table): Unit

    Permalink

    Computing the gradient of the module with respect to its own parameters.

    Computing the gradient of the module with respect to its own parameters. Many modules do not perform this step as they do not have any parameters. The state variable name for the parameters is module dependent. The module is expected to accumulate the gradients with respect to the parameters in some variable.

    Definition Classes
    CellAbstractModule
  5. def addTimes(other: Cell[T]): Unit

    Permalink
    Definition Classes
    Cell
  6. def apply(name: String): Option[AbstractModule[Activity, Activity, T]]

    Permalink

    Find a module with given name.

    Find a module with given name. If there is no module with given name, it will return None. If there are multiple modules with the given name, an exception will be thrown.

    Definition Classes
    AbstractModule
  7. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  8. var bRegularizer: Regularizer[T]

    Permalink
  9. def backward(input: Table, gradOutput: Table): Table

    Permalink

    Performs a back-propagation step through the module, with respect to the given input.

    Performs a back-propagation step through the module, with respect to the given input. In general this method makes the assumption forward(input) has been called before, with the same input. This is necessary for optimization reasons. If you do not respect this rule, backward() will compute incorrect gradients.

    input

    input data

    gradOutput

    gradient of next layer

    returns

    gradient corresponding to input data

    Definition Classes
    CellAbstractModule
  10. var backwardTime: Long

    Permalink
    Attributes
    protected
    Definition Classes
    AbstractModule
  11. var backwardTimes: Array[Long]

    Permalink
    Definition Classes
    Cell
  12. def buildGates()(input1: ModuleNode[T], input2: ModuleNode[T]): (ModuleNode[T], ModuleNode[T], ModuleNode[T], ModuleNode[T])

    Permalink
  13. def buildLSTM(): Graph[T]

    Permalink
  14. def buildModel(): Sequential[T]

    Permalink
  15. def canEqual(other: Any): Boolean

    Permalink
    Definition Classes
    LSTMAbstractModule
  16. var cell: AbstractModule[Activity, Activity, T]

    Permalink

    Any recurrent kernels should have a cell member variable which represents the module in the kernel.

    Any recurrent kernels should have a cell member variable which represents the module in the kernel.

    The cell receive an input with a format of T(input, preHiddens), and the output should be a format of T(output, hiddens). The hiddens represents the kernel's output hiddens at the current time step, which will be transferred to next time step. For instance, a simple RnnCell, hiddens is h, for LSTM, hiddens is T(h, c), and for both of them, the output variable represents h. Similarly the preHiddens is the kernel's output hiddens at the previous time step.

    Definition Classes
    LSTMCell
  17. var cellLayer: Sequential[T]

    Permalink
  18. def checkEngineType(): LSTM.this.type

    Permalink

    get execution engine type

    get execution engine type

    Definition Classes
    AbstractModule
  19. def clearState(): LSTM.this.type

    Permalink

    Clear cached activities to save storage space or network bandwidth.

    Clear cached activities to save storage space or network bandwidth. Note that we use Tensor.set to keep some information like tensor share

    The subclass should override this method if it allocate some extra resource, and call the super.clearState in the override method

    Definition Classes
    AbstractModule
  20. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  21. def cloneModule(): AbstractModule[Table, Table, T]

    Permalink
    Definition Classes
    AbstractModule
  22. def copyStatus(src: Module[T]): LSTM.this.type

    Permalink

    Copy the useful running status from src to this.

    Copy the useful running status from src to this.

    The subclass should override this method if it has some parameters besides weight and bias. Such as runningMean and runningVar of BatchNormalization.

    src

    source Module

    returns

    this

    Definition Classes
    AbstractModule
  23. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  24. def equals(other: Any): Boolean

    Permalink
    Definition Classes
    LSTMAbstractModule → AnyRef → Any
  25. def evaluate(dataSet: LocalDataSet[MiniBatch[T]], vMethods: Array[ValidationMethod[T]]): Array[(ValidationResult, ValidationMethod[T])]

    Permalink
    Definition Classes
    AbstractModule
  26. def evaluate(dataset: RDD[Sample[T]], vMethods: Array[ValidationMethod[T]], batchSize: Option[Int] = None): Array[(ValidationResult, ValidationMethod[T])]

    Permalink

    use ValidationMethod to evaluate module

    use ValidationMethod to evaluate module

    dataset

    dataset for test

    vMethods

    validation methods

    batchSize

    total batchsize of all partitions, optional param and default 4 * partitionNum of dataset

    Definition Classes
    AbstractModule
  27. def evaluate(): LSTM.this.type

    Permalink
    Definition Classes
    AbstractModule
  28. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  29. final def forward(input: Table): Table

    Permalink

    Takes an input object, and computes the corresponding output of the module.

    Takes an input object, and computes the corresponding output of the module. After a forward, the output state variable should have been updated to the new value.

    input

    input data

    returns

    output data

    Definition Classes
    AbstractModule
  30. var forwardTime: Long

    Permalink
    Attributes
    protected
    Definition Classes
    AbstractModule
  31. var forwardTimes: Array[Long]

    Permalink
    Definition Classes
    Cell
  32. def freeze(names: String*): LSTM.this.type

    Permalink

    freeze the module, i.e.

    freeze the module, i.e. their parameters(weight/bias, if exists) are not changed in training process if names is not empty, set an array of layers that match the given names to be "freezed",

    names

    an array of layer names

    returns

    current graph model

    Definition Classes
    AbstractModule
  33. var gates: Sequential[T]

    Permalink
  34. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  35. def getName(): String

    Permalink

    Get the module name, default name is className@namePostfix

    Get the module name, default name is className@namePostfix

    Definition Classes
    AbstractModule
  36. def getNamePostfix: String

    Permalink
    Definition Classes
    AbstractModule
  37. def getNumericType(): TensorDataType

    Permalink

    returns

    Float or Double

    Definition Classes
    AbstractModule
  38. def getParameters(): (Tensor[T], Tensor[T])

    Permalink

    This method compact all parameters and gradients of the model into two tensors.

    This method compact all parameters and gradients of the model into two tensors. So it's easier to use optim method

    Definition Classes
    AbstractModule
  39. def getParametersTable(): Table

    Permalink

    This function returns a table contains ModuleName, the parameter names and parameter value in this module.

    This function returns a table contains ModuleName, the parameter names and parameter value in this module. The result table is a structure of Table(ModuleName -> Table(ParameterName -> ParameterValue)), and the type is Table[String, Table[String, Tensor[T]]].

    For example, get the weight of a module named conv1: table[Table]("conv1")[Tensor[T]]("weight").

    Custom modules should override this function if they have parameters.

    returns

    Table

    Definition Classes
    CellAbstractModule
  40. def getPrintName(): String

    Permalink
    Attributes
    protected
    Definition Classes
    AbstractModule
  41. def getScaleB(): Double

    Permalink

    Get the scale of gradientBias

    Get the scale of gradientBias

    Definition Classes
    AbstractModule
  42. def getScaleW(): Double

    Permalink

    Get the scale of gradientWeight

    Get the scale of gradientWeight

    Definition Classes
    AbstractModule
  43. def getTimes(): Array[(AbstractModule[_ <: Activity, _ <: Activity, T], Long, Long)]

    Permalink
    Definition Classes
    CellAbstractModule
  44. def getWeightsBias(): Array[Tensor[T]]

    Permalink

    Get weight and bias for the module

    Get weight and bias for the module

    returns

    array of weights and bias

    Definition Classes
    AbstractModule
  45. var gradInput: Table

    Permalink

    The cached gradient of activities.

    The cached gradient of activities. So we don't compute it again when need it

    Definition Classes
    AbstractModule
  46. def hasName: Boolean

    Permalink
    Definition Classes
    AbstractModule
  47. def hashCode(): Int

    Permalink
    Definition Classes
    LSTMAbstractModule → AnyRef → Any
  48. def hidResize(hidden: Activity, batchSize: Int, stepShape: Array[Int]): Activity

    Permalink

    resize the hidden parameters wrt the batch size, hiddens shapes.

    resize the hidden parameters wrt the batch size, hiddens shapes.

    e.g. RnnCell contains 1 hidden parameter (H), thus it will return Tensor(size) LSTM contains 2 hidden parameters (C and H) and will return T(Tensor(), Tensor())\ and recursively intialize all the tensors in the Table.

    batchSize

    batchSize

    stepShape

    For rnn/lstm/gru, it's embedding size. For convlstm/ convlstm3D, it's a list of outputPlane, length, width, height

    Definition Classes
    Cell
  49. val hiddenSize: Int

    Permalink

    Hidden unit size in the LSTM

  50. def hiddenSizeOfPreTopo: Int

    Permalink
    Definition Classes
    LSTMCell
  51. val hiddensShape: Array[Int]

    Permalink

    represents the shape of hiddens which would be transferred to the next recurrent time step.

    represents the shape of hiddens which would be transferred to the next recurrent time step. E.g. For RnnCell, it should be Array(hiddenSize) For LSTM, it should be Array(hiddenSize, hiddenSize) (because each time step a LSTM return two hiddens h and c in order, which have the same size.)

    Definition Classes
    Cell
  52. val inputSize: Int

    Permalink

    the size of each input vector

  53. def inputs(first: (ModuleNode[T], Int), nodesWithIndex: (ModuleNode[T], Int)*): ModuleNode[T]

    Permalink

    Build graph: some other modules point to current module

    Build graph: some other modules point to current module

    first

    distinguish from another inputs when input parameter list is empty

    nodesWithIndex

    upstream module nodes and the output tensor index. The start index is 1.

    returns

    node containing current module

    Definition Classes
    AbstractModule
  54. def inputs(nodes: ModuleNode[T]*): ModuleNode[T]

    Permalink

    Build graph: some other modules point to current module

    Build graph: some other modules point to current module

    nodes

    upstream module nodes

    returns

    node containing current module

    Definition Classes
    AbstractModule
  55. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  56. final def isTraining(): Boolean

    Permalink
    Definition Classes
    AbstractModule
  57. var line: String

    Permalink
    Attributes
    protected
    Definition Classes
    AbstractModule
  58. def loadModelWeights(srcModel: Module[Float], matchAll: Boolean = true): LSTM.this.type

    Permalink

    copy weights from another model, mapping by layer name

    copy weights from another model, mapping by layer name

    srcModel

    model to copy from

    matchAll

    whether to match all layers' weights and bias,

    returns

    current module

    Definition Classes
    AbstractModule
  59. def loadWeights(weightPath: String, matchAll: Boolean = true): LSTM.this.type

    Permalink

    load pretrained weights and bias to current module

    load pretrained weights and bias to current module

    weightPath

    file to store weights and bias

    matchAll

    whether to match all layers' weights and bias, if not, only load existing pretrained weights and bias

    returns

    current module

    Definition Classes
    AbstractModule
  60. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  61. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  62. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  63. var output: Table

    Permalink

    The cached output.

    The cached output. So we don't compute it again when need it

    Definition Classes
    AbstractModule
  64. val p: Double

    Permalink

    is used for Dropout probability.

    is used for Dropout probability. For more details about RNN dropouts, please refer to [RnnDrop: A Novel Dropout for RNNs in ASR] (http://www.stat.berkeley.edu/~tsmoon/files/Conference/asru2015.pdf) [A Theoretically Grounded Application of Dropout in Recurrent Neural Networks] (https://arxiv.org/pdf/1512.05287.pdf)

  65. def parameters(): (Array[Tensor[T]], Array[Tensor[T]])

    Permalink

    This function returns two arrays.

    This function returns two arrays. One for the weights and the other the gradients Custom modules should override this function if they have parameters

    returns

    (Array of weights, Array of grad)

    Definition Classes
    CellAbstractModule
  66. var preTopology: TensorModule[T]

    Permalink

    The preTopology defines operations to pre-process the input when it is not dependent on the time dimension.

    The preTopology defines operations to pre-process the input when it is not dependent on the time dimension. For example, the i2h in SimpleRNN Cell can be calculated before the recurrence since all the input slices are independent.

    This is particular useful to boost the performance of the recurrent layer.

    Please define your own preTopology according to your Cell structure. Please refer to SimpleRNN or LSTM for reference.

    Definition Classes
    LSTMCell
  67. def predict(dataset: RDD[Sample[T]], batchSize: Int = 1, shareBuffer: Boolean = false): RDD[Activity]

    Permalink

    module predict, return the probability distribution

    module predict, return the probability distribution

    dataset

    dataset for prediction

    batchSize

    total batchSize for all partitions. if -1, default is 4 * partitionNumber of datatset

    shareBuffer

    whether to share same memory for each batch predict results

    Definition Classes
    AbstractModule
  68. def predictClass(dataset: RDD[Sample[T]], batchSize: Int = 1): RDD[Int]

    Permalink

    module predict, return the predict label

    module predict, return the predict label

    dataset

    dataset for prediction

    batchSize

    total batchSize for all partitions. if -1, default is 4 * partitionNumber of dataset

    Definition Classes
    AbstractModule
  69. def quantize(): Module[T]

    Permalink
    Definition Classes
    AbstractModule
  70. def regluarized(isRegularized: Boolean): Unit

    Permalink

    Use this method to set the whether the recurrent cell is regularized

    Use this method to set the whether the recurrent cell is regularized

    isRegularized

    whether to be regularized or not

    Definition Classes
    Cell
  71. var regularizers: Array[Regularizer[T]]

    Permalink

    If the subclass has regularizers, it need to put the regularizers into an array and pass the array into the Cell constructor as an argument.

    If the subclass has regularizers, it need to put the regularizers into an array and pass the array into the Cell constructor as an argument. See LSTM as a concrete example.

    Definition Classes
    Cell
  72. def reset(): Unit

    Permalink
    Definition Classes
    LSTMCellAbstractModule
  73. def resetTimes(): Unit

    Permalink
    Definition Classes
    CellAbstractModule
  74. def saveCaffe(prototxtPath: String, modelPath: String, useV2: Boolean = true, overwrite: Boolean = false): LSTM.this.type

    Permalink
    Definition Classes
    AbstractModule
  75. def saveDefinition(path: String, overWrite: Boolean = false): LSTM.this.type

    Permalink

    Save this module definition to path.

    Save this module definition to path.

    path

    path to save module, local file system, HDFS and Amazon S3 is supported. HDFS path should be like "hdfs://[host]:[port]/xxx" Amazon S3 path should be like "s3a://bucket/xxx"

    overWrite

    if overwrite

    returns

    self

    Definition Classes
    AbstractModule
  76. def saveModule(path: String, overWrite: Boolean = false): LSTM.this.type

    Permalink

    Save this module to path with protobuf format

    Save this module to path with protobuf format

    path

    path to save module, local file system, HDFS and Amazon S3 is supported. HDFS path should be like "hdfs://[host]:[port]/xxx" Amazon S3 path should be like "s3a://bucket/xxx"

    overWrite

    if overwrite

    returns

    self

    Definition Classes
    AbstractModule
  77. def saveTF(inputs: Seq[(String, Seq[Int])], path: String, byteOrder: ByteOrder = ByteOrder.LITTLE_ENDIAN, dataFormat: TensorflowDataFormat = TensorflowDataFormat.NHWC): LSTM.this.type

    Permalink
    Definition Classes
    AbstractModule
  78. def saveTorch(path: String, overWrite: Boolean = false): LSTM.this.type

    Permalink
    Definition Classes
    AbstractModule
  79. def saveWeights(path: String, overWrite: Boolean): Unit

    Permalink

    save weights and bias to file

    save weights and bias to file

    path

    file to save

    overWrite

    whether to overwrite or not

    Definition Classes
    AbstractModule
  80. var scaleB: Double

    Permalink
    Attributes
    protected
    Definition Classes
    AbstractModule
  81. var scaleW: Double

    Permalink

    The scale of gradient weight and gradient bias before gradParameters being accumulated.

    The scale of gradient weight and gradient bias before gradParameters being accumulated.

    Attributes
    protected
    Definition Classes
    AbstractModule
  82. def setLine(line: String): LSTM.this.type

    Permalink
    Definition Classes
    AbstractModule
  83. def setName(name: String): LSTM.this.type

    Permalink

    Set the module name

    Set the module name

    Definition Classes
    AbstractModule
  84. def setNamePostfix(namePostfix: String): Unit

    Permalink
    Definition Classes
    AbstractModule
  85. def setScaleB(b: Double): LSTM.this.type

    Permalink

    Set the scale of gradientBias

    Set the scale of gradientBias

    b

    the value of the scale of gradientBias

    returns

    this

    Definition Classes
    AbstractModule
  86. def setScaleW(w: Double): LSTM.this.type

    Permalink

    Set the scale of gradientWeight

    Set the scale of gradientWeight

    w

    the value of the scale of gradientWeight

    returns

    this

    Definition Classes
    AbstractModule
  87. def setWeightsBias(newWeights: Array[Tensor[T]]): LSTM.this.type

    Permalink

    Set weight and bias for the module

    Set weight and bias for the module

    newWeights

    array of weights and bias

    Definition Classes
    AbstractModule
  88. var subModules: Array[AbstractModule[_ <: Activity, _ <: Activity, T]]

    Permalink
    Definition Classes
    Cell
  89. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  90. var times: Array[(AbstractModule[_ <: Activity, _ <: Activity, T], Long, Long)]

    Permalink
    Definition Classes
    Cell
  91. def toGraph(startNodes: ModuleNode[T]*): Graph[T]

    Permalink

    Generate graph module with start nodes

    Generate graph module with start nodes

    Definition Classes
    AbstractModule
  92. def toString(): String

    Permalink
    Definition Classes
    LSTMAbstractModule → AnyRef → Any
  93. var train: Boolean

    Permalink

    Module status.

    Module status. It is useful for modules like dropout/batch normalization

    Attributes
    protected
    Definition Classes
    AbstractModule
  94. def training(): LSTM.this.type

    Permalink
    Definition Classes
    AbstractModule
  95. var uRegularizer: Regularizer[T]

    Permalink
  96. def unFreeze(names: String*): LSTM.this.type

    Permalink

    "unfreeze" module, i.e.

    "unfreeze" module, i.e. make the module parameters(weight/bias, if exists) to be trained(updated) in training process if names is not empty, unfreeze layers that match given names

    names

    array of module names to unFreeze

    Definition Classes
    AbstractModule
  97. def updateGradInput(input: Table, gradOutput: Table): Table

    Permalink

    Computing the gradient of the module with respect to its own input.

    Computing the gradient of the module with respect to its own input. This is returned in gradInput. Also, the gradInput state variable is updated accordingly.

    Definition Classes
    CellAbstractModule
  98. def updateOutput(input: Table): Table

    Permalink

    Computes the output using the current parameter set of the class and input.

    Computes the output using the current parameter set of the class and input. This function returns the result which is stored in the output field.

    Definition Classes
    CellAbstractModule
  99. def updateParameters(learningRate: T): Unit

    Permalink
    Definition Classes
    CellAbstractModule
  100. var wRegularizer: Regularizer[T]

    Permalink
  101. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  102. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  103. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  104. def zeroGradParameters(): Unit

    Permalink

    If the module has parameters, this will zero the accumulation of the gradients with respect to these parameters.

    If the module has parameters, this will zero the accumulation of the gradients with respect to these parameters. Otherwise, it does nothing.

    Definition Classes
    CellAbstractModule

Deprecated Value Members

  1. def save(path: String, overWrite: Boolean = false): LSTM.this.type

    Permalink

    Save this module to path.

    Save this module to path.

    path

    path to save module, local file system, HDFS and Amazon S3 is supported. HDFS path should be like "hdfs://[host]:[port]/xxx" Amazon S3 path should be like "s3a://bucket/xxx"

    overWrite

    if overwrite

    returns

    self

    Definition Classes
    AbstractModule
    Annotations
    @deprecated
    Deprecated

    please use recommended saveModule(path, overWrite)

Inherited from Cell[T]

Inherited from AbstractModule[Table, Table, T]

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped