p

lamp

data

package data

Ordering
  1. Alphabetic
Visibility
  1. Public
  2. Protected

Package Members

  1. package bert
  2. package schemas

Type Members

  1. trait BatchStream[+I, S] extends AnyRef
  2. sealed trait LoopState extends AnyRef
  3. case class NonEmptyBatch[I](batch: I) extends StreamControl[I] with Product with Serializable
  4. case class Peek(label: String) extends Module with Product with Serializable
  5. case class SWALoopState(model: Seq[STen], optimizer: Seq[STen], epoch: Int, lastValidationLoss: Option[Double], minValidationLoss: Option[Double], numberOfAveragedModels: Int, averagedModels: Option[Seq[Tensor]], learningCurve: List[(Int, Double, Option[Double])]) extends LoopState with Product with Serializable
  6. case class SimpleLoopState(model: Seq[STen], optimizer: Seq[STen], epoch: Int, lastValidationLoss: Option[Double], minValidationLoss: Option[Double], minValidationLossModel: Option[(Int, Seq[Tensor])], learningCurve: List[(Int, Double, Option[Double])]) extends LoopState with Product with Serializable
  7. case class SimpleThenSWALoopState(simple: SimpleLoopState, swa: Option[SWALoopState]) extends LoopState with Product with Serializable
  8. sealed trait StreamControl[+I] extends AnyRef
  9. trait TrainingCallback extends AnyRef
  10. trait ValidationCallback extends AnyRef

Value Members

  1. object BatchStream
  2. object BufferedImageHelper
  3. object DataParallel
  4. case object EmptyBatch extends StreamControl[Nothing] with Product with Serializable
  5. case object EndStream extends StreamControl[Nothing] with Product with Serializable
  6. object GraphBatchStream
  7. object IOLoops

    Contains a training loops and helpers around it

    Contains a training loops and helpers around it

    The two training loops implemented here are:

  8. object Reader
  9. object SWA
  10. object StateIO
  11. object StreamControl
  12. object Text
  13. object TrainingCallback
  14. object ValidationCallback
  15. object Writer

    Serializes tensors

    Serializes tensors

    This format is similar to the ONNX external tensor serialization format, but it uses JSON rather then protobuf.

    Format specification

    Sequences of tensors are serialized into a JSON descriptor and a data blob. The schema of the descriptor is the case class lamp.data.schemas.TensorList. The location field in this schema holds a path to the data blob. If this the location a relative POSIX then it is relative to the file path where the descriptor itself is written.

    The descriptor may be embedded into larger JSON structures.

    The data blob itself is the raw data in little endian byte order. Floating point is IEEE-754. The descriptor specifies the byte offset and byte length of the tensors inside the data blob. As such, the data blob contains no framing or other control bytes, but it may contain padding bytes between tensors.

Ungrouped