lamp-data
API
Generated with
lamp-data/lamp.data/BatchStream

BatchStream

lamp.data.BatchStream
See theBatchStream companion trait
object BatchStream

Attributes

Companion
trait
Graph
Supertypes
class Object
trait Matchable
class Any
Self type

Members list

Value members

Concrete methods

def fromFullBatch(features: STen, targets: STen, device: Device): BatchStream[(Variable, STen), Boolean, Unit]

Create a stream of a single full batch of features and targets

Create a stream of a single full batch of features and targets

Attributes

def fromFunction[A, C](numBatches: Int, makeNonEmptyBatch: Device => Resource[IO, StreamControl[(A, STen)]]): BatchStream[(A, STen), Int, Unit]
def fromFunctionWithBuffers[A, C](numBatches: Int, allocateBuffers1: Device => Resource[IO, C])(makeNonEmptyBatch: (C, Device) => Resource[IO, StreamControl[A]]): BatchStream[A, Int, C]
def fromIndices[A, C](indices: Array[Array[Int]])(makeNonEmptyBatch: (Array[Int], Device) => Resource[IO, StreamControl[(A, STen)]]): BatchStream[(A, STen), Int, Unit]

Creates a stream from an array of indices and a lambda using a subset of those indexes to allocate the batch

Creates a stream from an array of indices and a lambda using a subset of those indexes to allocate the batch

The indices refer to some other external data structure

Attributes

def fromIndicesWithBuffers[A, C](indices: Array[Array[Int]], allocateBuffers1: Device => Resource[IO, C])(makeNonEmptyBatch: (Array[Int], C, Device) => Resource[IO, StreamControl[A]]): BatchStream[A, Int, C]

Creates a stream from an array of indices and a lambda using a subset of those indexes to allocate the batch

Creates a stream from an array of indices and a lambda using a subset of those indexes to allocate the batch

The indices refer to some other external data structure

Attributes

def fromVector[A](resources: Vector[Resource[IO, StreamControl[A]]]): BatchStream[A, Int, Unit]

Creates a stream from a vector of items

Creates a stream from a vector of items

Attributes

def minibatchesFromFull(minibatchSize: Int, dropLast: Boolean, features: STen, target: STen, rng: Random): BatchStream[(Variable, STen), Int, BufferPair]

Create a stream from the first dimension of a tensor

Create a stream from the first dimension of a tensor

Attributes

def single[A](resource: Resource[IO, StreamControl[A]]): BatchStream[A, Boolean, Unit]

Creates a stream from a single item

Creates a stream from a single item

Attributes

def stagedFromIndices[A, B, C](indices: Array[Array[Int]], bucketSize: Int, allocateBuffers0: Device => Resource[IO, C])(loadInstancesToStaging: Array[Int] => Resource[IO, B], makeNonEmptyBatch: (B, Array[Int], C, Device) => Resource[IO, StreamControl[A]]): BatchStream[A, State[A, B], C]

A two stage data loader which first loads items of type B, then from B loads items of type A

A two stage data loader which first loads items of type B, then from B loads items of type A

Makes sense if loading B is quicker than loading an equivalent amount of A e.g. because B is a preformed batch of A-s on secondary medium

Attributes

Generated with