Table

ra3.Table
See theTable companion object
case class Table(columns: Vector[TaggedColumn], colNames: Vector[String], uniqueId: String, partitions: Option[PartitionData])

Reference to a set of aligned columns (i.e. a table) persisted onto secondary storage.

Each table must have a unique identifier, initially given by the importCsv method.

Tables have String column names.

Tables consists of columns. Columns are stored as segments. Segments are the unit of IO operations, i.e. ra3 never reads less then a segment into memory. The in memory (buffered) counterpart of a segment is a Buffer. The maximum number of elements in a segment is thus what is readable into a single java array, that is shortly below 2^31.

Each column of the same table has the same segmentation, i.e. they have the same number of segments and their segments have the same size and those segments are aligned.

Segments store segment level statistics and some operations complete withour buffering the segment.

Attributes

Companion
object
Experimental
true
Graph
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

Members list

Value members

Concrete methods

def apply(s: String): TaggedColumn

Selects a column by name, or throws if not exists.

Selects a column by name, or throws if not exists.

Attributes

def apply(i: Int): TaggedColumn
def as[T1 <: Tuple]: Const[ReturnValueTuple[T1]]

Schema definition

Schema definition

Entry point of TableExpr DSL

Attributes

def bufferStream(implicit tsc: TaskSystemComponents): Stream[IO, BufferedTable]

Returns an fs2 Stream with a BufferedTable for each segment number

Returns an fs2 Stream with a BufferedTable for each segment number

Attributes

def numCols: Int
def numRows: Long
def showSample(nrows: Int, ncols: Int)(implicit tsc: TaskSystemComponents): IO[String]

Renders a sample of the table's data content

Renders a sample of the table's data content

Attributes

transparent inline def stream[T <: Tuple](implicit tsc: TaskSystemComponents): Stream[IO, T]

Returns an fs2 Stream with a tuple for each element

Returns an fs2 Stream with a tuple for each element

Attributes

override def toString: String

Returns a string representation of the object.

Returns a string representation of the object.

The default representation is platform dependent.

Attributes

Returns

a string representation of the object.

Definition Classes
Any

Inherited methods

def exportToCsv(columnSeparator: Char, quoteChar: Char, recordSeparator: String, compression: Option[CompressionFormat])(implicit tsc: TaskSystemComponents): IO[List[SharedFile]]

Attributes

Inherited from:
RelationalAlgebra (hidden)
def productElementNames: Iterator[String]

Attributes

Inherited from:
Product
def productIterator: Iterator[Any]

Attributes

Inherited from:
Product