io - GeoMesa Arrow GeoTools Abstractions 3.1.2 API - org.locationtech.geomesa.arrow.io

Type Members

class DeltaWriter extends Closeable with StrictLogging

Builds up dictionaries and write record batches.
Builds up dictionaries and write record batches. Dictionaries are encoded as deltas to minimize redundant messages.
class DictionaryBuildingWriter extends Closeable

Writes an arrow file of simple features.
Writes an arrow file of simple features. Dictionaries will be built up as features are observed. Dictionaries are encoded based on maxSize. Values will not be correctly encoded if more than maxSize distinct values are encountered
trait SimpleFeatureArrowFileReader extends Closeable

For reading simple features from an arrow file written by SimpleFeatureArrowFileWriter.
For reading simple features from an arrow file written by SimpleFeatureArrowFileWriter.
Expects arrow streaming format (no footer). Can handle multiple 'files' in a single input stream
class SimpleFeatureArrowFileWriter extends Closeable with Flushable with LazyLogging

For writing simple features to an arrow file.
For writing simple features to an arrow file.
Uses arrow streaming format (no footer). Closing the SimpleFeatureArrowFileWriter closes the given os.

Value Members

object BatchWriter
object ConcatenatedFileWriter
object DeltaWriter extends StrictLogging
object DictionaryBuildingWriter
object FormatVersion
object Metadata
object SimpleFeatureArrowFileReader
object SimpleFeatureArrowFileWriter
def createFileFromBatches(sft: SimpleFeatureType, dictionaries: Map[String, ArrowDictionary], encoding: SimpleFeatureEncoding, ipcOpts: IpcOption, sort: Option[(String, Boolean)], batches: CloseableIterator[Array[Byte]], firstBatchHasHeader: Boolean): CloseableIterator[Array[Byte]]

Create an arrow file from record batches
Create an arrow file from record batches
sft
simple feature type
dictionaries
dictionaries
encoding
feature encoding
sort
sorting of the batches, if any
batches
batches
firstBatchHasHeader
does the first batch have the arrow file header or not
def createRoot(vector: FieldVector, metadata: Map[String, String] = null): VectorSchemaRoot

Creates a vector schema root for the given vector
Creates a vector schema root for the given vector
vector
vector
metadata
field metadata
def createTransferPair(from: FieldVector, to: FieldVector): (Int, Int) ⇒ Unit

Create a transfer pair between two vectors.
Create a transfer pair between two vectors. This handles geometry vectors correctly, which the underlying arrow transfer pairs do not.
from
from vector
to
to vector
returns
transfer(fromIndex, toIndex)
def getSortAsMetadata(field: String, reverse: Boolean): Map[String, String]

Creates metadata for sort fields
Creates metadata for sort fields
field
sort field
reverse
reverse sorted or not
returns
metadata map
def getSortFromMetadata(metadata: Map[String, String]): Option[(String, Boolean)]

Checks schema metadata for sort fields
Checks schema metadata for sort fields
metadata
schema metadata
returns
(sort field, reverse sorted or not)
package reader
package records
def writeHeaderAndFirstBatch(result: SimpleFeatureVector, dictionaries: Map[String, ArrowDictionary], ipcOpts: IpcOption, sort: Option[(String, Boolean)], count: Int): Array[Byte]

Write out the header, dictionaries, and first batch of an arrow streaming file
Write out the header, dictionaries, and first batch of an arrow streaming file
result
vector loaded with first batch
dictionaries
dictionaries
sort
sort
count
number of records in first batch

io

package io

Type Members

class DeltaWriter extends Closeable with StrictLogging

class DictionaryBuildingWriter extends Closeable

trait SimpleFeatureArrowFileReader extends Closeable

class SimpleFeatureArrowFileWriter extends Closeable with Flushable with LazyLogging

Value Members

object BatchWriter

object ConcatenatedFileWriter

object DeltaWriter extends StrictLogging

object DictionaryBuildingWriter

object FormatVersion

object Metadata

object SimpleFeatureArrowFileReader

object SimpleFeatureArrowFileWriter

def createFileFromBatches(sft: SimpleFeatureType, dictionaries: Map[String, ArrowDictionary], encoding: SimpleFeatureEncoding, ipcOpts: IpcOption, sort: Option[(String, Boolean)], batches: CloseableIterator[Array[Byte]], firstBatchHasHeader: Boolean): CloseableIterator[Array[Byte]]

def createRoot(vector: FieldVector, metadata: Map[String, String] = null): VectorSchemaRoot

def createTransferPair(from: FieldVector, to: FieldVector): (Int, Int) ⇒ Unit

def getSortAsMetadata(field: String, reverse: Boolean): Map[String, String]

def getSortFromMetadata(metadata: Map[String, String]): Option[(String, Boolean)]

package reader

package records

def writeHeaderAndFirstBatch(result: SimpleFeatureVector, dictionaries: Map[String, ArrowDictionary], ipcOpts: IpcOption, sort: Option[(String, Boolean)], count: Int): Array[Byte]

Inherited from AnyRef

Inherited from Any

Ungrouped