ArrowScan

Type Members

trait ArrowAggregate extends Result

Trait for aggregating arrow files
abstract class ArrowResultsToFeatures[T] extends ResultsToFeatures[T]

Converts arrow-encoded results to features
Converts arrow-encoded results to features
T
result type
case class ArrowScanConfig(config: Map[String, String], reduce: FeatureReducer) extends Product with Serializable
class BatchAggregate extends ArrowAggregate

Returns record batches without any metadata.
Returns record batches without any metadata. Dictionaries must be known up front. Doesn't sort
class BatchReducer extends FeatureReducer

Reduce function for batches with a common schema.
Reduce function for batches with a common schema.
First feature contains metadata for arrow file and dictionary batch, subsequent features contain record batches, final feature contains EOF indicator
class DeltaAggregate extends ArrowAggregate

Returns batches of [threading key][dictionary deltas][record batch].
Returns batches of [threading key][dictionary deltas][record batch]. Will sort each batch, but not between batches.
class DeltaReducer extends FeatureReducer

Reduce function for delta batches.
Reduce function for delta batches.
First feature contains metadata for arrow file and dictionary batch, subsequent features contain record batches, final feature contains EOF indicator
class FileReducer extends FeatureReducer

Reduce function for whole arrow files coming back from the aggregating scan.
Reduce function for whole arrow files coming back from the aggregating scan. Each feature will have a single arrow file
class MultiFileAggregate extends ArrowAggregate

Returns full arrow files, with metadata.
Returns full arrow files, with metadata. Builds dictionaries on the fly. Doesn't sort
class MultiFileSortingAggregate extends ArrowAggregate

Returns full arrow files, with metadata.
Returns full arrow files, with metadata. Builds dictionaries on the fly. Sorts each file, but not between files
class SortingBatchAggregate extends ArrowAggregate

Returns record batches without any metadata.
Returns record batches without any metadata. Dictionaries must be known up front. Sorts each batch, but not between batches

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
object Configuration
val DictionaryOrdering: Ordering[AnyRef]
val DictionaryTopK: SystemProperty
object ReducerConfig
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
def configure(sft: SimpleFeatureType, index: GeoMesaFeatureIndex[_, _], stats: GeoMesaStats, filter: Option[Filter], ecql: Option[Filter], hints: Hints): ArrowScanConfig

Configure the iterator
Configure the iterator
sft
simple feature type
index
feature index
stats
stats, used for querying dictionaries
filter
full filter from the query, if any
ecql
secondary push down filter, if any
hints
query hints
def createDictionaries(stats: GeoMesaStats, sft: SimpleFeatureType, filter: Option[Filter], attributes: Seq[String], provided: Map[String, Array[AnyRef]], cached: Map[String, TopK[AnyRef]]): Map[String, ArrowDictionary]

Determine dictionary values, as required.
Determine dictionary values, as required. Priority:
1. values provided by the user 2. cached topk stats 3. enumeration stats query against result set
stats
stats
sft
simple feature type
filter
full filter for the query being run, used if querying enumeration values
attributes
names of attributes to dictionary encode
provided
provided dictionary values, if any, keyed by attribute name
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
def getBatchSize(hints: Hints): Int

Gets the batch size from query hints, or falls back to the system property
Gets the batch size from query hints, or falls back to the system property
hints
query hints
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
def hashCode(): Int

Definition Classes
AnyRef → Any
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
def resultFeature(): SimpleFeature

Simple feature used for returning from scans
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
AnyRef → Any
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

Related Docs: trait ArrowScan | package iterators

object ArrowScan

Type Members

trait ArrowAggregate extends Result

abstract class ArrowResultsToFeatures[T] extends ResultsToFeatures[T]

case class ArrowScanConfig(config: Map[String, String], reduce: FeatureReducer) extends Product with Serializable

class BatchAggregate extends ArrowAggregate

class BatchReducer extends FeatureReducer

class DeltaAggregate extends ArrowAggregate

class DeltaReducer extends FeatureReducer

class FileReducer extends FeatureReducer

class MultiFileAggregate extends ArrowAggregate

class MultiFileSortingAggregate extends ArrowAggregate

class SortingBatchAggregate extends ArrowAggregate

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

object Configuration

val DictionaryOrdering: Ordering[AnyRef]

val DictionaryTopK: SystemProperty

object ReducerConfig

final def asInstanceOf[T0]: T0

def clone(): AnyRef

def configure(sft: SimpleFeatureType, index: GeoMesaFeatureIndex[_, _], stats: GeoMesaStats, filter: Option[Filter], ecql: Option[Filter], hints: Hints): ArrowScanConfig

def createDictionaries(stats: GeoMesaStats, sft: SimpleFeatureType, filter: Option[Filter], attributes: Seq[String], provided: Map[String, Array[AnyRef]], cached: Map[String, TopK[AnyRef]]): Map[String, ArrowDictionary]

final def eq(arg0: AnyRef): Boolean

def equals(arg0: Any): Boolean

def finalize(): Unit

def getBatchSize(hints: Hints): Int

final def getClass(): Class[_]

def hashCode(): Int

final def isInstanceOf[T0]: Boolean

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

def resultFeature(): SimpleFeature

final def synchronized[T0](arg0: ⇒ T0): T0

def toString(): String

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Inherited from AnyRef

Inherited from Any

Ungrouped