Index

A B C D E F G I J L M N O P R S U V W
All Classes and Interfaces|All Packages|Serialized Form

A

AceroSubstraitConsumer - Class in org.apache.arrow.dataset.substrait

Class to expose Java Substrait API for end users, currently operations supported are only to Consume Substrait Plan in Plan format (JSON) or Binary format (ByteBuffer).

AceroSubstraitConsumer(BufferAllocator) - Constructor for class org.apache.arrow.dataset.substrait.AceroSubstraitConsumer

ARROW_IPC - Enum constant in enum org.apache.arrow.dataset.file.FileFormat

ArrowScannerReader - Class in org.apache.arrow.dataset.scanner

An implementation of ArrowReader that reads the dataset from Scanner.

ArrowScannerReader(Scanner, BufferAllocator) - Constructor for class org.apache.arrow.dataset.scanner.ArrowScannerReader

Constructs a scanner reader using a Scanner.

B

build() - Method in class org.apache.arrow.dataset.scanner.ScanOptions.Builder

Builder(long) - Constructor for class org.apache.arrow.dataset.scanner.ScanOptions.Builder

Constructor.

bytesRead() - Method in class org.apache.arrow.dataset.jni.NativeScanner.NativeReader

bytesRead() - Method in class org.apache.arrow.dataset.scanner.ArrowScannerReader

C

close() - Method in class org.apache.arrow.dataset.jni.NativeDataset

close() - Method in class org.apache.arrow.dataset.jni.NativeDatasetFactory

Close this factory by release the pointer of the native instance.

close() - Method in class org.apache.arrow.dataset.jni.NativeMemoryPool

close() - Method in class org.apache.arrow.dataset.jni.NativeScanner

close() - Method in class org.apache.arrow.dataset.jni.NativeScanTask

Deprecated.

closeDataset(long) - Method in class org.apache.arrow.dataset.jni.JniWrapper

Release the Dataset by destroying its reference held by JNI wrapper.

closeDatasetFactory(long) - Method in class org.apache.arrow.dataset.jni.JniWrapper

Release the DatasetFactory by destroying its reference held by JNI wrapper.

closeReadSource() - Method in class org.apache.arrow.dataset.jni.NativeScanner.NativeReader

closeReadSource() - Method in class org.apache.arrow.dataset.scanner.ArrowScannerReader

closeScanner(long) - Method in class org.apache.arrow.dataset.jni.JniWrapper

Release the Scanner by destroying its reference held by JNI wrapper.

columns(Optional<String[]>) - Method in class org.apache.arrow.dataset.scanner.ScanOptions.Builder

Set the Projected columns.

convertMapToStringArray(Map<String, String>) - Static method in class org.apache.arrow.dataset.utils.MapUtil

Convert the map to string array as JNI bridge.

createDataset(long, byte[]) - Method in class org.apache.arrow.dataset.jni.JniWrapper

Create Dataset from a DatasetFactory and get the native pointer of the Dataset.

createListenable(ReservationListener) - Static method in class org.apache.arrow.dataset.jni.NativeMemoryPool

Create a listenable memory pool (see also: arrow::ReservationListenableMemoryPool) with a specific listener.

createScanner(long, String[], ByteBuffer, ByteBuffer, long, int, String[], long) - Method in class org.apache.arrow.dataset.jni.JniWrapper

Create Scanner from a Dataset and get the native pointer of the Dataset.

CSV - Enum constant in enum org.apache.arrow.dataset.file.FileFormat

CsvConvertOptions - Class in org.apache.arrow.dataset.scanner.csv

CsvConvertOptions(Map<String, String>) - Constructor for class org.apache.arrow.dataset.scanner.csv.CsvConvertOptions

CsvFragmentScanOptions - Class in org.apache.arrow.dataset.scanner.csv

CsvFragmentScanOptions(CsvConvertOptions, Map<String, String>, Map<String, String>) - Constructor for class org.apache.arrow.dataset.scanner.csv.CsvFragmentScanOptions

CSV scan options, map to CPP struct CsvFragmentScanOptions.

D

Dataset - Interface in org.apache.arrow.dataset.source

A container of Fragments which are the internal iterable unit of read data.

DatasetFactory - Interface in org.apache.arrow.dataset.source

DatasetFactory provides a way to inspect a Dataset potential schema before materializing it.

DatasetFileWriter - Class in org.apache.arrow.dataset.file

JNI-based utility to write datasets into files.

DatasetFileWriter() - Constructor for class org.apache.arrow.dataset.file.DatasetFileWriter

DirectReservationListener - Class in org.apache.arrow.dataset.jni

Reserving Java direct memory bytes from java.nio.Bits.

E

ensureLoaded() - Method in class org.apache.arrow.dataset.jni.JniLoader

If required JNI libraries are not loaded, then load them.

ensureS3Finalized() - Method in class org.apache.arrow.dataset.jni.JniWrapper

Ensure the S3 APIs are shutdown, but only if not already done.

execute() - Method in class org.apache.arrow.dataset.jni.NativeScanTask

Deprecated.

execute() - Method in interface org.apache.arrow.dataset.scanner.ScanTask

Deprecated.

Execute this ScanTask and return a Reader instance.

F

fileFormat() - Method in class org.apache.arrow.dataset.scanner.csv.CsvFragmentScanOptions

File format.

fileFormat() - Method in interface org.apache.arrow.dataset.scanner.FragmentScanOptions

FileFormat - Enum in org.apache.arrow.dataset.file

File format definitions.

FileSystemDatasetFactory - Class in org.apache.arrow.dataset.file

Java binding of the C++ FileSystemDatasetFactory.

FileSystemDatasetFactory(BufferAllocator, NativeMemoryPool, FileFormat, String) - Constructor for class org.apache.arrow.dataset.file.FileSystemDatasetFactory

FileSystemDatasetFactory(BufferAllocator, NativeMemoryPool, FileFormat, String[]) - Constructor for class org.apache.arrow.dataset.file.FileSystemDatasetFactory

FileSystemDatasetFactory(BufferAllocator, NativeMemoryPool, FileFormat, String[], Optional<FragmentScanOptions>) - Constructor for class org.apache.arrow.dataset.file.FileSystemDatasetFactory

FileSystemDatasetFactory(BufferAllocator, NativeMemoryPool, FileFormat, String, Optional<FragmentScanOptions>) - Constructor for class org.apache.arrow.dataset.file.FileSystemDatasetFactory

finish() - Method in class org.apache.arrow.dataset.jni.NativeDatasetFactory

finish() - Method in interface org.apache.arrow.dataset.source.DatasetFactory

Create a Dataset with auto-inferred schema.

finish(Schema) - Method in class org.apache.arrow.dataset.jni.NativeDatasetFactory

finish(Schema) - Method in interface org.apache.arrow.dataset.source.DatasetFactory

Create a Dataset with predefined schema.

fragmentScanOptions(FragmentScanOptions) - Method in class org.apache.arrow.dataset.scanner.ScanOptions.Builder

Set the FragmentScanOptions.

FragmentScanOptions - Interface in org.apache.arrow.dataset.scanner

The file fragment scan options interface.

G

get() - Static method in class org.apache.arrow.dataset.file.JniWrapper

get() - Static method in class org.apache.arrow.dataset.jni.JniLoader

get() - Static method in class org.apache.arrow.dataset.jni.JniWrapper

getAllocator() - Method in class org.apache.arrow.dataset.jni.NativeContext

Returns the allocator which is in use.

getArrowSchema() - Method in class org.apache.arrow.dataset.scanner.csv.CsvConvertOptions

getBatchSize() - Method in class org.apache.arrow.dataset.scanner.ScanOptions

getBytesAllocated() - Method in class org.apache.arrow.dataset.jni.NativeMemoryPool

Get current allocated bytes.

getColumns() - Method in class org.apache.arrow.dataset.scanner.ScanOptions

getConfigs() - Method in class org.apache.arrow.dataset.scanner.csv.CsvConvertOptions

getConvertOptions() - Method in class org.apache.arrow.dataset.scanner.csv.CsvFragmentScanOptions

getCurrentDirectMemReservation() - Method in class org.apache.arrow.dataset.jni.DirectReservationListener

Get current reservation of jVM direct memory.

getDefault() - Static method in class org.apache.arrow.dataset.jni.NativeMemoryPool

Get the default memory pool.

getFragmentScanOptions() - Method in class org.apache.arrow.dataset.scanner.ScanOptions

getMemoryPool() - Method in class org.apache.arrow.dataset.jni.NativeContext

Returns the native memory pool.

getNativeInstanceId() - Method in class org.apache.arrow.dataset.jni.NativeMemoryPool

Return native instance ID of this memory pool.

getParseOptions() - Method in class org.apache.arrow.dataset.scanner.csv.CsvFragmentScanOptions

getReadOptions() - Method in class org.apache.arrow.dataset.scanner.csv.CsvFragmentScanOptions

getSchemaFromScanner(long) - Method in class org.apache.arrow.dataset.jni.JniWrapper

Get a serialized schema from native instance of a Scanner.

getSubstraitFilter() - Method in class org.apache.arrow.dataset.scanner.ScanOptions

getSubstraitProjection() - Method in class org.apache.arrow.dataset.scanner.ScanOptions

I

id() - Method in enum org.apache.arrow.dataset.file.FileFormat

inspect() - Method in class org.apache.arrow.dataset.jni.NativeDatasetFactory

inspect() - Method in interface org.apache.arrow.dataset.source.DatasetFactory

Get unified schema for the resulting Dataset.

inspectSchema(long) - Method in class org.apache.arrow.dataset.jni.JniWrapper

Get a serialized schema from native instance of a DatasetFactory.

instance() - Static method in class org.apache.arrow.dataset.jni.DirectReservationListener

J

JniLoader - Class in org.apache.arrow.dataset.jni

The JniLoader for Dataset API's native implementation.

JniWrapper - Class in org.apache.arrow.dataset.file

JniWrapper for filesystem based Dataset implementations.

JniWrapper - Class in org.apache.arrow.dataset.jni

JNI wrapper for Dataset API's native implementation.

JSON - Enum constant in enum org.apache.arrow.dataset.file.FileFormat

L

loadDictionary(ArrowDictionaryBatch) - Method in class org.apache.arrow.dataset.jni.NativeScanner.NativeReader

loadDictionary(ArrowDictionaryBatch) - Method in class org.apache.arrow.dataset.scanner.ArrowScannerReader

loadNextBatch() - Method in class org.apache.arrow.dataset.jni.NativeScanner.NativeReader

loadNextBatch() - Method in class org.apache.arrow.dataset.scanner.ArrowScannerReader

loadRecordBatch(ArrowRecordBatch) - Method in class org.apache.arrow.dataset.jni.NativeScanner.NativeReader

loadRecordBatch(ArrowRecordBatch) - Method in class org.apache.arrow.dataset.scanner.ArrowScannerReader

M

makeFileSystemDatasetFactory(String, int, String[]) - Method in class org.apache.arrow.dataset.file.JniWrapper

Create FileSystemDatasetFactory and return its native pointer.

makeFileSystemDatasetFactoryWithFiles(String[], int, String[]) - Method in class org.apache.arrow.dataset.file.JniWrapper

Create FileSystemDatasetFactory and return its native pointer.

MapUtil - Class in org.apache.arrow.dataset.utils

The utility class for Map.

N

NativeContext - Class in org.apache.arrow.dataset.jni

Context for relevant classes of NativeDataset.

NativeContext(BufferAllocator, NativeMemoryPool) - Constructor for class org.apache.arrow.dataset.jni.NativeContext

Constructor.

NativeDataset - Class in org.apache.arrow.dataset.jni

Native implementation of Dataset.

NativeDataset(NativeContext, long) - Constructor for class org.apache.arrow.dataset.jni.NativeDataset

NativeDatasetFactory - Class in org.apache.arrow.dataset.jni

Native implementation of DatasetFactory.

NativeDatasetFactory(BufferAllocator, NativeMemoryPool, long) - Constructor for class org.apache.arrow.dataset.jni.NativeDatasetFactory

Constructor.

NativeInstanceReleasedException - Exception in org.apache.arrow.dataset.jni

Thrown if trying to operate on a native instance that is already released.

NativeInstanceReleasedException() - Constructor for exception org.apache.arrow.dataset.jni.NativeInstanceReleasedException

NativeInstanceReleasedException(String) - Constructor for exception org.apache.arrow.dataset.jni.NativeInstanceReleasedException

NativeMemoryPool - Class in org.apache.arrow.dataset.jni

C++ memory pool(arrow::MemoryPool)'s Java mapped instance.

NativeScanner - Class in org.apache.arrow.dataset.jni

Native implementation of Scanner.

NativeScanner(NativeContext, long) - Constructor for class org.apache.arrow.dataset.jni.NativeScanner

NativeScanner.NativeReader - Class in org.apache.arrow.dataset.jni

ArrowReader implementation for NativeDataset.

NativeScanTask - Class in org.apache.arrow.dataset.jni

Deprecated.

NativeScanTask(NativeScanner) - Constructor for class org.apache.arrow.dataset.jni.NativeScanTask

Deprecated.

Constructor.

newScan(ScanOptions) - Method in class org.apache.arrow.dataset.jni.NativeDataset

newScan(ScanOptions) - Method in interface org.apache.arrow.dataset.source.Dataset

Create a new Scanner using the provided scan options.

nextRecordBatch(long, long) - Method in class org.apache.arrow.dataset.jni.JniWrapper

Read next record batch from the specified scanner.

NONE - Enum constant in enum org.apache.arrow.dataset.file.FileFormat

O

ORC - Enum constant in enum org.apache.arrow.dataset.file.FileFormat

org.apache.arrow.dataset - module org.apache.arrow.dataset

org.apache.arrow.dataset.file - package org.apache.arrow.dataset.file

org.apache.arrow.dataset.jni - package org.apache.arrow.dataset.jni

org.apache.arrow.dataset.scanner - package org.apache.arrow.dataset.scanner

org.apache.arrow.dataset.scanner.csv - package org.apache.arrow.dataset.scanner.csv

org.apache.arrow.dataset.source - package org.apache.arrow.dataset.source

org.apache.arrow.dataset.substrait - package org.apache.arrow.dataset.substrait

org.apache.arrow.dataset.utils - package org.apache.arrow.dataset.utils

P

PARQUET - Enum constant in enum org.apache.arrow.dataset.file.FileFormat

R

readSchema() - Method in class org.apache.arrow.dataset.jni.NativeScanner.NativeReader

readSchema() - Method in class org.apache.arrow.dataset.scanner.ArrowScannerReader

releaseBuffer(long) - Method in class org.apache.arrow.dataset.jni.JniWrapper

Release the Buffer by destroying its reference held by JNI wrapper.

ReservationListener - Interface in org.apache.arrow.dataset.jni

Listener of buffer memory reservation.

reserve(long) - Method in class org.apache.arrow.dataset.jni.DirectReservationListener

Reserve bytes by invoking java.nio.java.Bitjava.nio.Bitss#reserveMemory.

reserve(long) - Method in interface org.apache.arrow.dataset.jni.ReservationListener

Reserve bytes.

runQuery(String) - Method in class org.apache.arrow.dataset.substrait.AceroSubstraitConsumer

Run Substrait plan.

runQuery(String, Map<String, ArrowReader>) - Method in class org.apache.arrow.dataset.substrait.AceroSubstraitConsumer

Run Substrait plan.

runQuery(ByteBuffer) - Method in class org.apache.arrow.dataset.substrait.AceroSubstraitConsumer

Run Substrait plan.

runQuery(ByteBuffer, Map<String, ArrowReader>) - Method in class org.apache.arrow.dataset.substrait.AceroSubstraitConsumer

Read binary Substrait plan, execute and return an ArrowReader to read Schema and ArrowRecordBatches.

S

scan() - Method in class org.apache.arrow.dataset.jni.NativeScanner

Deprecated.

scan() - Method in interface org.apache.arrow.dataset.scanner.Scanner

Deprecated.

use scanBatches() instead.

scanBatches() - Method in class org.apache.arrow.dataset.jni.NativeScanner

scanBatches() - Method in interface org.apache.arrow.dataset.scanner.Scanner

Read the dataset as a stream of record batches.

Scanner - Interface in org.apache.arrow.dataset.scanner

A high level interface for scanning data over dataset.

ScanOptions - Class in org.apache.arrow.dataset.scanner

Options used during scanning.

ScanOptions(long) - Constructor for class org.apache.arrow.dataset.scanner.ScanOptions

ScanOptions(long, Optional<String[]>) - Constructor for class org.apache.arrow.dataset.scanner.ScanOptions

Constructor.

ScanOptions(String[], long) - Constructor for class org.apache.arrow.dataset.scanner.ScanOptions

Deprecated.

Deprecated. Use ScanOptions(long, Optional) instead.

ScanOptions.Builder - Class in org.apache.arrow.dataset.scanner

Builder for Options used during scanning.

ScanTask - Interface in org.apache.arrow.dataset.scanner

Deprecated.

schema() - Method in class org.apache.arrow.dataset.jni.NativeScanner

schema() - Method in interface org.apache.arrow.dataset.scanner.Scanner

Get the schema of this Scanner.

serialize() - Method in class org.apache.arrow.dataset.scanner.csv.CsvFragmentScanOptions

This is an internal function to invoke by serializer.

serialize() - Method in interface org.apache.arrow.dataset.scanner.FragmentScanOptions

set(String, String) - Method in class org.apache.arrow.dataset.scanner.csv.CsvConvertOptions

setArrowSchema(ArrowSchema) - Method in class org.apache.arrow.dataset.scanner.csv.CsvConvertOptions

substraitFilter(ByteBuffer) - Method in class org.apache.arrow.dataset.scanner.ScanOptions.Builder

Set the Substrait extended expression for Filter.

substraitProjection(ByteBuffer) - Method in class org.apache.arrow.dataset.scanner.ScanOptions.Builder

Set the Substrait extended expression for Projection new columns.

U

unreserve(long) - Method in class org.apache.arrow.dataset.jni.DirectReservationListener

Unreserve bytes by invoking java.nio.java.Bitjava.nio.Bitss#unreserveMemory.

unreserve(long) - Method in interface org.apache.arrow.dataset.jni.ReservationListener

Unreserve bytes.

V

valueOf(String) - Static method in enum org.apache.arrow.dataset.file.FileFormat

Returns the enum constant of this type with the specified name.

values() - Static method in enum org.apache.arrow.dataset.file.FileFormat

Returns an array containing the constants of this enum type, in the order they are declared.

W

write(BufferAllocator, ArrowReader, FileFormat, String) - Static method in class org.apache.arrow.dataset.file.DatasetFileWriter

Write the contents of an ArrowReader as a dataset, with default partitioning settings.

write(BufferAllocator, ArrowReader, FileFormat, String, String[], int, String) - Static method in class org.apache.arrow.dataset.file.DatasetFileWriter

Write the contents of an ArrowReader as a dataset.

writeFromScannerToFile(long, long, String, String[], int, String) - Method in class org.apache.arrow.dataset.file.JniWrapper

Write the content in a ArrowArrayStream into files.

A B C D E F G I J L M N O P R S U V W
All Classes and Interfaces|All Packages|Serialized Form