Index
All Classes and Interfaces|All Packages|Constant Field Values|Serialized Form
A
- ActionsIterator - Class in io.delta.kernel.internal.replay
-
This class takes as input a list of delta files (.json, .checkpoint.parquet) and produces an iterator of (ColumnarBatch, isFromCheckpoint) tuples, where the schema of the ColumnarBatch semantically represents actions (or, a subset of action fields) parsed from the Delta Log.
- ActionsIterator(Engine, List<FileStatus>, StructType, StructType, Optional<Predicate>) - Constructor for class io.delta.kernel.internal.replay.ActionsIterator
- ActionsIterator(Engine, List<FileStatus>, StructType, Optional<Predicate>) - Constructor for class io.delta.kernel.internal.replay.ActionsIterator
- ActionWrapper - Class in io.delta.kernel.internal.replay
-
Internal wrapper class holding information needed to perform log replay.
- activeAddFilesCounter - Variable in class io.delta.kernel.internal.metrics.ScanMetrics
- ActiveAddFilesIterator - Class in io.delta.kernel.internal.replay
-
This class takes an iterator of (
ColumnarBatch
, isFromCheckpoint), where the columnar data inside the columnar batch represents has top level columns "add" and "remove", and produces an iterator ofFilteredColumnarBatch
with only the "add" column and with a selection vector indicating which AddFiles are still active in the table (have not been tombstoned). - add(long) - Method in class io.delta.kernel.internal.deletionvectors.RoaringBitmapArray
- add(StructField) - Method in class io.delta.kernel.types.StructType
- add(String, DataType) - Method in class io.delta.kernel.types.StructType
- add(String, DataType, boolean) - Method in class io.delta.kernel.types.StructType
- add(String, DataType, boolean, FieldMetadata) - Method in class io.delta.kernel.types.StructType
- add(String, DataType, FieldMetadata) - Method in class io.delta.kernel.types.StructType
- ADD - Enum constant in enum io.delta.kernel.internal.DeltaLogActionUtils.DeltaAction
- ADD_FILE_DV_ORDINAL - Static variable in class io.delta.kernel.internal.replay.LogReplay
- ADD_FILE_ORDINAL - Static variable in class io.delta.kernel.internal.actions.SingleAction
- ADD_FILE_ORDINAL - Static variable in class io.delta.kernel.internal.InternalScanFileUtils
- ADD_FILE_ORDINAL - Static variable in class io.delta.kernel.internal.replay.LogReplay
- ADD_FILE_PARTITION_COL_REF - Static variable in class io.delta.kernel.internal.InternalScanFileUtils
-
Column
expression referring to the `partitionValues` in scan `add` file. - ADD_FILE_PATH_ORDINAL - Static variable in class io.delta.kernel.internal.replay.LogReplay
- ADD_FILE_STATS_ORDINAL - Static variable in class io.delta.kernel.internal.InternalScanFileUtils
- AddCDCFile - Class in io.delta.kernel.internal.actions
-
Metadata about
cdc
action in the Delta Log. - AddCDCFile() - Constructor for class io.delta.kernel.internal.actions.AddCDCFile
- addDomainMetadata(String, String) - Method in class io.delta.kernel.internal.TransactionImpl
- addDomainMetadata(String, String) - Method in interface io.delta.kernel.Transaction
-
Commit the provided domain metadata as part of this transaction.
- addDomainMetadataInternal(String, String) - Method in class io.delta.kernel.internal.TransactionImpl
- AddFile - Class in io.delta.kernel.internal.actions
-
Delta log action representing an `AddFile`
- AddFile(Row) - Constructor for class io.delta.kernel.internal.actions.AddFile
- ADDFILE_FIELD_NAME - Static variable in class io.delta.kernel.internal.replay.LogReplay
- addFilesCounter - Variable in class io.delta.kernel.internal.metrics.ScanMetrics
- addFilesFromDeltaFilesCounter - Variable in class io.delta.kernel.internal.metrics.ScanMetrics
- allFilesWithCompactionsReversed() - Method in class io.delta.kernel.internal.snapshot.LogSegment
- allLogFilesReversed() - Method in class io.delta.kernel.internal.snapshot.LogSegment
- allLogFilesUnsorted() - Method in class io.delta.kernel.internal.snapshot.LogSegment
- ALWAYS_FALSE - Static variable in class io.delta.kernel.expressions.AlwaysFalse
- ALWAYS_TRUE - Static variable in class io.delta.kernel.expressions.AlwaysTrue
- AlwaysFalse - Class in io.delta.kernel.expressions
-
Predicate which always evaluates to
false
. - AlwaysTrue - Class in io.delta.kernel.expressions
-
Predicate which always evaluates to
true
. - And - Class in io.delta.kernel.expressions
-
AND
expression - And(Predicate, Predicate) - Constructor for class io.delta.kernel.expressions.And
- APPEND_ONLY_ENABLED - Static variable in class io.delta.kernel.internal.TableConfig
-
Whether this Delta table is append-only.
- APPEND_ONLY_W_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- appendNestedField(String) - Method in class io.delta.kernel.expressions.Column
-
Returns a new column that appends the input column name to the current column.
- ArrayType - Class in io.delta.kernel.types
-
Represent
array
data type - ArrayType(DataType, boolean) - Constructor for class io.delta.kernel.types.ArrayType
- ArrayType(StructField) - Constructor for class io.delta.kernel.types.ArrayType
- ArrayValue - Interface in io.delta.kernel.data
-
Abstraction to represent a single array value in a
ColumnVector
. - asPredicate(Expression) - Static method in class io.delta.kernel.internal.util.ExpressionUtils
-
Return an expression cast as a predicate, throw an error if it is not a predicate
- assertLogFilesBelongToTable(Path, List<FileStatus>) - Static method in class io.delta.kernel.internal.replay.LogReplayUtils
-
Verifies that a set of delta or checkpoint files to be read actually belongs to this table.
- assignBaseRowIdAndDefaultRowCommitVersion(SnapshotImpl, Protocol, Optional<Long>, Optional<Long>, long, CloseableIterable<Row>) - Static method in class io.delta.kernel.internal.rowtracking.RowTracking
-
Assigns or reassigns baseRowIds and defaultRowCommitVersions to
AddFile
actions in the provideddataActions
. - assignMaterializedColumnNamesIfNeeded(Metadata) - Static method in class io.delta.kernel.internal.rowtracking.MaterializedRowTrackingColumn
-
Assigns materialized column names for ROW_ID and ROW_COMMIT_VERSION if row tracking is enabled and the column names have not been assigned yet.
- at(int) - Method in class io.delta.kernel.types.StructType
- autoUpgradeProtocolBasedOnMetadata(Metadata, Collection<TableFeature>, Protocol) - Static method in class io.delta.kernel.internal.tablefeatures.TableFeatures
-
Upgrade the current protocol to satisfy all auto-update capable features required by the given metadata.
B
- Base85Codec - Class in io.delta.kernel.internal.deletionvectors
-
This implements Base85 using the 4 byte block aligned encoding and character set from Z85.
- Base85Codec() - Constructor for class io.delta.kernel.internal.deletionvectors.Base85Codec
- BasePrimitiveType - Class in io.delta.kernel.types
-
Base class for all primitive types
DataType
. - BINARY - Static variable in class io.delta.kernel.types.BinaryType
- BinaryType - Class in io.delta.kernel.types
-
The data type representing
byte[]
values. - blockIfColumnMappingEnabled(Row) - Static method in class io.delta.kernel.internal.util.ColumnMapping
-
Utility method to block writing into a table with column mapping enabled.
- BOOLEAN - Static variable in class io.delta.kernel.types.BooleanType
- BooleanType - Class in io.delta.kernel.types
-
Data type representing
boolean
type values. - BREAK - Enum constant in enum io.delta.kernel.utils.CloseableIterator.BreakableFilterResult
-
Indicates that the iteration should stop immediately and that no further elements should be processed by
CloseableIterator.breakableFilter(Function)
. - breakableFilter(Function<T, CloseableIterator.BreakableFilterResult>) - Method in interface io.delta.kernel.utils.CloseableIterator
-
Returns a new
CloseableIterator
that applies aCloseableIterator.BreakableFilterResult
-based filtering function to determine whether elements of this iterator should be included or excluded, or whether the iteration should terminate. - build() - Method in class io.delta.kernel.internal.ScanBuilderImpl
- build() - Method in interface io.delta.kernel.ScanBuilder
- build() - Method in class io.delta.kernel.types.FieldMetadata.Builder
- build(Engine) - Method in class io.delta.kernel.internal.ReplaceTableTransactionBuilderImpl
- build(Engine) - Method in class io.delta.kernel.internal.TransactionBuilderImpl
- build(Engine) - Method in interface io.delta.kernel.TransactionBuilder
-
Build the transaction.
- buildArrayValue(List<?>, DataType) - Static method in class io.delta.kernel.internal.util.VectorUtils
-
Creates an
ArrayValue
from list of objects. - buildColumnVector(List<?>, DataType) - Static method in class io.delta.kernel.internal.util.VectorUtils
-
Utility method to create a
ColumnVector
for given list of object, the object should be primitive type or an Row instance. - builder() - Static method in class io.delta.kernel.types.FieldMetadata
- Builder() - Constructor for class io.delta.kernel.types.FieldMetadata.Builder
- buildLatestSnapshot(Engine, SnapshotQueryContext) - Method in class io.delta.kernel.internal.snapshot.SnapshotManager
-
Construct the latest snapshot for given table.
- BYTE - Static variable in class io.delta.kernel.types.ByteType
- ByteType - Class in io.delta.kernel.types
-
The data type representing
byte
type values.
C
- cannotModifyAppendOnlyTable(String) - Static method in class io.delta.kernel.internal.DeltaErrors
- cannotModifyTableProperty(String) - Static method in class io.delta.kernel.internal.DeltaErrors
- cannotToggleRowTrackingOnExistingTable() - Static method in class io.delta.kernel.internal.DeltaErrors
- canUpgradeTo(Protocol) - Method in class io.delta.kernel.internal.actions.Protocol
-
Determine whether this protocol can be safely upgraded to a new protocol `to`.
- captureFileSizeHistogramResult() - Method in class io.delta.kernel.internal.stats.FileSizeHistogram
- captureScanMetricsResult() - Method in class io.delta.kernel.internal.metrics.ScanMetrics
- captureSnapshotMetricsResult() - Method in class io.delta.kernel.internal.metrics.SnapshotMetrics
- captureTransactionMetricsResult() - Method in class io.delta.kernel.internal.metrics.TransactionMetrics
- CaseInsensitiveMap<V> - Class in io.delta.kernel.internal.util
-
A map that is case-insensitive in its keys.
- CaseInsensitiveMap() - Constructor for class io.delta.kernel.internal.util.CaseInsensitiveMap
- casePreservingEligibleClusterColumns(StructType, List<Column>) - Static method in class io.delta.kernel.internal.util.SchemaUtils
-
Verify the clustering columns exists in the table schema.
- casePreservingPartitionColNames(StructType, List<String>) - Static method in class io.delta.kernel.internal.util.SchemaUtils
-
Delta expects partition column names to be same case preserving as the name in the schema.
- casePreservingPartitionColNames(List<String>, Map<String, Literal>) - Static method in class io.delta.kernel.internal.util.SchemaUtils
-
Convert the partition column names in
partitionValues
map into the same case as the column in the table metadata. - category - Variable in enum io.delta.kernel.internal.files.ParsedLogData.ParsedLogType
- CDC - Enum constant in enum io.delta.kernel.internal.DeltaLogActionUtils.DeltaAction
- CHANGE_DATA_FEED_ENABLED - Static variable in class io.delta.kernel.internal.TableConfig
-
Enable change data feed output.
- CHANGE_DATA_FEED_W_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- checkArgument(boolean) - Static method in class io.delta.kernel.internal.util.Preconditions
-
Precondition-style validation that throws
IllegalArgumentException
. - checkArgument(boolean, String) - Static method in class io.delta.kernel.internal.util.Preconditions
-
Precondition-style validation that throws
IllegalArgumentException
. - checkArgument(boolean, String, Object...) - Static method in class io.delta.kernel.internal.util.Preconditions
-
Precondition-style validation that throws
IllegalArgumentException
. - checkpoint(Engine, long) - Method in class io.delta.kernel.internal.TableImpl
- checkpoint(Engine, long) - Method in interface io.delta.kernel.Table
-
Checkpoint the table at given version.
- checkpoint(Engine, Clock, SnapshotImpl) - Static method in class io.delta.kernel.internal.checkpoints.Checkpointer
- CHECKPOINT - Enum constant in enum io.delta.kernel.hook.PostCommitHook.PostCommitHookType
-
Writes a new checkpoint at the version committed by the transaction.
- CHECKPOINT - Enum constant in enum io.delta.kernel.internal.files.ParsedLogData.ParsedLogCategory
- CHECKPOINT - Enum constant in enum io.delta.kernel.internal.util.FileNames.DeltaLogFileType
- CHECKPOINT_CLASSIC - Enum constant in enum io.delta.kernel.internal.replay.DeltaLogFile.LogType
- CHECKPOINT_INTERVAL - Static variable in class io.delta.kernel.internal.TableConfig
-
How often to checkpoint the delta log? For every N (this config) commits to the log, we will suggest write out a checkpoint file that can speed up the Delta table state reconstruction.
- CHECKPOINT_POLICY - Static variable in class io.delta.kernel.internal.TableConfig
- CHECKPOINT_SCHEMA - Static variable in class io.delta.kernel.internal.actions.SingleAction
-
Get the schema of reading entries from Delta Log delta and checkpoint files for construction of new checkpoint.
- CHECKPOINT_V2_RW_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- CheckpointAlreadyExistsException - Exception in io.delta.kernel.exceptions
-
Thrown when trying to create a checkpoint at version
v
, but there already exists a checkpoint at versionv
. - CheckpointAlreadyExistsException(long) - Constructor for exception io.delta.kernel.exceptions.CheckpointAlreadyExistsException
- Checkpointer - Class in io.delta.kernel.internal.checkpoints
-
Class to load and write the
CheckpointMetaData
from `_last_checkpoint` file. - Checkpointer(Path) - Constructor for class io.delta.kernel.internal.checkpoints.Checkpointer
- checkpointFileSingular(Path, long) - Static method in class io.delta.kernel.internal.util.FileNames
-
Returns the path for a singular checkpoint up to the given version.
- checkpointFileWithParts(Path, long, int) - Static method in class io.delta.kernel.internal.util.FileNames
-
Returns the paths for all parts of the checkpoint up to the given version.
- CheckpointHook - Class in io.delta.kernel.internal.hook
-
Write a new checkpoint at the version committed by the txn.
- CheckpointHook(Path, long) - Constructor for class io.delta.kernel.internal.hook.CheckpointHook
- CheckpointInstance - Class in io.delta.kernel.internal.checkpoints
-
Metadata about Delta checkpoint.
- CheckpointInstance(long) - Constructor for class io.delta.kernel.internal.checkpoints.CheckpointInstance
- CheckpointInstance(long, Optional<Integer>) - Constructor for class io.delta.kernel.internal.checkpoints.CheckpointInstance
- CheckpointInstance(String) - Constructor for class io.delta.kernel.internal.checkpoints.CheckpointInstance
- CheckpointInstance.CheckpointFormat - Enum in io.delta.kernel.internal.checkpoints
- CheckpointMetaData - Class in io.delta.kernel.internal.checkpoints
- CheckpointMetaData(long, long, Optional<Long>) - Constructor for class io.delta.kernel.internal.checkpoints.CheckpointMetaData
- checkpointVersion(Path) - Static method in class io.delta.kernel.internal.util.FileNames
-
Returns the version for the given checkpoint path.
- checkpointVersion(String) - Static method in class io.delta.kernel.internal.util.FileNames
- checkState(boolean, String) - Static method in class io.delta.kernel.internal.util.Preconditions
-
Ensures the truth of an expression involving the state of the calling instance.
- checksum(Engine, long) - Method in class io.delta.kernel.internal.TableImpl
- checksum(Engine, long) - Method in interface io.delta.kernel.Table
-
Computes and writes a checksum file for the table at given version.
- CHECKSUM - Enum constant in enum io.delta.kernel.internal.files.ParsedLogData.ParsedLogCategory
- CHECKSUM - Enum constant in enum io.delta.kernel.internal.files.ParsedLogData.ParsedLogType
- CHECKSUM - Enum constant in enum io.delta.kernel.internal.util.FileNames.DeltaLogFileType
- CHECKSUM_FULL - Enum constant in enum io.delta.kernel.hook.PostCommitHook.PostCommitHookType
-
Writes a checksum file at the version committed by the transaction.
- CHECKSUM_SIMPLE - Enum constant in enum io.delta.kernel.hook.PostCommitHook.PostCommitHookType
-
Writes a checksum file at the version committed by the transaction.
- checksumFile(Path, long) - Static method in class io.delta.kernel.internal.util.FileNames
-
Returns the path to the checksum file for the given version.
- ChecksumFullHook - Class in io.delta.kernel.internal.hook
-
A post-commit hook that writes a new checksum file at the version committed by the transaction.
- ChecksumFullHook(Path, long) - Constructor for class io.delta.kernel.internal.hook.ChecksumFullHook
- ChecksumReader - Class in io.delta.kernel.internal.checksum
-
Utility method to load protocol and metadata from the Delta log checksum files.
- ChecksumReader() - Constructor for class io.delta.kernel.internal.checksum.ChecksumReader
- ChecksumSimpleHook - Class in io.delta.kernel.internal.hook
-
A post-commit hook that writes a new checksum file at the version committed by the transaction.
- ChecksumSimpleHook(CRCInfo, Path) - Constructor for class io.delta.kernel.internal.hook.ChecksumSimpleHook
- ChecksumUtils - Class in io.delta.kernel.internal.checksum
- checksumVersion(Path) - Static method in class io.delta.kernel.internal.util.FileNames
- checksumVersion(String) - Static method in class io.delta.kernel.internal.util.FileNames
- ChecksumWriter - Class in io.delta.kernel.internal.checksum
-
Writers for writing checksum files from a snapshot
- ChecksumWriter(Path) - Constructor for class io.delta.kernel.internal.checksum.ChecksumWriter
- ChildVectorBasedRow - Class in io.delta.kernel.internal.data
-
A
Row
implementation that wraps a set of child vectors for a specificrowId
. - ChildVectorBasedRow(int, StructType) - Constructor for class io.delta.kernel.internal.data.ChildVectorBasedRow
- CLASSIC - Enum constant in enum io.delta.kernel.internal.checkpoints.CheckpointInstance.CheckpointFormat
- CLASSIC_CHECKPOINT - Enum constant in enum io.delta.kernel.internal.files.ParsedLogData.ParsedLogType
- cleanupExpiredLogs(Engine, Clock, Path, long) - Static method in class io.delta.kernel.internal.snapshot.MetadataCleanup
-
Delete the Delta log files (delta and checkpoint files) that are expired according to the table metadata retention settings.
- clear() - Method in class io.delta.kernel.internal.util.CaseInsensitiveMap
- Clock - Interface in io.delta.kernel.internal.util
-
An interface to represent clocks, so that they can be mocked out in unit tests.
- close() - Method in interface io.delta.kernel.data.ColumnVector
-
Cleans up memory for this column vector.
- close() - Method in class io.delta.kernel.internal.data.GenericColumnVector
- close() - Method in class io.delta.kernel.internal.data.SelectionColumnVector
- close() - Method in interface io.delta.kernel.internal.metrics.Timer.Timed
- close() - Method in class io.delta.kernel.internal.replay.ActionsIterator
- close() - Method in class io.delta.kernel.internal.replay.ActiveAddFilesIterator
- close() - Method in class io.delta.kernel.internal.replay.CreateCheckpointIterator
- CloseableIterable<T> - Interface in io.delta.kernel.utils
-
Extend the Java
Iterable
interface to provide a way to close the iterator. - CloseableIterator<T> - Interface in io.delta.kernel.utils
-
Closeable extension of
Iterator
- CloseableIterator.BreakableFilterResult - Enum in io.delta.kernel.utils
-
Represents the result of applying the filter condition in the
CloseableIterator.breakableFilter(Function)
method of aCloseableIterator
. - closeCloseables(AutoCloseable...) - Static method in class io.delta.kernel.internal.util.Utils
-
Close the given one or more
AutoCloseable
s. - closeCloseablesSilently(AutoCloseable...) - Static method in class io.delta.kernel.internal.util.Utils
-
Close the given list of
AutoCloseable
objects. - CLUSTERING_W_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- ClusteringMetadataDomain - Class in io.delta.kernel.internal.clustering
-
Represents the metadata domain for clustering.
- ClusteringUtils - Class in io.delta.kernel.internal.clustering
- CollationIdentifier - Class in io.delta.kernel.types
-
Identifies collation for string type.
- COLLATIONS_METADATA_KEY - Static variable in class io.delta.kernel.types.StructField
- collectLeafColumns(StructType, Set<String>, int) - Static method in class io.delta.kernel.internal.util.SchemaUtils
-
Collects all leaf columns from the given schema (including flattened columns only for StructTypes), up to maxColumns.
- colName - Variable in enum io.delta.kernel.internal.DeltaLogActionUtils.DeltaAction
- column(int) - Method in class io.delta.kernel.types.StructType
-
Creates a
Column
expression for the field at the givenordinal
- Column - Class in io.delta.kernel.expressions
-
An expression type that refers to a column (case-sensitive) in the input.
- Column(String) - Constructor for class io.delta.kernel.expressions.Column
-
Create a column expression for referring to a column.
- Column(String[]) - Constructor for class io.delta.kernel.expressions.Column
-
Create a column expression to refer to a nested column.
- COLUMN_MAPPING_ID_KEY - Static variable in class io.delta.kernel.internal.util.ColumnMapping
- COLUMN_MAPPING_MAX_COLUMN_ID - Static variable in class io.delta.kernel.internal.TableConfig
-
This table property is used to control the maximum column mapping ID.
- COLUMN_MAPPING_MAX_COLUMN_ID_KEY - Static variable in class io.delta.kernel.internal.util.ColumnMapping
- COLUMN_MAPPING_MODE - Static variable in class io.delta.kernel.internal.TableConfig
-
This table property is used to control the column mapping mode.
- COLUMN_MAPPING_MODE_KEY - Static variable in class io.delta.kernel.internal.util.ColumnMapping
- COLUMN_MAPPING_NESTED_IDS_KEY - Static variable in class io.delta.kernel.internal.util.ColumnMapping
- COLUMN_MAPPING_PHYSICAL_NAME_KEY - Static variable in class io.delta.kernel.internal.util.ColumnMapping
- COLUMN_MAPPING_RW_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- ColumnarBatch - Interface in io.delta.kernel.data
-
Represents zero or more rows of records with same schema type.
- ColumnarBatchRow - Class in io.delta.kernel.internal.data
-
Row abstraction around a columnar batch and a particular row within the columnar batch.
- ColumnarBatchRow(ColumnarBatch, int) - Constructor for class io.delta.kernel.internal.data.ColumnarBatchRow
- columnInvariantsNotSupported() - Static method in class io.delta.kernel.internal.DeltaErrors
- ColumnMapping - Class in io.delta.kernel.internal.util
-
Utilities related to the column mapping feature.
- ColumnMapping.ColumnMappingMode - Enum in io.delta.kernel.internal.util
- columnNotFoundInSchema(Column, StructType) - Static method in class io.delta.kernel.internal.DeltaErrors
- ColumnVector - Interface in io.delta.kernel.data
-
Represents zero or more values of a single column.
- combine(CloseableIterator<T>) - Method in interface io.delta.kernel.utils.CloseableIterator
-
Combine the current iterator with another iterator.
- commit(Engine, CloseableIterable<Row>) - Method in class io.delta.kernel.internal.TransactionImpl
- commit(Engine, CloseableIterable<Row>) - Method in interface io.delta.kernel.Transaction
-
Commit the transaction including the data action rows generated by
Transaction.generateAppendActions(io.delta.kernel.engine.Engine, io.delta.kernel.data.Row, io.delta.kernel.utils.CloseableIterator<io.delta.kernel.utils.DataFileStatus>, io.delta.kernel.DataWriteContext)
. - COMMIT - Enum constant in enum io.delta.kernel.internal.replay.DeltaLogFile.LogType
- COMMIT - Enum constant in enum io.delta.kernel.internal.util.FileNames.DeltaLogFileType
- commitAttemptsCounter - Variable in class io.delta.kernel.internal.metrics.TransactionMetrics
- CommitInfo - Class in io.delta.kernel.internal.actions
-
Delta log action representing a commit information action.
- CommitInfo(Optional<Long>, long, String, String, Map<String, String>, boolean, String, Map<String, String>) - Constructor for class io.delta.kernel.internal.actions.CommitInfo
- COMMITINFO - Enum constant in enum io.delta.kernel.internal.DeltaLogActionUtils.DeltaAction
- compareTo(CheckpointInstance) - Method in class io.delta.kernel.internal.checkpoints.CheckpointInstance
-
Comparison rules: 1.
- compareTo(Path) - Method in class io.delta.kernel.internal.fs.Path
- computeStateAndWriteChecksum(Engine, SnapshotImpl) - Static method in class io.delta.kernel.internal.checksum.ChecksumUtils
-
Computes the state of a Delta table and writes a checksum file for the provided snapshot's version.
- concatWithDot(List<String>) - Static method in class io.delta.kernel.internal.util.SchemaUtils
- concurrentDomainMetadataAction(DomainMetadata, DomainMetadata) - Static method in class io.delta.kernel.internal.DeltaErrors
- concurrentTransaction(String, long, long) - Static method in class io.delta.kernel.internal.DeltaErrors
- ConcurrentTransactionException - Exception in io.delta.kernel.exceptions
-
Thrown when concurrent transaction both attempt to update the table with same transaction identifier set through
TransactionBuilder.withTransactionId(Engine, String, long)
(String)}. - ConcurrentTransactionException(String, long, long) - Constructor for exception io.delta.kernel.exceptions.ConcurrentTransactionException
- ConcurrentWriteException - Exception in io.delta.kernel.exceptions
-
Thrown when a concurrent transaction has written data after the current transaction has started.
- ConcurrentWriteException() - Constructor for exception io.delta.kernel.exceptions.ConcurrentWriteException
- ConcurrentWriteException(String) - Constructor for exception io.delta.kernel.exceptions.ConcurrentWriteException
- CONFLICT_RESOLUTION_SCHEMA - Static variable in class io.delta.kernel.internal.actions.SingleAction
-
Schema to use when reading the winning commit files for conflict resolution.
- ConflictChecker - Class in io.delta.kernel.internal.replay
-
Class containing the conflict resolution logic when writing to a Delta table.
- ConflictChecker.TransactionRebaseState - Class in io.delta.kernel.internal.replay
-
Class containing the rebase state from winning transactions that the current transaction needs to rebase against before attempting the commit.
- conflictWithReservedInternalColumnName(String) - Static method in class io.delta.kernel.internal.DeltaErrors
- CONSTRAINTS_W_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- constructDataSkippingFilter(Predicate, StructType) - Static method in class io.delta.kernel.internal.skipping.DataSkippingUtils
-
Constructs a data skipping filter to prune files using column statistics given a query data filter if possible.
- contains(long) - Method in class io.delta.kernel.internal.deletionvectors.RoaringBitmapArray
-
Checks whether the value is included, which is equivalent to checking if the corresponding bit is set.
- contains(String) - Method in class io.delta.kernel.types.FieldMetadata
- containsAddOrRemoveFileActions(StructType) - Static method in class io.delta.kernel.internal.replay.LogReplay
- containsKey(Object) - Method in class io.delta.kernel.internal.util.CaseInsensitiveMap
- containsNull() - Method in class io.delta.kernel.types.ArrayType
- containsValue(Object) - Method in class io.delta.kernel.internal.util.CaseInsensitiveMap
- convertDataFileStatus(StructType, URI, DataFileStatus, Map<String, Literal>, boolean, Map<String, String>) - Static method in class io.delta.kernel.internal.actions.AddFile
-
Utility to generate
AddFile
action instance from the givenDataFileStatus
and partition values. - convertRemoveDataFileStatus(StructType, URI, DataFileStatus, Map<String, Literal>, boolean) - Static method in class io.delta.kernel.internal.actions.GenerateIcebergCompatActionUtils
- convertToPhysicalSchema(StructType, StructType, ColumnMapping.ColumnMappingMode) - Static method in class io.delta.kernel.internal.util.ColumnMapping
-
Helper method that converts the logical schema (requested by the connector) to physical schema of the data stored in data files based on the table's column mapping mode.
- count() - Method in class io.delta.kernel.internal.metrics.Timer
- Counter - Class in io.delta.kernel.internal.metrics
-
A long counter that uses
AtomicLong
to count events. - Counter() - Constructor for class io.delta.kernel.internal.metrics.Counter
- CRC_FILE_SCHEMA - Static variable in class io.delta.kernel.internal.checksum.CRCInfo
- CRCInfo - Class in io.delta.kernel.internal.checksum
- CRCInfo(long, Metadata, Protocol, long, long, Optional<String>, Optional<Set<DomainMetadata>>, Optional<FileSizeHistogram>) - Constructor for class io.delta.kernel.internal.checksum.CRCInfo
- create(long...) - Static method in class io.delta.kernel.internal.deletionvectors.RoaringBitmapArray
- CREATE_TABLE - Enum constant in enum io.delta.kernel.Operation
-
Recorded when the table is created.
- createAddFileRow(StructType, String, MapValue, long, long, boolean, Optional<DeletionVectorDescriptor>, Optional<MapValue>, Optional<Long>, Optional<Long>, Optional<DataFileStatistics>) - Static method in class io.delta.kernel.internal.actions.AddFile
-
Utility to generate an 'AddFile' row from the given fields.
- createAddFileSingleAction(Row) - Static method in class io.delta.kernel.internal.actions.SingleAction
- CreateCheckpointIterator - Class in io.delta.kernel.internal.replay
-
Replays a history of actions from the transaction log to reconstruct the checkpoint state of the table.
- CreateCheckpointIterator(Engine, LogSegment, long) - Constructor for class io.delta.kernel.internal.replay.CreateCheckpointIterator
- createCommitInfoSingleAction(Row) - Static method in class io.delta.kernel.internal.actions.SingleAction
- createDefaultHistogram() - Static method in class io.delta.kernel.internal.stats.FileSizeHistogram
-
Creates a default FileSizeHistogram with predefined bin boundaries and zero counts.
- createDomainMetadataSingleAction(Row) - Static method in class io.delta.kernel.internal.actions.SingleAction
- createMetadataSingleAction(Row) - Static method in class io.delta.kernel.internal.actions.SingleAction
- createPrimitive(String) - Static method in class io.delta.kernel.types.BasePrimitiveType
-
Create a primitive type
DataType
- createProtocolSingleAction(Row) - Static method in class io.delta.kernel.internal.actions.SingleAction
- createRemoveFileRowWithExtendedFileMetadata(String, long, boolean, MapValue, long, Optional<DataFileStatistics>, StructType) - Static method in class io.delta.kernel.internal.actions.GenerateIcebergCompatActionUtils
- createRemoveFileSingleAction(Row) - Static method in class io.delta.kernel.internal.actions.SingleAction
- createReplaceTableTransactionBuilder(Engine, String) - Method in class io.delta.kernel.internal.TableImpl
- createSelectionVector(boolean[], int, int) - Method in interface io.delta.kernel.engine.ExpressionHandler
-
Create a selection vector, a boolean type
ColumnVector
, on top of the range of values given in values array. - createTransactionBuilder(Engine, String, Operation) - Method in class io.delta.kernel.internal.TableImpl
- createTransactionBuilder(Engine, String, Operation) - Method in interface io.delta.kernel.Table
-
Create a
TransactionBuilder
which can create aTransaction
object to mutate the table. - createTxnSingleAction(Row) - Static method in class io.delta.kernel.internal.actions.SingleAction
- CUR_DIR - Static variable in class io.delta.kernel.internal.fs.Path
-
The current directory, ".".
D
- DATA_SKIPPING_NUM_INDEXED_COLS - Static variable in class io.delta.kernel.internal.TableConfig
-
The number of columns to collect stats on for data skipping.
- DataFileStatistics - Class in io.delta.kernel.statistics
-
Encapsulates statistics for a data file in a Delta Lake table and provides methods to serialize those stats to JSON with basic physical-type validation.
- DataFileStatistics(long, Map<Column, Literal>, Map<Column, Literal>, Map<Column, Long>) - Constructor for class io.delta.kernel.statistics.DataFileStatistics
-
Create a new instance of
DataFileStatistics
. - DataFileStatus - Class in io.delta.kernel.utils
-
Extends
FileStatus
to include additional details such as column level statistics of the data file in the Delta Lake table. - DataFileStatus(String, long, long, Optional<DataFileStatistics>) - Constructor for class io.delta.kernel.utils.DataFileStatus
-
Create a new instance of
DataFileStatus
. - dataSchemaMismatch(String, StructType, StructType) - Static method in class io.delta.kernel.internal.DeltaErrors
- DataSkippingPredicate - Class in io.delta.kernel.internal.skipping
-
A
Predicate
with a set of columns referenced by the expression. - DataSkippingUtils - Class in io.delta.kernel.internal.skipping
- DataSkippingUtils() - Constructor for class io.delta.kernel.internal.skipping.DataSkippingUtils
- DataType - Class in io.delta.kernel.types
-
Base class for all data types.
- DataType() - Constructor for class io.delta.kernel.types.DataType
- DataTypeJsonSerDe - Class in io.delta.kernel.internal.types
-
Serialize and deserialize Delta data types
DataType
to JSON and from JSON class based on the serialization rules outlined in the Delta Protocol. - DataWriteContext - Interface in io.delta.kernel
-
Contains the context for writing data to Delta table.
- DataWriteContextImpl - Class in io.delta.kernel.internal
-
Implements the
DataWriteContext
interface. - DataWriteContextImpl(String, Map<String, Literal>, List<Column>) - Constructor for class io.delta.kernel.internal.DataWriteContextImpl
-
Creates a new instance of WriteContext.
- DATE - Static variable in class io.delta.kernel.types.DateType
- DateTimeConstants - Class in io.delta.kernel.internal.util
- DateTimeConstants() - Constructor for class io.delta.kernel.internal.util.DateTimeConstants
- DateType - Class in io.delta.kernel.types
-
A date type, supporting "0001-01-01" through "9999-12-31".
- DAYS_PER_WEEK - Static variable in class io.delta.kernel.internal.util.DateTimeConstants
- daysSinceEpoch(Date) - Static method in class io.delta.kernel.internal.util.InternalUtils
-
Utility method to get the number of days since epoch this given date is.
- DecimalType - Class in io.delta.kernel.types
-
The data type representing
java.math.BigDecimal
values. - DecimalType(int, int) - Constructor for class io.delta.kernel.types.DecimalType
- DECODE_MAP - Static variable in class io.delta.kernel.internal.deletionvectors.Base85Codec
- decodeAlignedBytes(String) - Static method in class io.delta.kernel.internal.deletionvectors.Base85Codec
-
Decode an arbitrary byte array.
- decodeBytes(String, int) - Static method in class io.delta.kernel.internal.deletionvectors.Base85Codec
-
Decode an arbitrary byte array.
- decodeUUID(String) - Static method in class io.delta.kernel.internal.deletionvectors.Base85Codec
-
Decode a 16 byte UUID.
- DEFAULT_READ_VERSION - Static variable in class io.delta.kernel.internal.TransactionImpl
- DEFAULT_WRITE_VERSION - Static variable in class io.delta.kernel.internal.TransactionImpl
- DelegateRow - Class in io.delta.kernel.internal.data
-
This wraps an existing
Row
and allows overriding values for some particular ordinals. - DelegateRow(Row, Map<Integer, Object>) - Constructor for class io.delta.kernel.internal.data.DelegateRow
- delete(String) - Method in interface io.delta.kernel.engine.FileSystemClient
-
Delete the file at given path.
- DELETION_VECTORS_CREATION_ENABLED - Static variable in class io.delta.kernel.internal.TableConfig
-
Whether commands modifying this Delta table are allowed to create new deletion vectors.
- DELETION_VECTORS_RW_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- DeletionVectorDescriptor - Class in io.delta.kernel.internal.actions
-
Information about a deletion vector attached to a file action.
- DeletionVectorDescriptor(String, String, Optional<Integer>, int, long) - Constructor for class io.delta.kernel.internal.actions.DeletionVectorDescriptor
- DeletionVectorStoredBitmap - Class in io.delta.kernel.internal.deletionvectors
-
Bitmap for a Deletion Vector, implemented as a thin wrapper around a Deletion Vector Descriptor.
- DeletionVectorStoredBitmap(DeletionVectorDescriptor, Optional<String>) - Constructor for class io.delta.kernel.internal.deletionvectors.DeletionVectorStoredBitmap
- DeletionVectorUtils - Class in io.delta.kernel.internal.deletionvectors
-
Utility methods regarding deletion vectors.
- DeletionVectorUtils() - Constructor for class io.delta.kernel.internal.deletionvectors.DeletionVectorUtils
- DELTA - Enum constant in enum io.delta.kernel.internal.files.ParsedLogData.ParsedLogCategory
- DeltaErrors - Class in io.delta.kernel.internal
-
Contains methods to create user-facing Delta exceptions.
- DeltaErrors.SupplierWithIOException<T> - Interface in io.delta.kernel.internal
- DeltaErrorsInternal - Class in io.delta.kernel.internal
-
Contains methods to create developer-facing exceptions.
- deltaFile(Path, long) - Static method in class io.delta.kernel.internal.util.FileNames
-
Returns the delta (json format) path for a given delta file.
- DeltaHistoryManager - Class in io.delta.kernel.internal
- DeltaHistoryManager.Commit - Class in io.delta.kernel.internal
- DeltaLogActionUtils - Class in io.delta.kernel.internal
-
Exposes APIs to read the raw actions within the *commit files* of the _delta_log.
- DeltaLogActionUtils.DeltaAction - Enum in io.delta.kernel.internal
-
Represents a Delta action.
- DeltaLogFile - Class in io.delta.kernel.internal.replay
-
Internal wrapper class holding information needed to perform log replay.
- DeltaLogFile.LogType - Enum in io.delta.kernel.internal.replay
- DeltaOperationReport - Interface in io.delta.kernel.metrics
-
Defines the common fields that are shared by reports for Delta operations
- DeltaOperationReportImpl - Class in io.delta.kernel.internal.metrics
-
Basic POJO implementation of
DeltaOperationReport
- deltaVersion(Path) - Static method in class io.delta.kernel.internal.util.FileNames
-
Returns the version for the given delta path.
- deltaVersion(String) - Static method in class io.delta.kernel.internal.util.FileNames
- denormalized() - Method in class io.delta.kernel.internal.actions.Protocol
-
Protocol denormalization is the process of converting a legacy protocol to the equivalent table features protocol.
- denormalizedNormalized() - Method in class io.delta.kernel.internal.actions.Protocol
-
Helper method that applies both denormalization and normalization.
- deserializeFromJson(String) - Static method in class io.delta.kernel.internal.util.StatsUtils
-
Utility method to deserialize statistics from a JSON string.
- deserializeStructType(String) - Static method in class io.delta.kernel.internal.types.DataTypeJsonSerDe
-
Deserializes a JSON string representing a Delta data type to a
DataType
. - determineFileType(FileStatus) - Static method in class io.delta.kernel.internal.util.FileNames
- disablingIcebergWriterCompatV1OnExistingTable(String) - Static method in class io.delta.kernel.internal.DeltaErrors
- DOMAIN_METADATA_READ_SCHEMA - Static variable in class io.delta.kernel.internal.replay.LogReplay
-
Read schema when searching for just the domain metadata
- DOMAIN_METADATA_W_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- DOMAIN_NAME - Static variable in class io.delta.kernel.internal.clustering.ClusteringMetadataDomain
- DOMAIN_NAME - Static variable in class io.delta.kernel.internal.rowtracking.RowTrackingMetadataDomain
- DomainDoesNotExistException - Exception in io.delta.kernel.exceptions
-
Thrown when attempting to remove a domain metadata that does not exist in the read snapshot.
- DomainDoesNotExistException(String, String, long) - Constructor for exception io.delta.kernel.exceptions.DomainDoesNotExistException
- DomainMetadata - Class in io.delta.kernel.internal.actions
-
Delta log action representing an `DomainMetadata` action
- DomainMetadata(String, String, boolean) - Constructor for class io.delta.kernel.internal.actions.DomainMetadata
-
The domain metadata action contains a configuration string for a named metadata domain.
- domainMetadataUnsupported() - Static method in class io.delta.kernel.internal.DeltaErrors
- DomainMetadataUtils - Class in io.delta.kernel.internal.util
- DOUBLE - Static variable in class io.delta.kernel.types.DoubleType
- DoubleType - Class in io.delta.kernel.types
-
The data type representing
double
type values. - duplicateAddFilesCounter - Variable in class io.delta.kernel.internal.metrics.ScanMetrics
- duplicateColumnsInSchema(StructType, List<String>) - Static method in class io.delta.kernel.internal.DeltaErrors
E
- empty() - Static method in class io.delta.kernel.types.FieldMetadata
- empty(Path) - Static method in class io.delta.kernel.internal.snapshot.LogSegment
- emptyIterable() - Static method in interface io.delta.kernel.utils.CloseableIterable
-
Return an
CloseableIterable
object for an empty collection. - enablingClusteringOnPartitionedTableNotAllowed(String, Set<String>, List<Column>) - Static method in class io.delta.kernel.internal.DeltaErrors
- enablingIcebergWriterCompatV1OnExistingTable(String) - Static method in class io.delta.kernel.internal.DeltaErrors
- ENCODE_MAP - Static variable in class io.delta.kernel.internal.deletionvectors.Base85Codec
- encodeBytes(byte[]) - Static method in class io.delta.kernel.internal.deletionvectors.Base85Codec
-
Encode an arbitrary byte array.
- ENCODED_UUID_LENGTH - Static variable in class io.delta.kernel.internal.deletionvectors.Base85Codec
- encodeUUID(UUID) - Static method in class io.delta.kernel.internal.deletionvectors.Base85Codec
-
Encode a 16 byte UUID.
- endVersion - Variable in class io.delta.kernel.internal.files.ParsedLogCompactionData
- endVersionNotFound(String, long, long) - Static method in class io.delta.kernel.internal.DeltaErrors
- Engine - Interface in io.delta.kernel.engine
-
Interface encapsulating all clients needed by the Delta Kernel in order to read the Delta table.
- entrySet() - Method in class io.delta.kernel.internal.util.CaseInsensitiveMap
- equals(Object) - Method in class io.delta.kernel.expressions.Column
- equals(Object) - Method in class io.delta.kernel.expressions.Literal
- equals(Object) - Method in class io.delta.kernel.internal.actions.AddFile
- equals(Object) - Method in class io.delta.kernel.internal.actions.DeletionVectorDescriptor
- equals(Object) - Method in class io.delta.kernel.internal.actions.DomainMetadata
- equals(Object) - Method in class io.delta.kernel.internal.actions.Format
- equals(Object) - Method in class io.delta.kernel.internal.actions.Metadata
- equals(Object) - Method in class io.delta.kernel.internal.actions.Protocol
- equals(Object) - Method in class io.delta.kernel.internal.checkpoints.CheckpointInstance
- equals(Object) - Method in class io.delta.kernel.internal.checksum.CRCInfo
- equals(Object) - Method in class io.delta.kernel.internal.DeltaHistoryManager.Commit
- equals(Object) - Method in class io.delta.kernel.internal.files.ParsedLogCompactionData
- equals(Object) - Method in class io.delta.kernel.internal.files.ParsedLogData
- equals(Object) - Method in class io.delta.kernel.internal.files.ParsedMultiPartCheckpointData
- equals(Object) - Method in class io.delta.kernel.internal.fs.Path
- equals(Object) - Method in class io.delta.kernel.internal.rowtracking.RowTrackingMetadataDomain
- equals(Object) - Method in class io.delta.kernel.internal.stats.FileSizeHistogram
- equals(Object) - Method in class io.delta.kernel.internal.util.Tuple2
- equals(Object) - Method in class io.delta.kernel.statistics.DataFileStatistics
- equals(Object) - Method in class io.delta.kernel.types.ArrayType
- equals(Object) - Method in class io.delta.kernel.types.BasePrimitiveType
- equals(Object) - Method in class io.delta.kernel.types.CollationIdentifier
-
Collation identifiers are identical when the provider, name, and version are the same.
- equals(Object) - Method in class io.delta.kernel.types.DataType
- equals(Object) - Method in class io.delta.kernel.types.DecimalType
- equals(Object) - Method in class io.delta.kernel.types.FieldMetadata
- equals(Object) - Method in class io.delta.kernel.types.MapType
- equals(Object) - Method in class io.delta.kernel.types.StringType
- equals(Object) - Method in class io.delta.kernel.types.StructField
- equals(Object) - Method in class io.delta.kernel.types.StructType
- equals(Object) - Method in class io.delta.kernel.utils.FileStatus
- equivalent(DataType) - Method in class io.delta.kernel.types.ArrayType
- equivalent(DataType) - Method in class io.delta.kernel.types.DataType
-
Are the data types same? The metadata or column names could be different.
- equivalent(DataType) - Method in class io.delta.kernel.types.MapType
- equivalent(DataType) - Method in class io.delta.kernel.types.StructType
- eval(ColumnarBatch) - Method in interface io.delta.kernel.expressions.ExpressionEvaluator
-
Evaluate the expression on given
ColumnarBatch
data. - eval(ColumnarBatch, Optional<ColumnVector>) - Method in interface io.delta.kernel.expressions.PredicateEvaluator
-
Evaluate the predicate on given inputData.
- Evolving - Annotation Type in io.delta.kernel.annotation
-
APIs that are meant to evolve towards becoming stable APIs, but are not stable APIs yet.
- EXCLUDE - Enum constant in enum io.delta.kernel.utils.CloseableIterator.BreakableFilterResult
-
Indicates that the current element should be excluded from the resulting iterator produced by
CloseableIterator.breakableFilter(Function)
. - EXPIRED_LOG_CLEANUP_ENABLED - Static variable in class io.delta.kernel.internal.TableConfig
-
Whether to clean up expired checkpoints and delta logs.
- Expression - Interface in io.delta.kernel.expressions
-
Base interface for all Kernel expressions.
- ExpressionEvaluator - Interface in io.delta.kernel.expressions
-
Interface for implementing an
Expression
evaluator. - ExpressionHandler - Interface in io.delta.kernel.engine
-
Provides expression evaluation capability to Delta Kernel.
- ExpressionUtils - Class in io.delta.kernel.internal.util
- ExpressionUtils() - Constructor for class io.delta.kernel.internal.util.ExpressionUtils
- extractFeaturePropertyOverrides(Metadata) - Static method in class io.delta.kernel.internal.tablefeatures.TableFeatures
-
Extracts features overrides from Metadata properties and returns an updated metadata if any overrides are present.
- extractSidecarsFromBatch(FileStatus, long, ColumnarBatch) - Method in class io.delta.kernel.internal.replay.ActionsIterator
-
Reads SidecarFile actions from ColumnarBatch, removing sidecar actions from the ColumnarBatch.
F
- factory() - Static method in class io.delta.kernel.internal.util.JsonUtils
- FeatureAutoEnabledByMetadata - Interface in io.delta.kernel.internal.tablefeatures
-
Defines behavior for
TableFeature
that can be automatically enabled via a change in a table's metadata, e.g., through setting particular values of certain feature-specific table properties. - featureName() - Method in class io.delta.kernel.internal.tablefeatures.TableFeature
- FieldMetadata - Class in io.delta.kernel.types
-
The metadata for a given
StructField
. - FieldMetadata.Builder - Class in io.delta.kernel.types
-
Builder class for
FieldMetadata
. - fieldNames() - Method in class io.delta.kernel.types.StructType
- fields() - Method in class io.delta.kernel.types.StructType
- FileNames - Class in io.delta.kernel.internal.util
- FileNames.DeltaLogFileType - Enum in io.delta.kernel.internal.util
- filePath - Variable in class io.delta.kernel.internal.checkpoints.CheckpointInstance
- FileReadRequest - Interface in io.delta.kernel.engine
-
Represents a request to read a range of bytes from a given file.
- FileSizeHistogram - Class in io.delta.kernel.internal.stats
-
A histogram that tracks file size distributions and their counts.
- FileSizeHistogramResult - Interface in io.delta.kernel.metrics
-
Stores the file size histogram information to track file size distribution and their counts.
- FileStatus - Class in io.delta.kernel.utils
-
Class for encapsulating metadata about a file in Delta Lake table.
- fileStatusOpt - Variable in class io.delta.kernel.internal.files.ParsedLogData
- FileSystemClient - Interface in io.delta.kernel.engine
-
Provides file system related functionalities to Delta Kernel.
- filter(Function<T, Boolean>) - Method in interface io.delta.kernel.utils.CloseableIterator
-
Returns a new
CloseableIterator
that includes only the elements of this iterator for which the givenmapper
function returnstrue
. - FilteredColumnarBatch - Class in io.delta.kernel.data
-
Represents a filtered version of
ColumnarBatch
. - FilteredColumnarBatch(ColumnarBatch, Optional<ColumnVector>) - Constructor for class io.delta.kernel.data.FilteredColumnarBatch
- filterOutUnchangedProperties(Map<String, String>) - Method in class io.delta.kernel.internal.actions.Metadata
-
Filter out the key-value pair matches exactly with the old properties.
- filterRecursively(DataType, boolean, boolean, Function<StructField, Boolean>) - Static method in class io.delta.kernel.internal.util.SchemaUtils
-
Finds `StructField`s that match a given check `f`.
- findColIndex(StructType, String) - Static method in class io.delta.kernel.internal.util.SchemaUtils
-
Search (case-insensitive) for the given
colName
in theschema
and return its position in theschema
. - findLastCompleteCheckpointBefore(Engine, Path, long) - Static method in class io.delta.kernel.internal.checkpoints.Checkpointer
-
Find the last complete checkpoint before (strictly less than) a given version.
- findLastCompleteCheckpointBeforeHelper(Engine, Path, long) - Static method in class io.delta.kernel.internal.checkpoints.Checkpointer
-
Helper method for `findLastCompleteCheckpointBefore` which also return the number of files searched.
- FLOAT - Static variable in class io.delta.kernel.types.FloatType
- FloatType - Class in io.delta.kernel.types
-
The data type representing
float
type values. - forEach(Consumer<? super T>) - Method in interface io.delta.kernel.utils.CloseableIterable
- forError(SnapshotQueryContext, Exception) - Static method in class io.delta.kernel.internal.metrics.SnapshotReportImpl
-
Creates a
SnapshotReport
for a failed snapshot query. - forFileStatus(FileStatus) - Static method in class io.delta.kernel.internal.files.ParsedCheckpointData
- forFileStatus(FileStatus) - Static method in class io.delta.kernel.internal.files.ParsedLogCompactionData
- forFileStatus(FileStatus) - Static method in class io.delta.kernel.internal.files.ParsedLogData
- forFileStatus(FileStatus) - Static method in class io.delta.kernel.internal.files.ParsedMultiPartCheckpointData
- forFileStatus(FileStatus) - Static method in class io.delta.kernel.internal.replay.DeltaLogFile
- forInlineData(int, int, ColumnarBatch) - Static method in class io.delta.kernel.internal.files.ParsedLogCompactionData
- forInlineData(long, int, int, ColumnarBatch) - Static method in class io.delta.kernel.internal.files.ParsedMultiPartCheckpointData
- forInlineData(long, ParsedLogData.ParsedLogType, ColumnarBatch) - Static method in class io.delta.kernel.internal.files.ParsedCheckpointData
- forInlineData(long, ParsedLogData.ParsedLogType, ColumnarBatch) - Static method in class io.delta.kernel.internal.files.ParsedLogData
- forLatestSnapshot(String) - Static method in class io.delta.kernel.internal.metrics.SnapshotQueryContext
-
Creates a
SnapshotQueryContext
for a Snapshot created by a latest snapshot query - format - Variable in class io.delta.kernel.internal.checkpoints.CheckpointInstance
- Format - Class in io.delta.kernel.internal.actions
- Format() - Constructor for class io.delta.kernel.internal.actions.Format
- Format(String, Map<String, String>) - Constructor for class io.delta.kernel.internal.actions.Format
- FORMAT_HUDI - Static variable in class io.delta.kernel.internal.TableConfig.UniversalFormats
-
The value to use to enable uniform exports to Hudi for TableConfig.UNIVERSAL_FORMAT_ENABLED_FORMATS.
- FORMAT_ICEBERG - Static variable in class io.delta.kernel.internal.TableConfig.UniversalFormats
-
The value that enables uniform exports to Iceberg for TableConfig.UNIVERSAL_FORMAT_ENABLED_FORMATS.
- forNewTable() - Static method in class io.delta.kernel.internal.metrics.TransactionMetrics
- forPath(Engine, String) - Static method in class io.delta.kernel.internal.TableImpl
- forPath(Engine, String) - Static method in interface io.delta.kernel.Table
-
Instantiate a table object for the Delta Lake table at the given path.
- forPath(Engine, String, Clock) - Static method in class io.delta.kernel.internal.TableImpl
-
Instantiate a table object for the Delta Lake table at the given path.
- forSuccess(SnapshotQueryContext) - Static method in class io.delta.kernel.internal.metrics.SnapshotReportImpl
-
Creates a
SnapshotReport
for a successful snapshot query. - forTimestampSnapshot(String, long) - Static method in class io.delta.kernel.internal.metrics.SnapshotQueryContext
-
Creates a
SnapshotQueryContext
for a Snapshot created by a AS OF TIMESTAMP query - forVersionSnapshot(String, long) - Static method in class io.delta.kernel.internal.metrics.SnapshotQueryContext
-
Creates a
SnapshotQueryContext
for a Snapshot created by a AS OF VERSION query - fromClusteringColumns(List<Column>) - Static method in class io.delta.kernel.internal.clustering.ClusteringMetadataDomain
-
Constructs a ClusteringMetadataDomain with the clustering columns.
- fromColumnarBatch(long, ColumnarBatch, int, String) - Static method in class io.delta.kernel.internal.checksum.CRCInfo
- fromColumnVector(ColumnVector, int) - Static method in class io.delta.kernel.internal.actions.CommitInfo
- fromColumnVector(ColumnVector, int) - Static method in class io.delta.kernel.internal.actions.DeletionVectorDescriptor
- fromColumnVector(ColumnVector, int) - Static method in class io.delta.kernel.internal.actions.DomainMetadata
- fromColumnVector(ColumnVector, int) - Static method in class io.delta.kernel.internal.actions.Format
- fromColumnVector(ColumnVector, int) - Static method in class io.delta.kernel.internal.actions.Metadata
- fromColumnVector(ColumnVector, int) - Static method in class io.delta.kernel.internal.actions.Protocol
- fromColumnVector(ColumnVector, int) - Static method in class io.delta.kernel.internal.actions.SetTransaction
- fromColumnVector(ColumnVector, int) - Static method in class io.delta.kernel.internal.checkpoints.SidecarFile
- fromColumnVector(ColumnVector, int) - Static method in class io.delta.kernel.internal.stats.FileSizeHistogram
-
Creates a FileSizeHistogram from a column vector.
- fromCrcInfo(CRCInfo) - Static method in class io.delta.kernel.internal.snapshot.SnapshotHint
-
Constructs a new SnapshotHint based on a CRCInfo
- fromFileSizeHistogramResult(FileSizeHistogramResult) - Static method in class io.delta.kernel.internal.stats.FileSizeHistogram
- fromJsonConfiguration(String) - Static method in class io.delta.kernel.internal.clustering.ClusteringMetadataDomain
-
Creates an instance of
ClusteringMetadataDomain
from a JSON configuration string. - fromJsonConfiguration(String) - Static method in class io.delta.kernel.internal.rowtracking.RowTrackingMetadataDomain
-
Creates an instance of
RowTrackingMetadataDomain
from a JSON configuration string. - fromMetadata(Metadata) - Method in class io.delta.kernel.internal.TableConfig
-
Returns the value of the table property from the given metadata.
- fromMetadata(FieldMetadata) - Method in class io.delta.kernel.types.FieldMetadata.Builder
-
Adds all metadata from
meta.metadata
to the builder'smetadata
. - fromMetadata(Map<String, String>) - Method in class io.delta.kernel.internal.TableConfig
-
Returns the value of the table property from the given configuration.
- fromRow(Row) - Static method in class io.delta.kernel.internal.actions.DeletionVectorDescriptor
- fromRow(Row) - Static method in class io.delta.kernel.internal.actions.DomainMetadata
-
Creates a
DomainMetadata
instance from a Row with the schema beingDomainMetadata.FULL_SCHEMA
. - fromRow(Row) - Static method in class io.delta.kernel.internal.actions.Metadata
- fromRow(Row) - Static method in class io.delta.kernel.internal.actions.Protocol
-
Helper method to get the Protocol from the row representation.
- fromRow(Row) - Static method in class io.delta.kernel.internal.checkpoints.CheckpointMetaData
- fromSnapshot(SnapshotImpl) - Static method in class io.delta.kernel.internal.clustering.ClusteringMetadataDomain
-
Creates an optional instance of
ClusteringMetadataDomain
from aSnapshotImpl
if present. - fromSnapshot(SnapshotImpl) - Static method in class io.delta.kernel.internal.rowtracking.RowTrackingMetadataDomain
-
Creates an instance of
RowTrackingMetadataDomain
from aSnapshotImpl
. - fromString(String) - Static method in class io.delta.kernel.types.CollationIdentifier
- fromStructVector(ColumnVector, int) - Static method in class io.delta.kernel.internal.data.StructRow
- fromTableConfig(String) - Static method in enum io.delta.kernel.internal.util.ColumnMapping.ColumnMappingMode
- FULL_SCHEMA - Static variable in class io.delta.kernel.internal.actions.AddCDCFile
-
Full schema of the
cdc
action in the Delta Log. - FULL_SCHEMA - Static variable in class io.delta.kernel.internal.actions.AddFile
-
Full schema of the
add
action in the Delta Log. - FULL_SCHEMA - Static variable in class io.delta.kernel.internal.actions.CommitInfo
- FULL_SCHEMA - Static variable in class io.delta.kernel.internal.actions.DomainMetadata
-
Full schema of the
DomainMetadata
action in the Delta Log. - FULL_SCHEMA - Static variable in class io.delta.kernel.internal.actions.Format
- FULL_SCHEMA - Static variable in class io.delta.kernel.internal.actions.Metadata
- FULL_SCHEMA - Static variable in class io.delta.kernel.internal.actions.Protocol
- FULL_SCHEMA - Static variable in class io.delta.kernel.internal.actions.RemoveFile
-
Full schema of the
remove
action in the Delta Log. - FULL_SCHEMA - Static variable in class io.delta.kernel.internal.actions.SetTransaction
- FULL_SCHEMA - Static variable in class io.delta.kernel.internal.actions.SingleAction
- FULL_SCHEMA - Static variable in class io.delta.kernel.internal.stats.FileSizeHistogram
G
- generate(JsonGenerator) - Method in interface io.delta.kernel.internal.util.JsonUtils.ToJson
- generate(JsonUtils.ToJson) - Static method in class io.delta.kernel.internal.util.JsonUtils
-
Utility class for writing JSON with a Jackson
JsonGenerator
. - generateAppendActions(Engine, Row, CloseableIterator<DataFileStatus>, DataWriteContext) - Static method in interface io.delta.kernel.Transaction
-
For given data files, generate Delta actions that can be committed in a transaction.
- GENERATED_COLUMNS_W_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- GenerateIcebergCompatActionUtils - Class in io.delta.kernel.internal.actions
-
Utilities to convert Iceberg add/removes to Delta Kernel add/removes
- GenerateIcebergCompatActionUtils() - Constructor for class io.delta.kernel.internal.actions.GenerateIcebergCompatActionUtils
- generateIcebergCompatWriterV1AddAction(Row, DataFileStatus, Map<String, Literal>, boolean) - Static method in class io.delta.kernel.internal.actions.GenerateIcebergCompatActionUtils
- generateIcebergCompatWriterV1AddAction(Row, DataFileStatus, Map<String, Literal>, boolean, Map<String, String>) - Static method in class io.delta.kernel.internal.actions.GenerateIcebergCompatActionUtils
-
Create an add action
Row
that can be passed toTransaction.commit(Engine, CloseableIterable)
from an Iceberg add. - generateIcebergCompatWriterV1RemoveAction(Row, DataFileStatus, Map<String, Literal>, boolean) - Static method in class io.delta.kernel.internal.actions.GenerateIcebergCompatActionUtils
-
Create a remove action
Row
that can be passed toTransaction.commit(Engine, CloseableIterable)
from an Iceberg remove. - generateScanFileRow(FileStatus) - Static method in class io.delta.kernel.internal.InternalScanFileUtils
-
Create a scan file row conforming to the schema
InternalScanFileUtils.SCAN_FILE_SCHEMA
for given file status. - GenericColumnVector - Class in io.delta.kernel.internal.data
-
A generic implementation of
ColumnVector
that wraps a list of values. - GenericColumnVector(List<?>, DataType) - Constructor for class io.delta.kernel.internal.data.GenericColumnVector
- GenericRow - Class in io.delta.kernel.internal.data
-
Exposes a given map of values as a
Row
- GenericRow(StructType, Map<Integer, Object>) - Constructor for class io.delta.kernel.internal.data.GenericRow
- get() - Method in interface io.delta.kernel.internal.DeltaErrors.SupplierWithIOException
- get() - Method in class io.delta.kernel.internal.lang.Lazy
- get(Object) - Method in class io.delta.kernel.internal.util.CaseInsensitiveMap
- get(String) - Method in class io.delta.kernel.types.FieldMetadata
- get(String) - Method in class io.delta.kernel.types.StructType
- getAbsolutePath(String) - Method in class io.delta.kernel.internal.actions.DeletionVectorDescriptor
- getActiveCommitAtTimestamp(Engine, Path, long, boolean, boolean, boolean) - Static method in class io.delta.kernel.internal.DeltaHistoryManager
-
Returns the latest commit that happened at or before
timestamp
. - getActiveDomainMetadataMap() - Method in class io.delta.kernel.internal.replay.LogReplay
- getActiveDomainMetadataMap() - Method in class io.delta.kernel.internal.SnapshotImpl
-
Get the domain metadata map from the log replay, which lazily loads and replays a history of domain metadata actions, resolving them to produce the current state of the domain metadata.
- getAddFileDV(ColumnVector, int) - Static method in class io.delta.kernel.internal.replay.ActiveAddFilesIterator
- getAddFilePath(ColumnVector, int) - Static method in class io.delta.kernel.internal.replay.ActiveAddFilesIterator
- getAddFilesAsColumnarBatches(Engine, boolean, Optional<Predicate>, ScanMetrics) - Method in class io.delta.kernel.internal.replay.LogReplay
-
Returns an iterator of
FilteredColumnarBatch
representing all the active AddFiles in the table. - getAddFileStatus(Row) - Static method in class io.delta.kernel.internal.InternalScanFileUtils
- getAddReadSchema(boolean) - Static method in class io.delta.kernel.internal.replay.LogReplay
-
Read schema when searching only for AddFiles
- getAddRemoveReadSchema(boolean) - Static method in class io.delta.kernel.internal.replay.LogReplay
-
Read schema when searching for all the active AddFiles
- getAllPrimitiveTypes() - Static method in class io.delta.kernel.types.BasePrimitiveType
-
For testing only
- getAppId() - Method in class io.delta.kernel.internal.actions.SetTransaction
- getArray(int) - Method in interface io.delta.kernel.data.ColumnVector
-
Return the array value located at
rowId
. - getArray(int) - Method in interface io.delta.kernel.data.Row
-
Return array value of the column located at the given ordinal.
- getArray(int) - Method in class io.delta.kernel.internal.data.ChildVectorBasedRow
- getArray(int) - Method in class io.delta.kernel.internal.data.DelegateRow
- getArray(int) - Method in class io.delta.kernel.internal.data.GenericColumnVector
- getArray(int) - Method in class io.delta.kernel.internal.data.GenericRow
- getBaseRowId() - Method in class io.delta.kernel.internal.actions.AddFile
- getBaseRowId() - Method in class io.delta.kernel.internal.actions.RemoveFile
- getBaseRowId(Row) - Static method in class io.delta.kernel.internal.InternalScanFileUtils
- getBaseSnapshotVersion() - Method in class io.delta.kernel.internal.metrics.TransactionReportImpl
- getBaseSnapshotVersion() - Method in interface io.delta.kernel.metrics.TransactionReport
-
The version of the table the transaction was created from.
- getBinary(int) - Method in interface io.delta.kernel.data.ColumnVector
-
Returns the binary type value for
rowId
. - getBinary(int) - Method in interface io.delta.kernel.data.Row
-
Return binary value of the column located at the given ordinal.
- getBinary(int) - Method in class io.delta.kernel.internal.data.ChildVectorBasedRow
- getBinary(int) - Method in class io.delta.kernel.internal.data.DelegateRow
- getBinary(int) - Method in class io.delta.kernel.internal.data.GenericColumnVector
- getBinary(int) - Method in class io.delta.kernel.internal.data.GenericRow
- getBoolean(int) - Method in interface io.delta.kernel.data.ColumnVector
-
Returns the boolean type value for
rowId
. - getBoolean(int) - Method in interface io.delta.kernel.data.Row
-
Return boolean value of the column located at the given ordinal.
- getBoolean(int) - Method in class io.delta.kernel.internal.data.ChildVectorBasedRow
- getBoolean(int) - Method in class io.delta.kernel.internal.data.DelegateRow
- getBoolean(int) - Method in class io.delta.kernel.internal.data.GenericColumnVector
- getBoolean(int) - Method in class io.delta.kernel.internal.data.GenericRow
- getBoolean(int) - Method in class io.delta.kernel.internal.data.SelectionColumnVector
- getBoolean(String) - Method in class io.delta.kernel.types.FieldMetadata
- getBooleanArray(String) - Method in class io.delta.kernel.types.FieldMetadata
- getByte(int) - Method in interface io.delta.kernel.data.ColumnVector
-
Returns the byte type value for
rowId
. - getByte(int) - Method in interface io.delta.kernel.data.Row
-
Return byte value of the column located at the given ordinal.
- getByte(int) - Method in class io.delta.kernel.internal.data.ChildVectorBasedRow
- getByte(int) - Method in class io.delta.kernel.internal.data.DelegateRow
- getByte(int) - Method in class io.delta.kernel.internal.data.GenericColumnVector
- getByte(int) - Method in class io.delta.kernel.internal.data.GenericRow
- getCardinality() - Method in class io.delta.kernel.internal.actions.DeletionVectorDescriptor
- getCategory() - Method in class io.delta.kernel.internal.files.ParsedLogData
- getChanges(Engine, long, long, Set<DeltaLogActionUtils.DeltaAction>) - Method in class io.delta.kernel.internal.TableImpl
-
Returns delta actions for each version between startVersion and endVersion.
- getCheckpoints() - Method in class io.delta.kernel.internal.snapshot.LogSegment
- getCheckpointVersionOpt() - Method in class io.delta.kernel.internal.snapshot.LogSegment
- getChild(int) - Method in interface io.delta.kernel.data.ColumnVector
-
Get the child vector associated with the given ordinal.
- getChild(int) - Method in class io.delta.kernel.internal.data.GenericColumnVector
- getChildren() - Method in class io.delta.kernel.expressions.Column
- getChildren() - Method in interface io.delta.kernel.expressions.Expression
- getChildren() - Method in class io.delta.kernel.expressions.Literal
- getChildren() - Method in class io.delta.kernel.expressions.PartitionValueExpression
- getChildren() - Method in class io.delta.kernel.expressions.ScalarExpression
- getClock() - Method in class io.delta.kernel.internal.TableImpl
- getClusteringColumns() - Method in class io.delta.kernel.internal.clustering.ClusteringMetadataDomain
- getClusteringColumnsOptional(SnapshotImpl) - Static method in class io.delta.kernel.internal.clustering.ClusteringUtils
-
Extract ClusteringColumns from a given snapshot.
- getClusteringDomainMetadata(List<Column>) - Static method in class io.delta.kernel.internal.clustering.ClusteringUtils
-
Get the domain metadata for the clustering columns.
- getCollationIdentifier() - Method in class io.delta.kernel.types.StringType
- getColumnarBatch() - Method in class io.delta.kernel.internal.replay.ActionWrapper
- getColumnId(StructField) - Static method in class io.delta.kernel.internal.util.ColumnMapping
-
Returns the column id for a given
StructField
- getColumnMappingMode(Row) - Static method in class io.delta.kernel.internal.data.ScanStateRow
-
Get the column mapping mode from the scan state
Row
returned byScan.getScanState(Engine)
. - getColumnMappingMode(Row) - Static method in class io.delta.kernel.internal.data.TransactionStateRow
-
Get the column mapping mode from the transaction state
Row
returned byTransaction.getTransactionState(Engine)
- getColumnMappingMode(Map<String, String>) - Static method in class io.delta.kernel.internal.util.ColumnMapping
-
Returns the column mapping mode from the given configuration.
- getColumnVector(int) - Method in interface io.delta.kernel.data.ColumnarBatch
-
Return the
ColumnVector
for the given ordinal in the columnar batch. - getCommitFilesForVersionRange(Engine, Path, long, long) - Static method in class io.delta.kernel.internal.DeltaLogActionUtils
-
For a table get the list of commit log files for the provided version range.
- getCommitInfoOpt(Engine, Path, long) - Static method in class io.delta.kernel.internal.actions.CommitInfo
-
Get the persisted commit info (if available) for the given delta file.
- getCommittedVersion() - Method in class io.delta.kernel.internal.metrics.TransactionReportImpl
- getCommittedVersion() - Method in interface io.delta.kernel.metrics.TransactionReport
- getCommitVersion() - Method in class io.delta.kernel.internal.hook.LogCompactionHook
- getCompactions() - Method in class io.delta.kernel.internal.snapshot.LogSegment
- getConfiguration() - Method in class io.delta.kernel.internal.actions.DomainMetadata
- getConfiguration() - Method in class io.delta.kernel.internal.actions.Metadata
- getConfiguration(Row) - Static method in class io.delta.kernel.internal.data.TransactionStateRow
-
Get the configuration from the transaction state
Row
returned byTransaction.getTransactionState(Engine)
- getConfigurationMapValue() - Method in class io.delta.kernel.internal.actions.Metadata
- getCorrespondingFiles(Path) - Method in class io.delta.kernel.internal.checkpoints.CheckpointInstance
- getCRCInfo(Engine, FileStatus) - Static method in class io.delta.kernel.internal.checksum.ChecksumReader
-
Load the CRCInfo from the provided checksum file.
- getCreateCheckpointIterator(Engine) - Method in class io.delta.kernel.internal.SnapshotImpl
- getCreatedTime() - Method in class io.delta.kernel.internal.actions.Metadata
- getCurrentCrcInfo() - Method in class io.delta.kernel.internal.replay.LogReplay
-
Returns the crc info for the current snapshot if it is cached
- getCurrentCrcInfo() - Method in class io.delta.kernel.internal.SnapshotImpl
-
Returns the crc info for the current snapshot if the checksum file is read
- getData() - Method in class io.delta.kernel.data.FilteredColumnarBatch
-
Return the data as
ColumnarBatch
. - getDataChange() - Method in class io.delta.kernel.internal.actions.AddFile
- getDataChange() - Method in class io.delta.kernel.internal.actions.RemoveFile
- getDataPath() - Method in class io.delta.kernel.internal.hook.LogCompactionHook
- getDataPath() - Method in class io.delta.kernel.internal.SnapshotImpl
- getDataSchema() - Method in class io.delta.kernel.internal.actions.Metadata
-
The logical data schema which excludes partition columns
- getDataSkippingFilter() - Method in class io.delta.kernel.internal.metrics.ScanReportImpl
- getDataSkippingFilter() - Method in interface io.delta.kernel.metrics.ScanReport
- getDataType() - Method in interface io.delta.kernel.data.ColumnVector
- getDataType() - Method in class io.delta.kernel.expressions.Literal
-
Get the datatype of the literal object.
- getDataType() - Method in class io.delta.kernel.expressions.PartitionValueExpression
-
Get the data type of the partition value.
- getDataType() - Method in class io.delta.kernel.internal.data.GenericColumnVector
- getDataType() - Method in class io.delta.kernel.internal.data.SelectionColumnVector
- getDataType() - Method in class io.delta.kernel.types.StructField
- getDecimal(int) - Method in interface io.delta.kernel.data.ColumnVector
-
Returns the decimal type value for
rowId
. - getDecimal(int) - Method in interface io.delta.kernel.data.Row
-
Return decimal value of the column located at the given ordinal.
- getDecimal(int) - Method in class io.delta.kernel.internal.data.ChildVectorBasedRow
- getDecimal(int) - Method in class io.delta.kernel.internal.data.DelegateRow
- getDecimal(int) - Method in class io.delta.kernel.internal.data.GenericColumnVector
- getDecimal(int) - Method in class io.delta.kernel.internal.data.GenericRow
- getDefaultRowCommitVersion() - Method in class io.delta.kernel.internal.actions.AddFile
- getDefaultRowCommitVersion() - Method in class io.delta.kernel.internal.actions.RemoveFile
- getDefaultRowCommitVersion(Row) - Static method in class io.delta.kernel.internal.InternalScanFileUtils
- getDeletionTimestamp() - Method in class io.delta.kernel.internal.actions.RemoveFile
- getDeletionVector() - Method in class io.delta.kernel.internal.actions.AddFile
- getDeletionVector() - Method in class io.delta.kernel.internal.actions.RemoveFile
- getDeletionVectorDescriptorFromRow(Row) - Static method in class io.delta.kernel.internal.InternalScanFileUtils
-
Create a
DeletionVectorDescriptor
fromadd
entry in the given scan file row. - getDeltas() - Method in class io.delta.kernel.internal.snapshot.LogSegment
- getDescription() - Method in class io.delta.kernel.internal.actions.Metadata
- getDescription() - Method in enum io.delta.kernel.Operation
-
Returns the string that will be recorded in the transaction log.
- getDomain() - Method in class io.delta.kernel.internal.actions.DomainMetadata
- getDomainMetadata() - Method in class io.delta.kernel.internal.checksum.CRCInfo
- getDomainMetadata(String) - Method in class io.delta.kernel.internal.SnapshotImpl
- getDomainMetadata(String) - Method in interface io.delta.kernel.Snapshot
-
Returns the configuration for the provided
domain
if it exists in the snapshot. - getDomainName() - Method in class io.delta.kernel.internal.clustering.ClusteringMetadataDomain
- getDomainName() - Method in class io.delta.kernel.internal.metadatadomain.JsonMetadataDomain
-
Returns the name of the domain.
- getDomainName() - Method in class io.delta.kernel.internal.rowtracking.RowTrackingMetadataDomain
- getDouble(int) - Method in interface io.delta.kernel.data.ColumnVector
-
Returns the double type value for
rowId
. - getDouble(int) - Method in interface io.delta.kernel.data.Row
-
Return double value of the column located at the given ordinal.
- getDouble(int) - Method in class io.delta.kernel.internal.data.ChildVectorBasedRow
- getDouble(int) - Method in class io.delta.kernel.internal.data.DelegateRow
- getDouble(int) - Method in class io.delta.kernel.internal.data.GenericColumnVector
- getDouble(int) - Method in class io.delta.kernel.internal.data.GenericRow
- getDouble(String) - Method in class io.delta.kernel.types.FieldMetadata
- getDoubleArray(String) - Method in class io.delta.kernel.types.FieldMetadata
- getEarliestDeltaFile(Engine, Path) - Static method in class io.delta.kernel.internal.DeltaHistoryManager
-
Get the earliest commit available for this table.
- getEarliestRecreatableCommit(Engine, Path) - Static method in class io.delta.kernel.internal.DeltaHistoryManager
-
Gets the earliest commit that we can recreate.
- getElementField() - Method in class io.delta.kernel.types.ArrayType
- getElements() - Method in interface io.delta.kernel.data.ArrayValue
-
A
ColumnVector
containing the array elements with exactlyArrayValue.getSize()
elements. - getElementType() - Method in class io.delta.kernel.types.ArrayType
- getEngineInfo() - Method in class io.delta.kernel.internal.actions.CommitInfo
- getEngineInfo() - Method in class io.delta.kernel.internal.metrics.TransactionReportImpl
- getEngineInfo() - Method in interface io.delta.kernel.metrics.TransactionReport
- getEntries() - Method in class io.delta.kernel.types.FieldMetadata
- getEvaluator(StructType, Expression, DataType) - Method in interface io.delta.kernel.engine.ExpressionHandler
-
Create an
ExpressionEvaluator
that can evaluate the given expression onColumnarBatch
s with the given batchSchema. - getException() - Method in class io.delta.kernel.internal.metrics.DeltaOperationReportImpl
- getException() - Method in interface io.delta.kernel.metrics.DeltaOperationReport
- getExplicitlySupportedFeatures() - Method in class io.delta.kernel.internal.actions.Protocol
-
Get the set of features that are explicitly supported by the protocol.
- getExpressionHandler() - Method in interface io.delta.kernel.engine.Engine
-
Get the connector provided
ExpressionHandler
. - getExtendedFileMetadata() - Method in class io.delta.kernel.internal.actions.RemoveFile
- getFile() - Method in class io.delta.kernel.internal.replay.DeltaLogFile
- getFileCounts() - Method in interface io.delta.kernel.metrics.FileSizeHistogramResult
-
The total number of files in each bin of
FileSizeHistogramResult.getSortedBinBoundaries()
- getFileSizeHistogram() - Method in class io.delta.kernel.internal.checksum.CRCInfo
-
The
FileSizeHistogram
stored in this CRCInfo. - getFileStatus() - Method in class io.delta.kernel.internal.files.ParsedLogData
-
Callers must check
ParsedLogData.isMaterialized()
before calling this method. - getFileSystemClient() - Method in interface io.delta.kernel.engine.Engine
-
Get the connector provided
FileSystemClient
. - getFileVersion(Path) - Static method in class io.delta.kernel.internal.util.FileNames
-
Get the version of the checkpoint, checksum or delta file.
- getFilter() - Method in class io.delta.kernel.internal.metrics.ScanReportImpl
- getFilter() - Method in interface io.delta.kernel.metrics.ScanReport
- getFirst(List<T>) - Static method in class io.delta.kernel.internal.lang.ListUtils
-
Remove once supported JDK (build) version is 21 or above
- getFloat(int) - Method in interface io.delta.kernel.data.ColumnVector
-
Returns the float type value for
rowId
. - getFloat(int) - Method in interface io.delta.kernel.data.Row
-
Return float value of the column located at the given ordinal.
- getFloat(int) - Method in class io.delta.kernel.internal.data.ChildVectorBasedRow
- getFloat(int) - Method in class io.delta.kernel.internal.data.DelegateRow
- getFloat(int) - Method in class io.delta.kernel.internal.data.GenericColumnVector
- getFloat(int) - Method in class io.delta.kernel.internal.data.GenericRow
- getFormat() - Method in class io.delta.kernel.internal.actions.Metadata
- getId() - Method in class io.delta.kernel.internal.actions.Metadata
- getImplicitlyAndExplicitlySupportedFeatures() - Method in class io.delta.kernel.internal.actions.Protocol
-
Get the set of features that are both implicitly and explicitly supported by the protocol.
- getImplicitlyAndExplicitlySupportedReaderWriterFeatures() - Method in class io.delta.kernel.internal.actions.Protocol
-
Get the set of reader writer features that are both implicitly and explicitly supported by the protocol.
- getImplicitlySupportedFeatures() - Method in class io.delta.kernel.internal.actions.Protocol
-
Get the set of features that are implicitly supported by the protocol.
- getInCommitTimestamp() - Method in class io.delta.kernel.internal.actions.CommitInfo
- getInlineData() - Method in class io.delta.kernel.internal.files.ParsedLogData
-
Callers must check
ParsedLogData.isInline()
before calling this method. - getInput() - Method in class io.delta.kernel.expressions.PartitionValueExpression
-
Get the expression reference to the serialized partition value.
- getInt(int) - Method in interface io.delta.kernel.data.ColumnVector
-
Returns the int type value for
rowId
. - getInt(int) - Method in interface io.delta.kernel.data.Row
-
Return integer value of the column located at the given ordinal.
- getInt(int) - Method in class io.delta.kernel.internal.data.ChildVectorBasedRow
- getInt(int) - Method in class io.delta.kernel.internal.data.DelegateRow
- getInt(int) - Method in class io.delta.kernel.internal.data.GenericColumnVector
- getInt(int) - Method in class io.delta.kernel.internal.data.GenericRow
- getIsBlindAppend() - Method in class io.delta.kernel.internal.actions.CommitInfo
- getIsFullyConsumed() - Method in class io.delta.kernel.internal.metrics.ScanReportImpl
- getIsFullyConsumed() - Method in interface io.delta.kernel.metrics.ScanReport
-
Whether the scan file iterator had been fully consumed when it was closed.
- getJsonHandler() - Method in interface io.delta.kernel.engine.Engine
-
Get the connector provided
JsonHandler
. - getKey() - Method in class io.delta.kernel.internal.TableConfig
-
Returns the key of the table property.
- getKeyField() - Method in class io.delta.kernel.types.MapType
- getKeys() - Method in interface io.delta.kernel.data.MapValue
-
A
ColumnVector
containing the keys. - getKeyType() - Method in class io.delta.kernel.types.MapType
- getLast(List<T>) - Static method in class io.delta.kernel.internal.lang.ListUtils
-
Remove once supported JDK (build) version is 21 or above
- getLastCommitTimestamp() - Method in class io.delta.kernel.internal.snapshot.LogSegment
- getLastSeenChecksum() - Method in class io.delta.kernel.internal.snapshot.LogSegment
-
Returns the most recent checksum file encountered during log directory listing, if available.
- getLastUpdated() - Method in class io.delta.kernel.internal.actions.SetTransaction
- getLatestCommitTimestamp() - Method in class io.delta.kernel.internal.replay.ConflictChecker.TransactionRebaseState
-
Return the latest commit timestamp of the table.
- getLatestCompleteCheckpointFromList(List<CheckpointInstance>, CheckpointInstance) - Static method in class io.delta.kernel.internal.checkpoints.Checkpointer
-
Given a list of checkpoint files, pick the latest complete checkpoint instance which is not later than `notLaterThan`.
- getLatestSnapshot(Engine) - Method in class io.delta.kernel.internal.TableImpl
- getLatestSnapshot(Engine) - Method in interface io.delta.kernel.Table
-
Get the latest snapshot of the table.
- getLatestTransactionIdentifier(Engine, String) - Method in class io.delta.kernel.internal.replay.LogReplay
- getLatestTransactionVersion(Engine, String) - Method in class io.delta.kernel.internal.SnapshotImpl
-
Get the latest transaction version for given applicationId.
- getLatestVersion() - Method in class io.delta.kernel.internal.replay.ConflictChecker.TransactionRebaseState
-
Return the latest winning version of the table.
- getLeft() - Method in class io.delta.kernel.expressions.And
- getLeft() - Method in class io.delta.kernel.expressions.Or
- getLeft(Expression) - Static method in class io.delta.kernel.internal.util.ExpressionUtils
-
Utility method to return the left child of the binary input expression
- getLoadInitialDeltaActionsDurationNs() - Method in interface io.delta.kernel.metrics.SnapshotMetricsResult
- getLogicalSchema(Row) - Static method in class io.delta.kernel.internal.data.ScanStateRow
-
Utility method to get the logical schema from the scan state
Row
returned byScan.getScanState(Engine)
. - getLogicalSchema(Row) - Static method in class io.delta.kernel.internal.data.TransactionStateRow
-
Get the logical schema of the table from the transaction state
Row
returned byTransaction.getTransactionState(Engine)
} - getLogPath() - Method in class io.delta.kernel.internal.hook.LogCompactionHook
- getLogPath() - Method in class io.delta.kernel.internal.snapshot.LogSegment
- getLogPath() - Method in class io.delta.kernel.internal.SnapshotImpl
- getLogSegment() - Method in class io.delta.kernel.internal.SnapshotImpl
- getLogSegmentForVersion(Engine, Optional<Long>) - Method in class io.delta.kernel.internal.snapshot.SnapshotManager
-
Generates a
LogSegment
for the given `versionToLoadOpt`. - getLogType() - Method in class io.delta.kernel.internal.replay.DeltaLogFile
- getLong(int) - Method in interface io.delta.kernel.data.ColumnVector
-
Returns the long type value for
rowId
. - getLong(int) - Method in interface io.delta.kernel.data.Row
-
Return long value of the column located at the given ordinal.
- getLong(int) - Method in class io.delta.kernel.internal.data.ChildVectorBasedRow
- getLong(int) - Method in class io.delta.kernel.internal.data.DelegateRow
- getLong(int) - Method in class io.delta.kernel.internal.data.GenericColumnVector
- getLong(int) - Method in class io.delta.kernel.internal.data.GenericRow
- getLong(String) - Method in class io.delta.kernel.types.FieldMetadata
- getLongArray(String) - Method in class io.delta.kernel.types.FieldMetadata
- getMap(int) - Method in interface io.delta.kernel.data.ColumnVector
-
Return the map value located at
rowId
. - getMap(int) - Method in interface io.delta.kernel.data.Row
-
Return map value of the column located at the given ordinal.
- getMap(int) - Method in class io.delta.kernel.internal.data.ChildVectorBasedRow
- getMap(int) - Method in class io.delta.kernel.internal.data.DelegateRow
- getMap(int) - Method in class io.delta.kernel.internal.data.GenericColumnVector
- getMap(int) - Method in class io.delta.kernel.internal.data.GenericRow
- getMaterializedColumnNamePrefix() - Method in class io.delta.kernel.internal.rowtracking.MaterializedRowTrackingColumn
-
Returns the prefix to use for generating the materialized column name.
- getMaterializedColumnNameProperty() - Method in class io.delta.kernel.internal.rowtracking.MaterializedRowTrackingColumn
-
Returns the configuration property name associated with this materialized column.
- getMaxColumn(Column) - Method in class io.delta.kernel.internal.skipping.StatsSchemaHelper
-
Given a logical column in the data schema provided when creating
this
, return the corresponding MAX column and an optional column adjustment expression from the statistic schema that stores the MAX values for the provided logical column. - getMaxRetries(Row) - Static method in class io.delta.kernel.internal.data.TransactionStateRow
-
Get the maxRetries from transaction state
Row
returned byTransaction.getTransactionState(Engine)
- getMaxValues() - Method in class io.delta.kernel.statistics.DataFileStatistics
-
Get the maximum values of the columns in the data file.
- getMessage() - Method in exception io.delta.kernel.exceptions.TableAlreadyExistsException
- getMetadata() - Method in class io.delta.kernel.internal.checksum.CRCInfo
-
The
Metadata
stored in this CRCInfo. - getMetadata() - Method in class io.delta.kernel.internal.replay.LogReplay
- getMetadata() - Method in class io.delta.kernel.internal.snapshot.SnapshotHint
- getMetadata() - Method in class io.delta.kernel.internal.SnapshotImpl
- getMetadata() - Method in class io.delta.kernel.types.StructField
- getMetadata(String) - Method in class io.delta.kernel.types.FieldMetadata.Builder
- getMetadata(String) - Method in class io.delta.kernel.types.FieldMetadata
- getMetadataArray(String) - Method in class io.delta.kernel.types.FieldMetadata
- getMetricsReporters() - Method in interface io.delta.kernel.engine.Engine
-
Get the engine's
MetricsReporter
instances to push reports to. - getMinColumn(Column) - Method in class io.delta.kernel.internal.skipping.StatsSchemaHelper
-
Given a logical column in the data schema provided when creating
this
, return the corresponding MIN column and an optional column adjustment expression from the statistic schema that stores the MIN values for the provided logical column. - getMinFileRetentionTimestampMillis() - Method in class io.delta.kernel.internal.hook.LogCompactionHook
- getMinReaderVersion() - Method in class io.delta.kernel.internal.actions.Protocol
- getMinValues() - Method in class io.delta.kernel.statistics.DataFileStatistics
-
Get the minimum values of the columns in the data file.
- getMinWriterVersion() - Method in class io.delta.kernel.internal.actions.Protocol
- getModificationTime() - Method in class io.delta.kernel.internal.actions.AddFile
- getModificationTime() - Method in class io.delta.kernel.internal.checkpoints.SidecarFile
- getModificationTime() - Method in class io.delta.kernel.utils.FileStatus
-
Get the modification time of the file in epoch millis.
- getName() - Method in class io.delta.kernel.expressions.ScalarExpression
- getName() - Method in class io.delta.kernel.internal.actions.Metadata
- getName() - Method in class io.delta.kernel.internal.fs.Path
-
Returns the final component of this path.
- getName() - Method in class io.delta.kernel.types.CollationIdentifier
- getName() - Method in class io.delta.kernel.types.StructField
- getName(String) - Static method in class io.delta.kernel.internal.fs.Path
- getNames() - Method in class io.delta.kernel.expressions.Column
- getNullCount() - Method in class io.delta.kernel.statistics.DataFileStatistics
-
Get the number of nulls of columns in the data file.
- getNullCountColumn(Column) - Method in class io.delta.kernel.internal.skipping.StatsSchemaHelper
-
Given a logical column in the data schema provided when creating
this
, return the corresponding NULL_COUNT column in the statistic schema that stores the null count values for the provided logical column. - getNumActiveAddFiles() - Method in interface io.delta.kernel.metrics.ScanMetricsResult
- getNumAddFiles() - Method in interface io.delta.kernel.metrics.TransactionMetricsResult
- getNumAddFilesSeen() - Method in interface io.delta.kernel.metrics.ScanMetricsResult
- getNumAddFilesSeenFromDeltaFiles() - Method in interface io.delta.kernel.metrics.ScanMetricsResult
- getNumberOfAddActions() - Method in class io.delta.kernel.internal.replay.CreateCheckpointIterator
-
Number of add files in the final checkpoint.
- getNumCommitAttempts() - Method in interface io.delta.kernel.metrics.TransactionMetricsResult
- getNumDuplicateAddFiles() - Method in interface io.delta.kernel.metrics.ScanMetricsResult
-
Returns the number of duplicate AddFile actions seen during log replay.
- getNumFiles() - Method in class io.delta.kernel.internal.checksum.CRCInfo
- getNumRecords() - Method in class io.delta.kernel.internal.actions.AddFile
- getNumRecords() - Method in class io.delta.kernel.statistics.DataFileStatistics
-
Get the number of records in the data file.
- getNumRecordsColumn() - Method in class io.delta.kernel.internal.skipping.StatsSchemaHelper
-
Returns the NUM_RECORDS column in the statistic schema
- getNumRemoveFiles() - Method in interface io.delta.kernel.metrics.TransactionMetricsResult
- getNumRemoveFilesSeenFromDeltaFiles() - Method in interface io.delta.kernel.metrics.ScanMetricsResult
- getNumTotalActions() - Method in interface io.delta.kernel.metrics.TransactionMetricsResult
- getOffset() - Method in class io.delta.kernel.internal.actions.DeletionVectorDescriptor
- getOperation() - Method in class io.delta.kernel.internal.actions.CommitInfo
- getOperation() - Method in class io.delta.kernel.internal.metrics.TransactionReportImpl
- getOperation() - Method in interface io.delta.kernel.metrics.TransactionReport
- getOperationMetrics() - Method in class io.delta.kernel.internal.actions.CommitInfo
- getOperationParameters() - Method in class io.delta.kernel.internal.actions.CommitInfo
- getOperationType() - Method in interface io.delta.kernel.metrics.DeltaOperationReport
- getOperationType() - Method in interface io.delta.kernel.metrics.ScanReport
- getOperationType() - Method in interface io.delta.kernel.metrics.SnapshotReport
- getOperationType() - Method in interface io.delta.kernel.metrics.TransactionReport
- getOptions() - Method in class io.delta.kernel.internal.actions.Format
- getParent() - Method in class io.delta.kernel.internal.fs.Path
-
Returns the parent of a path or null if at root.
- getParquetHandler() - Method in interface io.delta.kernel.engine.Engine
-
Get the connector provided
ParquetHandler
. - getPartitionColNames() - Method in class io.delta.kernel.internal.actions.Metadata
-
Set of lowercase partition column names
- getPartitionColumnNames() - Method in class io.delta.kernel.internal.SnapshotImpl
- getPartitionColumnNames() - Method in interface io.delta.kernel.Snapshot
-
Get the names of the partition columns in the Delta table at this snapshot.
- getPartitionColumns() - Method in class io.delta.kernel.internal.actions.Metadata
- getPartitionColumns(Row) - Static method in class io.delta.kernel.internal.data.ScanStateRow
-
Get the list of partition column names from the scan state
Row
returned byScan.getScanState(Engine)
. - getPartitionColumns(Engine) - Method in class io.delta.kernel.internal.TransactionImpl
- getPartitionColumns(Engine) - Method in interface io.delta.kernel.Transaction
-
Get the list of logical names of the partition columns.
- getPartitionColumnsList(Row) - Static method in class io.delta.kernel.internal.data.TransactionStateRow
-
Get the list of partition column names from the transaction state
Row
returned byTransaction.getTransactionState(Engine)
- getPartitionPredicate() - Method in class io.delta.kernel.internal.metrics.ScanReportImpl
- getPartitionPredicate() - Method in interface io.delta.kernel.metrics.ScanReport
- getPartitionValues() - Method in class io.delta.kernel.internal.actions.AddFile
- getPartitionValues() - Method in class io.delta.kernel.internal.actions.RemoveFile
- getPartitionValues() - Method in class io.delta.kernel.internal.DataWriteContextImpl
-
Returns the partition values for the data to be written.
- getPartitionValues(Row) - Static method in class io.delta.kernel.internal.InternalScanFileUtils
-
Get the partition columns and values belonging to the
AddFile
from given scan file row. - getPartitionValuesParsedRefInAddFile(String) - Static method in class io.delta.kernel.internal.InternalScanFileUtils
-
Get a references column for given partition column name in partitionValues_parsed column in scan file row.
- getPath() - Method in interface io.delta.kernel.engine.FileReadRequest
-
Get the fully qualified path of the file from which to read the data.
- getPath() - Method in class io.delta.kernel.internal.actions.AddFile
- getPath() - Method in class io.delta.kernel.internal.actions.RemoveFile
- getPath() - Method in class io.delta.kernel.internal.checkpoints.SidecarFile
- getPath() - Method in class io.delta.kernel.utils.FileStatus
-
Get the path to the file.
- getPath(Engine) - Method in class io.delta.kernel.internal.TableImpl
- getPath(Engine) - Method in interface io.delta.kernel.Table
-
The fully qualified path of this
Table
instance. - getPathOrInlineDv() - Method in class io.delta.kernel.internal.actions.DeletionVectorDescriptor
- getPhysicalColumnNameAndDataType(StructType, Column) - Static method in class io.delta.kernel.internal.util.ColumnMapping
-
Returns the physical column and data type for a given logical column based on the schema.
- getPhysicalDataReadSchema(Engine, Row) - Static method in class io.delta.kernel.internal.data.ScanStateRow
-
Utility method to get the physical data read schema from the scan state
Row
returned byScan.getScanState(Engine)
. - getPhysicalName(StructField) - Static method in class io.delta.kernel.internal.util.ColumnMapping
-
Returns the physical name for a given
StructField
- getPhysicalSchema() - Method in class io.delta.kernel.internal.actions.Metadata
-
The full schema (including partition columns) with the field names converted to their physical names (column names used in the data files) based on the table's column mapping mode.
- getPhysicalSchema(Row) - Static method in class io.delta.kernel.internal.data.ScanStateRow
-
Utility method to get the physical schema from the scan state
Row
returned byScan.getScanState(Engine)
. - getPhysicalSchema(Row) - Static method in class io.delta.kernel.internal.data.TransactionStateRow
-
Get the physical schema of the table from the transaction state
Row
returned byTransaction.getTransactionState(Engine)
} - getPostCommitHooks() - Method in class io.delta.kernel.TransactionCommitResult
-
Operations for connector to trigger post-commit.
- getPrecision() - Method in class io.delta.kernel.types.DecimalType
- getPredicateEvaluator(StructType, Predicate) - Method in interface io.delta.kernel.engine.ExpressionHandler
-
Create a
PredicateEvaluator
that can evaluate the given predicate expression and return a selection vector (ColumnVector
ofboolean
type). - getProtocol() - Method in class io.delta.kernel.internal.checksum.CRCInfo
-
The
Protocol
stored in this CRCInfo. - getProtocol() - Method in class io.delta.kernel.internal.replay.LogReplay
- getProtocol() - Method in class io.delta.kernel.internal.snapshot.SnapshotHint
- getProtocol() - Method in class io.delta.kernel.internal.SnapshotImpl
- getProtocol() - Method in class io.delta.kernel.internal.TransactionImpl
- getProvidedTimestamp() - Method in class io.delta.kernel.internal.metrics.SnapshotQueryContext
- getProvidedTimestamp() - Method in class io.delta.kernel.internal.metrics.SnapshotReportImpl
- getProvidedTimestamp() - Method in interface io.delta.kernel.metrics.SnapshotReport
- getProvider() - Method in class io.delta.kernel.internal.actions.Format
- getProvider() - Method in class io.delta.kernel.types.CollationIdentifier
- getReaderFeatures() - Method in class io.delta.kernel.internal.actions.Protocol
- getReadLength() - Method in interface io.delta.kernel.engine.FileReadRequest
-
Get the length of the data to read from the file starting at the startOffset.
- getReadSchema() - Method in class io.delta.kernel.internal.metrics.ScanReportImpl
- getReadSchema() - Method in interface io.delta.kernel.metrics.ScanReport
- getReadTableVersion() - Method in class io.delta.kernel.internal.TransactionImpl
- getReadTableVersion() - Method in interface io.delta.kernel.Transaction
-
Gets the latest version of the table used as the base of this transaction.
- getReferencedCols() - Method in class io.delta.kernel.internal.skipping.DataSkippingPredicate
- getRemainingFilter() - Method in class io.delta.kernel.internal.ScanImpl
- getRemainingFilter() - Method in interface io.delta.kernel.Scan
-
Get the remaining filter that is not guaranteed to be satisfied for the data Delta Kernel returns.
- getRemoveFileDV(ColumnVector, int) - Static method in class io.delta.kernel.internal.replay.ActiveAddFilesIterator
- getRemoveFilePath(ColumnVector, int) - Static method in class io.delta.kernel.internal.replay.ActiveAddFilesIterator
- getReportUUID() - Method in class io.delta.kernel.internal.metrics.DeltaOperationReportImpl
- getReportUUID() - Method in interface io.delta.kernel.metrics.DeltaOperationReport
- getRequiredInCommitTimestamp(Optional<CommitInfo>, String, Path) - Static method in class io.delta.kernel.internal.actions.CommitInfo
-
Returns the `inCommitTimestamp` of the given `commitInfoOpt` if it is defined.
- getRight() - Method in class io.delta.kernel.expressions.And
- getRight() - Method in class io.delta.kernel.expressions.Or
- getRight(Expression) - Static method in class io.delta.kernel.internal.util.ExpressionUtils
-
Utility method to return the right child of the binary input expression
- getRowIdHighWaterMark() - Method in class io.delta.kernel.internal.rowtracking.RowTrackingMetadataDomain
- getRows() - Method in interface io.delta.kernel.data.ColumnarBatch
- getRows() - Method in class io.delta.kernel.data.FilteredColumnarBatch
-
Iterator of rows that survived the filter.
- getScale() - Method in class io.delta.kernel.types.DecimalType
- getScanBuilder() - Method in class io.delta.kernel.internal.SnapshotImpl
- getScanBuilder() - Method in interface io.delta.kernel.Snapshot
-
Create a scan builder to construct a
Scan
to read data from this snapshot. - getScanFiles(Engine) - Method in class io.delta.kernel.internal.ScanImpl
-
Get an iterator of data files in this version of scan that survived the predicate pruning.
- getScanFiles(Engine) - Method in interface io.delta.kernel.Scan
-
Get an iterator of data files to scan.
- getScanFiles(Engine, boolean) - Method in class io.delta.kernel.internal.ScanImpl
-
Get an iterator of data files in this version of scan that survived the predicate pruning.
- getScanMetrics() - Method in class io.delta.kernel.internal.metrics.ScanReportImpl
- getScanMetrics() - Method in interface io.delta.kernel.metrics.ScanReport
- getScanState(Engine) - Method in class io.delta.kernel.internal.ScanImpl
- getScanState(Engine) - Method in interface io.delta.kernel.Scan
-
Get the scan state associated with the current scan.
- getSchema() - Method in interface io.delta.kernel.data.ColumnarBatch
- getSchema() - Method in interface io.delta.kernel.data.Row
- getSchema() - Method in class io.delta.kernel.internal.actions.Metadata
- getSchema() - Method in class io.delta.kernel.internal.data.ChildVectorBasedRow
- getSchema() - Method in class io.delta.kernel.internal.data.DelegateRow
- getSchema() - Method in class io.delta.kernel.internal.data.GenericRow
- getSchema() - Method in class io.delta.kernel.internal.SnapshotImpl
- getSchema() - Method in interface io.delta.kernel.Snapshot
-
Get the schema of the table at this snapshot.
- getSchema(Engine) - Method in class io.delta.kernel.internal.TransactionImpl
- getSchema(Engine) - Method in interface io.delta.kernel.Transaction
-
Get the schema of the table.
- getSchemaString() - Method in class io.delta.kernel.internal.actions.Metadata
- getSelectionVector() - Method in class io.delta.kernel.data.FilteredColumnarBatch
-
Optional selection vector containing one entry for each row in data indicating whether a row is selected or not selected.
- getSetTxnOpt() - Method in class io.delta.kernel.internal.TransactionImpl
- getShort(int) - Method in interface io.delta.kernel.data.ColumnVector
-
Returns the short type value for
rowId
. - getShort(int) - Method in interface io.delta.kernel.data.Row
-
Return short value of the column located at the given ordinal.
- getShort(int) - Method in class io.delta.kernel.internal.data.ChildVectorBasedRow
- getShort(int) - Method in class io.delta.kernel.internal.data.DelegateRow
- getShort(int) - Method in class io.delta.kernel.internal.data.GenericColumnVector
- getShort(int) - Method in class io.delta.kernel.internal.data.GenericRow
- getSingularElement(CloseableIterator<T>) - Static method in class io.delta.kernel.internal.util.InternalUtils
-
Utility method to read at most one element from a
CloseableIterator
. - getSingularRow(CloseableIterator<ColumnarBatch>) - Static method in class io.delta.kernel.internal.util.InternalUtils
-
Utility method to read at most one row from the given data
ColumnarBatch
iterator. - getSize() - Method in interface io.delta.kernel.data.ArrayValue
-
The number of elements in the array
- getSize() - Method in interface io.delta.kernel.data.ColumnarBatch
- getSize() - Method in interface io.delta.kernel.data.ColumnVector
- getSize() - Method in interface io.delta.kernel.data.MapValue
-
The number of elements in the map
- getSize() - Method in class io.delta.kernel.internal.actions.AddFile
- getSize() - Method in class io.delta.kernel.internal.actions.RemoveFile
- getSize() - Method in class io.delta.kernel.internal.data.GenericColumnVector
- getSize() - Method in class io.delta.kernel.internal.data.SelectionColumnVector
- getSize() - Method in class io.delta.kernel.utils.FileStatus
-
Get the size of the file in bytes.
- getSizeInBytes() - Method in class io.delta.kernel.internal.actions.DeletionVectorDescriptor
- getSizeInBytes() - Method in class io.delta.kernel.internal.checkpoints.SidecarFile
- getSnapshotAsOfTimestamp(Engine, long) - Method in class io.delta.kernel.internal.TableImpl
- getSnapshotAsOfTimestamp(Engine, long) - Method in interface io.delta.kernel.Table
-
Get the snapshot of the table at the given
timestamp
. - getSnapshotAsOfVersion(Engine, long) - Method in class io.delta.kernel.internal.TableImpl
- getSnapshotAsOfVersion(Engine, long) - Method in interface io.delta.kernel.Table
-
Get the snapshot at the given
versionId
. - getSnapshotAt(Engine, long, SnapshotQueryContext) - Method in class io.delta.kernel.internal.snapshot.SnapshotManager
-
Construct the snapshot for the given table at the version provided.
- getSnapshotForTimestamp(Engine, long, SnapshotQueryContext) - Method in class io.delta.kernel.internal.snapshot.SnapshotManager
-
Construct the snapshot for the given table at the provided timestamp.
- getSnapshotMetrics() - Method in class io.delta.kernel.internal.metrics.SnapshotQueryContext
- getSnapshotMetrics() - Method in class io.delta.kernel.internal.metrics.SnapshotReportImpl
- getSnapshotMetrics() - Method in interface io.delta.kernel.metrics.SnapshotReport
- getSnapshotReport() - Method in class io.delta.kernel.internal.SnapshotImpl
- getSnapshotReportUUID() - Method in class io.delta.kernel.internal.metrics.ScanReportImpl
- getSnapshotReportUUID() - Method in class io.delta.kernel.internal.metrics.TransactionReportImpl
- getSnapshotReportUUID() - Method in interface io.delta.kernel.metrics.ScanReport
- getSnapshotReportUUID() - Method in interface io.delta.kernel.metrics.TransactionReport
- getSortedBinBoundaries() - Method in interface io.delta.kernel.metrics.FileSizeHistogramResult
-
Sorted list of bin boundaries where each element represents the start of the bin (inclusive) and the next element represents the end of the bin (exclusive).
- getStartOffset() - Method in interface io.delta.kernel.engine.FileReadRequest
-
Get the start offset in the file from where to start reading the data.
- getStartVersion() - Method in class io.delta.kernel.internal.hook.LogCompactionHook
- getStatistics() - Method in class io.delta.kernel.utils.DataFileStatus
-
Get the statistics of the data file encapsulated in this object.
- getStatisticsColumns() - Method in interface io.delta.kernel.DataWriteContext
-
Returns the list of
Column
that the connector can optionally collect statistics. - getStatisticsColumns() - Method in class io.delta.kernel.internal.DataWriteContextImpl
-
Returns the list of
Column
that the connector can optionally collect statistics. - getStatisticsColumns(Row) - Static method in class io.delta.kernel.internal.TransactionImpl
-
Get the part of the schema of the table that needs the statistics to be collected per file.
- getStats() - Method in class io.delta.kernel.internal.actions.AddFile
- getStats() - Method in class io.delta.kernel.internal.actions.RemoveFile
- getStatsJson() - Method in class io.delta.kernel.internal.actions.AddFile
- getStatsJson() - Method in class io.delta.kernel.internal.actions.RemoveFile
- getStatsSchema(StructType) - Static method in class io.delta.kernel.internal.skipping.StatsSchemaHelper
-
Returns the expected statistics schema given a table schema.
- getStorageType() - Method in class io.delta.kernel.internal.actions.DeletionVectorDescriptor
- getString(int) - Method in interface io.delta.kernel.data.ColumnVector
-
Returns the string type value for
rowId
. - getString(int) - Method in interface io.delta.kernel.data.Row
-
Return string value of the column located at the given ordinal.
- getString(int) - Method in class io.delta.kernel.internal.data.ChildVectorBasedRow
- getString(int) - Method in class io.delta.kernel.internal.data.DelegateRow
- getString(int) - Method in class io.delta.kernel.internal.data.GenericColumnVector
- getString(int) - Method in class io.delta.kernel.internal.data.GenericRow
- getString(String) - Method in class io.delta.kernel.types.FieldMetadata
- getStringArray(String) - Method in class io.delta.kernel.types.FieldMetadata
- getStruct(int) - Method in interface io.delta.kernel.data.Row
-
Return struct value of the column located at the given ordinal.
- getStruct(int) - Method in class io.delta.kernel.internal.data.ChildVectorBasedRow
- getStruct(int) - Method in class io.delta.kernel.internal.data.DelegateRow
- getStruct(int) - Method in class io.delta.kernel.internal.data.GenericRow
- getTableFeature(String) - Static method in class io.delta.kernel.internal.tablefeatures.TableFeatures
-
Get the table feature by name.
- getTableFileSizeHistogram() - Method in interface io.delta.kernel.metrics.TransactionMetricsResult
- getTablePath() - Method in exception io.delta.kernel.exceptions.TableNotFoundException
- getTablePath() - Method in class io.delta.kernel.internal.metrics.DeltaOperationReportImpl
- getTablePath() - Method in class io.delta.kernel.internal.metrics.SnapshotQueryContext
- getTablePath() - Method in interface io.delta.kernel.metrics.DeltaOperationReport
- getTablePath(Row) - Static method in class io.delta.kernel.internal.data.TransactionStateRow
-
Get the table path from transaction state
Row
returned byTransaction.getTransactionState(Engine)
- getTableRoot(Row) - Static method in class io.delta.kernel.internal.data.ScanStateRow
-
Get the table root from scan state
Row
returned byScan.getScanState(Engine)
- getTableSchema() - Method in class io.delta.kernel.internal.metrics.ScanReportImpl
- getTableSchema() - Method in interface io.delta.kernel.metrics.ScanReport
- getTableSizeBytes() - Method in class io.delta.kernel.internal.checksum.CRCInfo
- getTableVersion() - Method in class io.delta.kernel.internal.metrics.ScanReportImpl
- getTableVersion() - Method in interface io.delta.kernel.metrics.ScanReport
- getTags() - Method in class io.delta.kernel.internal.actions.AddFile
- getTags() - Method in class io.delta.kernel.internal.actions.RemoveFile
- getTargetDirectory() - Method in interface io.delta.kernel.DataWriteContext
-
Returns the target directory where the data should be written.
- getTargetDirectory() - Method in class io.delta.kernel.internal.DataWriteContextImpl
-
Returns the target directory where the data should be written.
- getTargetDirectory(String, List<String>, Map<String, Literal>) - Static method in class io.delta.kernel.internal.util.PartitionUtils
-
Get the target directory for writing data for given partition values.
- getTimeMillis() - Method in interface io.delta.kernel.internal.util.Clock
- getTimeMillis() - Method in class io.delta.kernel.internal.util.ManualClock
- getTimestamp() - Method in class io.delta.kernel.internal.actions.CommitInfo
- getTimestamp() - Method in class io.delta.kernel.internal.DeltaHistoryManager.Commit
- getTimestamp() - Method in class io.delta.kernel.internal.replay.ActionWrapper
- getTimestamp(Engine) - Method in class io.delta.kernel.internal.SnapshotImpl
-
Get the timestamp (in milliseconds since the Unix epoch) of the latest commit in this Snapshot.
- getTimestamp(Engine) - Method in interface io.delta.kernel.Snapshot
-
Get the timestamp (in milliseconds since the Unix epoch) of the latest commit in this snapshot.
- getTimestampToVersionResolutionDurationNs() - Method in interface io.delta.kernel.metrics.SnapshotMetricsResult
- getTotalAddFilesSizeInBytes() - Method in interface io.delta.kernel.metrics.TransactionMetricsResult
- getTotalBytes() - Method in interface io.delta.kernel.metrics.FileSizeHistogramResult
-
The total number of bytes in each bin of
FileSizeHistogramResult.getSortedBinBoundaries()
- getTotalCommitDurationNs() - Method in interface io.delta.kernel.metrics.TransactionMetricsResult
- getTotalPlanningDurationNs() - Method in interface io.delta.kernel.metrics.ScanMetricsResult
-
Returns the total duration to find, filter, and consume the scan files.
- getTotalRemoveFilesSizeInBytes() - Method in interface io.delta.kernel.metrics.TransactionMetricsResult
- getTransactionMetrics() - Method in class io.delta.kernel.internal.metrics.TransactionReportImpl
- getTransactionMetrics() - Method in interface io.delta.kernel.metrics.TransactionReport
- getTransactionMetrics() - Method in class io.delta.kernel.TransactionCommitResult
- getTransactionState(Engine) - Method in class io.delta.kernel.internal.TransactionImpl
- getTransactionState(Engine) - Method in interface io.delta.kernel.Transaction
-
Get the state of the transaction.
- getTxnId() - Method in class io.delta.kernel.internal.actions.CommitInfo
- getTxnId() - Method in class io.delta.kernel.internal.checksum.CRCInfo
- getType() - Method in interface io.delta.kernel.hook.PostCommitHook
- getType() - Method in class io.delta.kernel.internal.hook.CheckpointHook
- getType() - Method in class io.delta.kernel.internal.hook.ChecksumFullHook
- getType() - Method in class io.delta.kernel.internal.hook.ChecksumSimpleHook
- getType() - Method in class io.delta.kernel.internal.hook.LogCompactionHook
- getUnaryChild(Expression) - Static method in class io.delta.kernel.internal.util.ExpressionUtils
-
Utility method to return the single child of the unary input expression
- getUniqueFileAction(ColumnVector, ColumnVector, int) - Static method in class io.delta.kernel.internal.replay.LogReplayUtils
- getUniqueId() - Method in class io.delta.kernel.internal.actions.DeletionVectorDescriptor
- getUpdatedCrcInfo() - Method in class io.delta.kernel.internal.replay.ConflictChecker.TransactionRebaseState
- getUpdatedDataActions() - Method in class io.delta.kernel.internal.replay.ConflictChecker.TransactionRebaseState
- getUpdatedDomainMetadatas() - Method in class io.delta.kernel.internal.replay.ConflictChecker.TransactionRebaseState
- getUpdatedMetadataWithICTEnablementInfo(Engine, long, SnapshotImpl, Metadata, long) - Static method in class io.delta.kernel.internal.util.InCommitTimestampUtils
-
Returns the updated
Metadata
with inCommitTimestamp enablement related info (version and timestamp) correctly set. - getValue() - Method in class io.delta.kernel.expressions.Literal
-
Get the literal value.
- getValueField() - Method in class io.delta.kernel.types.MapType
- getValues() - Method in interface io.delta.kernel.data.MapValue
-
A
ColumnVector
containing the values. - getValueType() - Method in class io.delta.kernel.types.MapType
- getVersion() - Method in class io.delta.kernel.internal.actions.SetTransaction
- getVersion() - Method in class io.delta.kernel.internal.checksum.CRCInfo
-
The version of the Delta table that this CRCInfo represents.
- getVersion() - Method in class io.delta.kernel.internal.DeltaHistoryManager.Commit
- getVersion() - Method in class io.delta.kernel.internal.metrics.SnapshotQueryContext
- getVersion() - Method in class io.delta.kernel.internal.metrics.SnapshotReportImpl
- getVersion() - Method in class io.delta.kernel.internal.replay.ActionWrapper
- getVersion() - Method in class io.delta.kernel.internal.replay.DeltaLogFile
-
Get the version for this log file.
- getVersion() - Method in class io.delta.kernel.internal.replay.LogReplay
- getVersion() - Method in class io.delta.kernel.internal.snapshot.LogSegment
- getVersion() - Method in class io.delta.kernel.internal.snapshot.SnapshotHint
- getVersion() - Method in class io.delta.kernel.internal.SnapshotImpl
- getVersion() - Method in interface io.delta.kernel.metrics.SnapshotReport
-
For a time-travel by version query, this is the version provided.
- getVersion() - Method in interface io.delta.kernel.Snapshot
-
Get the version of this snapshot in the table.
- getVersion() - Method in class io.delta.kernel.TransactionCommitResult
-
Contains the version of the transaction committed as.
- getVersion() - Method in class io.delta.kernel.types.CollationIdentifier
- getVersionAtOrAfterTimestamp(Engine, long) - Method in class io.delta.kernel.internal.TableImpl
-
Returns the latest version that was committed at or after
millisSinceEpochUTC
. - getVersionBeforeOrAtTimestamp(Engine, long) - Method in class io.delta.kernel.internal.TableImpl
-
Returns the latest version that was committed before or at
millisSinceEpochUTC
. - getWriteContext(Engine, Row, Map<String, Literal>) - Static method in interface io.delta.kernel.Transaction
-
Get the context for writing data into a table.
- getWriterFeatures() - Method in class io.delta.kernel.internal.actions.Protocol
H
- hasCheckConstraints(Metadata) - Static method in class io.delta.kernel.internal.tablefeatures.TableFeatures
- hasGeneratedColumns(Metadata) - Static method in class io.delta.kernel.internal.tablefeatures.TableFeatures
- hashCode() - Method in class io.delta.kernel.expressions.Column
- hashCode() - Method in class io.delta.kernel.internal.actions.AddFile
- hashCode() - Method in class io.delta.kernel.internal.actions.DeletionVectorDescriptor
- hashCode() - Method in class io.delta.kernel.internal.actions.DomainMetadata
- hashCode() - Method in class io.delta.kernel.internal.actions.Format
- hashCode() - Method in class io.delta.kernel.internal.actions.Metadata
- hashCode() - Method in class io.delta.kernel.internal.actions.Protocol
- hashCode() - Method in class io.delta.kernel.internal.checkpoints.CheckpointInstance
- hashCode() - Method in class io.delta.kernel.internal.checksum.CRCInfo
- hashCode() - Method in class io.delta.kernel.internal.DeltaHistoryManager.Commit
- hashCode() - Method in class io.delta.kernel.internal.files.ParsedLogCompactionData
- hashCode() - Method in class io.delta.kernel.internal.files.ParsedLogData
- hashCode() - Method in class io.delta.kernel.internal.files.ParsedMultiPartCheckpointData
- hashCode() - Method in class io.delta.kernel.internal.fs.Path
- hashCode() - Method in class io.delta.kernel.internal.rowtracking.RowTrackingMetadataDomain
- hashCode() - Method in class io.delta.kernel.internal.stats.FileSizeHistogram
- hashCode() - Method in class io.delta.kernel.internal.util.Tuple2
- hashCode() - Method in class io.delta.kernel.statistics.DataFileStatistics
- hashCode() - Method in class io.delta.kernel.types.ArrayType
- hashCode() - Method in class io.delta.kernel.types.BasePrimitiveType
- hashCode() - Method in class io.delta.kernel.types.DataType
- hashCode() - Method in class io.delta.kernel.types.DecimalType
- hashCode() - Method in class io.delta.kernel.types.FieldMetadata
- hashCode() - Method in class io.delta.kernel.types.MapType
- hashCode() - Method in class io.delta.kernel.types.StructField
- hashCode() - Method in class io.delta.kernel.types.StructType
- hashCode() - Method in class io.delta.kernel.utils.FileStatus
- hasIdentityColumns(Metadata) - Static method in class io.delta.kernel.internal.tablefeatures.TableFeatures
- hasInvariants(StructType) - Static method in class io.delta.kernel.internal.tablefeatures.TableFeatures
- hasKernelReadSupport() - Method in class io.delta.kernel.internal.tablefeatures.TableFeature
-
Does Kernel has support to read a table containing this feature? Default implementation returns true.
- hasKernelReadSupport() - Method in class io.delta.kernel.internal.tablefeatures.TableFeature.LegacyWriterFeature
- hasKernelReadSupport() - Method in class io.delta.kernel.internal.tablefeatures.TableFeature.WriterFeature
- hasKernelWriteSupport(Metadata) - Method in class io.delta.kernel.internal.tablefeatures.TableFeature
-
Does Kernel has support to write a table containing this feature? Default implementation returns true.
- hasNext() - Method in class io.delta.kernel.internal.replay.ActionsIterator
- hasNext() - Method in class io.delta.kernel.internal.replay.ActiveAddFilesIterator
- hasNext() - Method in class io.delta.kernel.internal.replay.CreateCheckpointIterator
- hasNext() - Method in interface io.delta.kernel.utils.CloseableIterator
-
Returns true if the iteration has more elements.
- HOURS_PER_DAY - Static variable in class io.delta.kernel.internal.util.DateTimeConstants
I
- ICEBERG_COMPAT_V2_ENABLED - Static variable in class io.delta.kernel.internal.TableConfig
-
Table property that enables modifying the table in accordance with the Delta-Iceberg Compatibility V2 protocol.
- ICEBERG_COMPAT_V2_W_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- ICEBERG_WRITER_COMPAT_V1 - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- ICEBERG_WRITER_COMPAT_V1_ENABLED - Static variable in class io.delta.kernel.internal.TableConfig
-
Table property that enables modifying the table in accordance with the Delta-Iceberg Writer Compatibility V1 (
icebergCompatWriterV1
) protocol. - icebergCompatIncompatibleTableFeatures(String, Set<TableFeature>) - Static method in class io.delta.kernel.internal.DeltaErrors
- icebergCompatIncompatibleVersionEnabled(String, String) - Static method in class io.delta.kernel.internal.DeltaErrors
- IcebergCompatInputContext(boolean, Metadata, Protocol) - Constructor for class io.delta.kernel.internal.icebergcompat.IcebergCompatMetadataValidatorAndUpdater.IcebergCompatInputContext
- IcebergCompatMetadataValidatorAndUpdater - Class in io.delta.kernel.internal.icebergcompat
-
Contains interfaces and common utility classes for defining the iceberg conversion compatibility checks and metadata updates.
- IcebergCompatMetadataValidatorAndUpdater() - Constructor for class io.delta.kernel.internal.icebergcompat.IcebergCompatMetadataValidatorAndUpdater
- IcebergCompatMetadataValidatorAndUpdater.IcebergCompatInputContext - Class in io.delta.kernel.internal.icebergcompat
-
Defines the input context for the metadata validator and updater.
- icebergCompatMissingNumRecordsStats(String, DataFileStatus) - Static method in class io.delta.kernel.internal.DeltaErrors
- icebergCompatRequiredFeatureMissing(String, String) - Static method in class io.delta.kernel.internal.DeltaErrors
- icebergCompatUnsupportedTypeColumns(String, List<DataType>) - Static method in class io.delta.kernel.internal.DeltaErrors
- icebergCompatUnsupportedTypePartitionColumn(String, DataType) - Static method in class io.delta.kernel.internal.DeltaErrors
- IcebergCompatV2MetadataValidatorAndUpdater - Class in io.delta.kernel.internal.icebergcompat
-
Utility methods for validation and compatibility checks for Iceberg V2.
- IcebergCompatV2MetadataValidatorAndUpdater() - Constructor for class io.delta.kernel.internal.icebergcompat.IcebergCompatV2MetadataValidatorAndUpdater
- IcebergUniversalFormatMetadataValidatorAndUpdater - Class in io.delta.kernel.internal.icebergcompat
-
Utility class that enforces dependencies of UNIVERSAL_FORMAT_* options.
- icebergWriterCompatInvalidPhysicalName(List<String>) - Static method in class io.delta.kernel.internal.DeltaErrors
- IcebergWriterCompatV1MetadataValidatorAndUpdater - Class in io.delta.kernel.internal.icebergcompat
-
Performs the validations and updates necessary to support the table feature IcebergWriterCompatV1 when it is enabled by the table property "delta.enableIcebergWriterCompatV1".
- IcebergWriterCompatV1MetadataValidatorAndUpdater() - Constructor for class io.delta.kernel.internal.icebergcompat.IcebergWriterCompatV1MetadataValidatorAndUpdater
- ID - Enum constant in enum io.delta.kernel.internal.util.ColumnMapping.ColumnMappingMode
- IDENTITY_COLUMNS_W_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- IN_COMMIT_TIMESTAMP_ENABLEMENT_TIMESTAMP - Static variable in class io.delta.kernel.internal.TableConfig
-
This table property is used to track the timestamp at which
inCommitTimestamps
were enabled. - IN_COMMIT_TIMESTAMP_ENABLEMENT_VERSION - Static variable in class io.delta.kernel.internal.TableConfig
-
This table property is used to track the version of the table at which
inCommitTimestamps
were enabled. - IN_COMMIT_TIMESTAMP_W_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- IN_COMMIT_TIMESTAMPS_ENABLED - Static variable in class io.delta.kernel.internal.TableConfig
-
This table property is used to track the enablement of the
inCommitTimestamps
. - INCLUDE - Enum constant in enum io.delta.kernel.utils.CloseableIterator.BreakableFilterResult
-
Indicates that the current element should be included in the resulting iterator produced by
CloseableIterator.breakableFilter(Function)
. - InCommitTimestampUtils - Class in io.delta.kernel.internal.util
- InCommitTimestampUtils() - Constructor for class io.delta.kernel.internal.util.InCommitTimestampUtils
- increment() - Method in class io.delta.kernel.internal.metrics.Counter
-
Increment the counter by 1.
- increment(long) - Method in class io.delta.kernel.internal.metrics.Counter
-
Increment the counter by the provided amount.
- indexOf(String) - Method in class io.delta.kernel.types.StructType
- INLINE_DV_MARKER - Static variable in class io.delta.kernel.internal.actions.DeletionVectorDescriptor
- inlineData() - Method in class io.delta.kernel.internal.actions.DeletionVectorDescriptor
- inlineDataOpt - Variable in class io.delta.kernel.internal.files.ParsedLogData
- inMemoryIterable(CloseableIterator<T>) - Static method in interface io.delta.kernel.utils.CloseableIterable
-
Return an
CloseableIterable
object that is backed by an in-memory collection of givenCloseableIterator
. - insert(long) - Method in class io.delta.kernel.internal.stats.FileSizeHistogram
-
Adds a file size to the histogram, incrementing the appropriate bin's count and total bytes.
- INTEGER - Static variable in class io.delta.kernel.types.IntegerType
- IntegerType - Class in io.delta.kernel.types
-
The data type representing
integer
type values. - InternalScanFileUtils - Class in io.delta.kernel.internal
-
Utilities to extract information out of the scan file rows returned by
Scan.getScanFiles(Engine)
. - InternalUtils - Class in io.delta.kernel.internal.util
- IntervalParserUtils - Class in io.delta.kernel.internal.util
-
Copy of `org/apache/spark/sql/catalyst/util/SparkIntervalUtils.scala` from Apache Spark.
- intoRows(CloseableIterator<FilteredColumnarBatch>) - Static method in class io.delta.kernel.internal.util.Utils
-
Convert a ClosableIterator of FilteredColumnarBatch into a CloseableIterator of Row
- invalidColumnName(String, String) - Static method in class io.delta.kernel.internal.DeltaErrors
- invalidConfigurationValueException(String, String, String) - Static method in class io.delta.kernel.internal.DeltaErrors
- InvalidConfigurationValueException - Exception in io.delta.kernel.exceptions
-
Thrown when an illegal value is specified for a table property.
- InvalidConfigurationValueException(String, String, String) - Constructor for exception io.delta.kernel.exceptions.InvalidConfigurationValueException
- InvalidTableException - Exception in io.delta.kernel.exceptions
-
Thrown when an invalid table is encountered; the table's log and/or checkpoint files are in an invalid state.
- InvalidTableException(String, String) - Constructor for exception io.delta.kernel.exceptions.InvalidTableException
- invalidTimestampFormatForPartitionValue(String) - Static method in class io.delta.kernel.internal.DeltaErrorsInternal
- invalidVersionRange(long, long) - Static method in class io.delta.kernel.internal.DeltaErrors
- INVARIANTS_W_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- io.delta.kernel - package io.delta.kernel
-
Delta Kernel interfaces for constructing table object representing a Delta Lake table, getting snapshot from the table and building a scan object to scan a subset of the data in the table.
- io.delta.kernel.annotation - package io.delta.kernel.annotation
- io.delta.kernel.data - package io.delta.kernel.data
-
Delta Kernel interfaces for representing data in columnar and row format.
- io.delta.kernel.engine - package io.delta.kernel.engine
-
Interfaces to allow the connector to bring their own implementation of functions such as reading parquet files, listing files in a file system, parsing a JSON string etc.
- io.delta.kernel.exceptions - package io.delta.kernel.exceptions
- io.delta.kernel.expressions - package io.delta.kernel.expressions
-
Expressions framework that defines the most common expressions which the connectors can use to pass predicates to Delta Kernel.
- io.delta.kernel.hook - package io.delta.kernel.hook
- io.delta.kernel.internal - package io.delta.kernel.internal
- io.delta.kernel.internal.actions - package io.delta.kernel.internal.actions
- io.delta.kernel.internal.annotation - package io.delta.kernel.internal.annotation
- io.delta.kernel.internal.checkpoints - package io.delta.kernel.internal.checkpoints
- io.delta.kernel.internal.checksum - package io.delta.kernel.internal.checksum
- io.delta.kernel.internal.clustering - package io.delta.kernel.internal.clustering
- io.delta.kernel.internal.compaction - package io.delta.kernel.internal.compaction
- io.delta.kernel.internal.data - package io.delta.kernel.internal.data
- io.delta.kernel.internal.deletionvectors - package io.delta.kernel.internal.deletionvectors
- io.delta.kernel.internal.files - package io.delta.kernel.internal.files
- io.delta.kernel.internal.fs - package io.delta.kernel.internal.fs
- io.delta.kernel.internal.hook - package io.delta.kernel.internal.hook
- io.delta.kernel.internal.icebergcompat - package io.delta.kernel.internal.icebergcompat
- io.delta.kernel.internal.lang - package io.delta.kernel.internal.lang
- io.delta.kernel.internal.metadatadomain - package io.delta.kernel.internal.metadatadomain
- io.delta.kernel.internal.metrics - package io.delta.kernel.internal.metrics
- io.delta.kernel.internal.replay - package io.delta.kernel.internal.replay
- io.delta.kernel.internal.rowtracking - package io.delta.kernel.internal.rowtracking
- io.delta.kernel.internal.skipping - package io.delta.kernel.internal.skipping
- io.delta.kernel.internal.snapshot - package io.delta.kernel.internal.snapshot
- io.delta.kernel.internal.stats - package io.delta.kernel.internal.stats
- io.delta.kernel.internal.tablefeatures - package io.delta.kernel.internal.tablefeatures
- io.delta.kernel.internal.types - package io.delta.kernel.internal.types
- io.delta.kernel.internal.util - package io.delta.kernel.internal.util
- io.delta.kernel.metrics - package io.delta.kernel.metrics
- io.delta.kernel.statistics - package io.delta.kernel.statistics
- io.delta.kernel.types - package io.delta.kernel.types
-
Data types defined by the Delta Kernel to exchange the type info between the Delta Kernel and the connectors.
- io.delta.kernel.utils - package io.delta.kernel.utils
-
Utilities.
- isAbsolute() - Method in class io.delta.kernel.internal.fs.Path
-
Returns true if the path component (i.e.
- isBlindAppend() - Method in class io.delta.kernel.internal.TransactionImpl
- isCheckpointFile(String) - Static method in class io.delta.kernel.internal.util.FileNames
- isChecksumFile(String) - Static method in class io.delta.kernel.internal.util.FileNames
- isClassicCheckpointFile(String) - Static method in class io.delta.kernel.internal.util.FileNames
- isClusteringTableFeatureSupported(Protocol) - Static method in class io.delta.kernel.internal.tablefeatures.TableFeatures
- isColumnMappingModeEnabled(ColumnMapping.ColumnMappingMode) - Static method in class io.delta.kernel.internal.util.ColumnMapping
- isCommitFile(String) - Static method in class io.delta.kernel.internal.util.FileNames
- isComplete() - Method in class io.delta.kernel.internal.snapshot.LogSegment
- isDataColumn() - Method in class io.delta.kernel.types.StructField
- isDomainMetadataSupported(Protocol) - Static method in class io.delta.kernel.internal.tablefeatures.TableFeatures
- isEmpty() - Method in class io.delta.kernel.internal.util.CaseInsensitiveMap
- isFromCheckpoint() - Method in class io.delta.kernel.internal.replay.ActionWrapper
- isIcebergCompatV2Enabled(Row) - Static method in class io.delta.kernel.internal.data.TransactionStateRow
-
Get the iceberg compatibility enabled or not from the transaction state
Row
returned byTransaction.getTransactionState(Engine)
- isIcebergV2Compatible(DataType, DataType) - Static method in class io.delta.kernel.internal.types.TypeWideningChecker
-
Checks if the type change is supported by Iceberg V2 schema evolution.
- isInline() - Method in class io.delta.kernel.internal.actions.DeletionVectorDescriptor
- isInline() - Method in class io.delta.kernel.internal.files.ParsedLogData
- isLegacyFeature() - Method in class io.delta.kernel.internal.tablefeatures.TableFeature
- isLogCompactionFile(String) - Static method in class io.delta.kernel.internal.util.FileNames
- isMaterialized() - Method in class io.delta.kernel.internal.files.ParsedLogData
- isMetadataColumn() - Method in class io.delta.kernel.types.StructField
- isMultiPartCheckpointFile(String) - Static method in class io.delta.kernel.internal.util.FileNames
- isNullable() - Method in class io.delta.kernel.types.StructField
- isNullAt(int) - Method in interface io.delta.kernel.data.ColumnVector
- isNullAt(int) - Method in interface io.delta.kernel.data.Row
- isNullAt(int) - Method in class io.delta.kernel.internal.data.ChildVectorBasedRow
- isNullAt(int) - Method in class io.delta.kernel.internal.data.DelegateRow
- isNullAt(int) - Method in class io.delta.kernel.internal.data.GenericColumnVector
- isNullAt(int) - Method in class io.delta.kernel.internal.data.GenericRow
- isNullAt(int) - Method in class io.delta.kernel.internal.data.SelectionColumnVector
- isOnDisk() - Method in class io.delta.kernel.internal.actions.DeletionVectorDescriptor
- isPrimitiveType(String) - Static method in class io.delta.kernel.types.BasePrimitiveType
-
Is the given type name a primitive type?
- isPublishedDeltaFile(String) - Static method in class io.delta.kernel.internal.util.FileNames
- isReaderWriterFeature() - Method in class io.delta.kernel.internal.tablefeatures.TableFeature
- isRemoved() - Method in class io.delta.kernel.internal.actions.DomainMetadata
- isRoot() - Method in class io.delta.kernel.internal.fs.Path
-
Returns true if and only if this path represents the root of a file system.
- isRowTrackingSupported(Protocol) - Static method in class io.delta.kernel.internal.tablefeatures.TableFeatures
- isSkippingEligibleDataType(DataType) - Static method in class io.delta.kernel.internal.skipping.StatsSchemaHelper
-
Returns true if the given data type is eligible for MIN/MAX data skipping.
- isSkippingEligibleLiteral(Literal) - Static method in class io.delta.kernel.internal.skipping.StatsSchemaHelper
-
Returns true if the given literal is skipping-eligible.
- isSkippingEligibleMinMaxColumn(Column) - Method in class io.delta.kernel.internal.skipping.StatsSchemaHelper
-
Returns true if the given column is skipping-eligible using min/max statistics.
- isSkippingEligibleNullCountColumn(Column) - Method in class io.delta.kernel.internal.skipping.StatsSchemaHelper
-
Returns true if the given column is skipping-eligible using null count statistics.
- isStagedDeltaFile(String) - Static method in class io.delta.kernel.internal.util.FileNames
- isUriPathAbsolute() - Method in class io.delta.kernel.internal.fs.Path
-
Returns true if the path component (i.e.
- isUserControlledDomain(String) - Static method in class io.delta.kernel.internal.actions.DomainMetadata
-
Whether the provided
domain
is a user-controlled domain - isV2CheckpointFile(String) - Static method in class io.delta.kernel.internal.util.FileNames
- isValueContainsNull() - Method in class io.delta.kernel.types.MapType
- isWideningSupported(DataType, DataType) - Static method in class io.delta.kernel.internal.types.TypeWideningChecker
-
Checks if a type change from sourceType to targetType is a supported widening operation.
- iterator() - Method in interface io.delta.kernel.utils.CloseableIterable
-
Overrides the default iterator method to return a
CloseableIterator
.
J
- JsonHandler - Interface in io.delta.kernel.engine
-
Provides JSON handling functionality to Delta Kernel.
- JsonMetadataDomain - Class in io.delta.kernel.internal.metadatadomain
-
Abstract class representing a metadata domain, whose configuration string is a JSON serialization of a domain object.
- JsonMetadataDomain() - Constructor for class io.delta.kernel.internal.metadatadomain.JsonMetadataDomain
- JsonUtils - Class in io.delta.kernel.internal.util
- JsonUtils.JsonValueWriter<T> - Interface in io.delta.kernel.internal.util
- JsonUtils.ToJson - Interface in io.delta.kernel.internal.util
K
- KERNEL_VERSION - Static variable in class io.delta.kernel.Meta
- KernelEngineException - Exception in io.delta.kernel.exceptions
-
Throws when the
Engine
encountered an error while executing an operation. - KernelEngineException(String, Throwable) - Constructor for exception io.delta.kernel.exceptions.KernelEngineException
- KernelException - Exception in io.delta.kernel.exceptions
-
Thrown when Kernel cannot execute the requested operation due to the operation being invalid or unsupported.
- KernelException() - Constructor for exception io.delta.kernel.exceptions.KernelException
- KernelException(String) - Constructor for exception io.delta.kernel.exceptions.KernelException
- KernelException(String, Throwable) - Constructor for exception io.delta.kernel.exceptions.KernelException
- KernelException(Throwable) - Constructor for exception io.delta.kernel.exceptions.KernelException
- keySet() - Method in class io.delta.kernel.internal.util.CaseInsensitiveMap
L
- LAST_CHECKPOINT_FILE_NAME - Static variable in class io.delta.kernel.internal.checkpoints.Checkpointer
-
The name of the last checkpoint file
- Lazy<T> - Class in io.delta.kernel.internal.lang
- Lazy(Supplier<T>) - Constructor for class io.delta.kernel.internal.lang.Lazy
- LegacyReaderWriterFeature(String, int, int) - Constructor for class io.delta.kernel.internal.tablefeatures.TableFeature.LegacyReaderWriterFeature
- LegacyWriterFeature(String, int) - Constructor for class io.delta.kernel.internal.tablefeatures.TableFeature.LegacyWriterFeature
- length() - Method in class io.delta.kernel.types.StructType
- listDeltaLogFilesAsIter(Engine, Set<FileNames.DeltaLogFileType>, Path, long, Optional<Long>, boolean) - Static method in class io.delta.kernel.internal.DeltaLogActionUtils
-
Returns a
CloseableIterator
of files of type $fileTypes in the _delta_log directory of the given $tablePath, in increasing order from $startVersion to the optional $endVersion. - listFrom(String) - Method in interface io.delta.kernel.engine.FileSystemClient
-
List the paths in the same directory that are lexicographically greater or equal to (UTF-8 sorting) the given `path`.
- listingPrefix(Path, long) - Static method in class io.delta.kernel.internal.util.FileNames
-
Returns the prefix of all delta log files for the given version.
- ListUtils - Class in io.delta.kernel.internal.lang
- Literal - Class in io.delta.kernel.expressions
-
A literal value.
- load(FileSystemClient) - Method in class io.delta.kernel.internal.deletionvectors.DeletionVectorStoredBitmap
- loadInitialDeltaActionsTimer - Variable in class io.delta.kernel.internal.metrics.SnapshotMetrics
- loadNewDvAndBitmap(Engine, String, DeletionVectorDescriptor) - Static method in class io.delta.kernel.internal.deletionvectors.DeletionVectorUtils
- LOG_COMPACTION - Enum constant in enum io.delta.kernel.hook.PostCommitHook.PostCommitHookType
-
Writes a log compaction file that merges a range of commit JSON files into a single file.
- LOG_COMPACTION - Enum constant in enum io.delta.kernel.internal.files.ParsedLogData.ParsedLogCategory
- LOG_COMPACTION - Enum constant in enum io.delta.kernel.internal.files.ParsedLogData.ParsedLogType
- LOG_COMPACTION - Enum constant in enum io.delta.kernel.internal.replay.DeltaLogFile.LogType
- LOG_COMPACTION - Enum constant in enum io.delta.kernel.internal.util.FileNames.DeltaLogFileType
- LOG_RETENTION - Static variable in class io.delta.kernel.internal.TableConfig
-
The shortest duration we have to keep delta/checkpoint files around before deleting them.
- LogCompactionHook - Class in io.delta.kernel.internal.hook
-
A post-commit hook that performs inline log compaction.
- LogCompactionHook(Path, Path, long, long, long) - Constructor for class io.delta.kernel.internal.hook.LogCompactionHook
- logCompactionPath(Path, long, long) - Static method in class io.delta.kernel.internal.util.FileNames
-
Return the path that should be used for a log compaction file.
- logCompactionVersions(Path) - Static method in class io.delta.kernel.internal.util.FileNames
-
Returns the start and end versions for the given compaction path.
- logCompactionVersions(String) - Static method in class io.delta.kernel.internal.util.FileNames
- LogCompactionWriter - Class in io.delta.kernel.internal.compaction
-
Utility for writing out log compactions.
- LogCompactionWriter(Path, Path, long, long, long) - Constructor for class io.delta.kernel.internal.compaction.LogCompactionWriter
- LogReplay - Class in io.delta.kernel.internal.replay
-
Replays a history of actions, resolving them to produce the current state of the table.
- LogReplay(Path, Path, Engine, LogSegment, Optional<SnapshotHint>, SnapshotMetrics) - Constructor for class io.delta.kernel.internal.replay.LogReplay
- LogReplayUtils - Class in io.delta.kernel.internal.replay
- LogReplayUtils.UniqueFileActionTuple - Class in io.delta.kernel.internal.replay
- LogSegment - Class in io.delta.kernel.internal.snapshot
- LogSegment(Path, long, List<FileStatus>, List<FileStatus>, List<FileStatus>, Optional<FileStatus>, long) - Constructor for class io.delta.kernel.internal.snapshot.LogSegment
-
Provides information around which files in the transaction log need to be read to create the given version of the log.
- LONG - Static variable in class io.delta.kernel.types.LongType
- LongType - Class in io.delta.kernel.types
-
The data type representing
long
type values.
M
- MANUAL_UPDATE - Enum constant in enum io.delta.kernel.Operation
-
For any operation that doesn't fit the above categories.
- ManualClock - Class in io.delta.kernel.internal.util
-
A clock whose time can be manually set and modified.
- ManualClock(long) - Constructor for class io.delta.kernel.internal.util.ManualClock
- map(Function<T, U>) - Method in interface io.delta.kernel.utils.CloseableIterator
- mapper() - Static method in class io.delta.kernel.internal.util.JsonUtils
- MapType - Class in io.delta.kernel.types
-
Data type representing a
map
type. - MapType(DataType, DataType, boolean) - Constructor for class io.delta.kernel.types.MapType
- MapType(StructField, StructField) - Constructor for class io.delta.kernel.types.MapType
- MapValue - Interface in io.delta.kernel.data
-
Abstraction to represent a single map value in a
ColumnVector
. - MATERIALIZED_ROW_COMMIT_VERSION_COLUMN_NAME - Static variable in class io.delta.kernel.internal.TableConfig
- MATERIALIZED_ROW_ID_COLUMN_NAME - Static variable in class io.delta.kernel.internal.TableConfig
- MaterializedRowTrackingColumn - Class in io.delta.kernel.internal.rowtracking
-
A collection of helper methods for working with materialized row tracking columns.
- MAX - Static variable in class io.delta.kernel.internal.skipping.StatsSchemaHelper
- MAX_VALUE - Static variable in class io.delta.kernel.internal.checkpoints.CheckpointInstance
-
Placeholder to identify the version that is always the latest on timeline
- merge(Protocol...) - Method in class io.delta.kernel.internal.actions.Protocol
-
Merge this protocol with multiple `protocols` to have the highest reader and writer versions plus all explicitly and implicitly supported features.
- Meta - Class in io.delta.kernel
- Meta() - Constructor for class io.delta.kernel.Meta
- Metadata - Class in io.delta.kernel.internal.actions
- Metadata(String, Optional<String>, Optional<String>, Format, String, StructType, ArrayValue, Optional<Long>, MapValue) - Constructor for class io.delta.kernel.internal.actions.Metadata
- METADATA - Enum constant in enum io.delta.kernel.internal.DeltaLogActionUtils.DeltaAction
- METADATA_ROW_INDEX_COLUMN - Static variable in class io.delta.kernel.types.StructField
- METADATA_ROW_INDEX_COLUMN_NAME - Static variable in class io.delta.kernel.types.StructField
-
The name of a row index metadata column.
- metadataChangedException() - Static method in class io.delta.kernel.internal.DeltaErrors
- MetadataChangedException - Exception in io.delta.kernel.exceptions
-
Thrown when the metadata of the Delta table has changed between the time of transaction start and the time of commit.
- MetadataChangedException() - Constructor for exception io.delta.kernel.exceptions.MetadataChangedException
- MetadataCleanup - Class in io.delta.kernel.internal.snapshot
- metadataRequiresFeatureToBeEnabled(Protocol, Metadata) - Method in interface io.delta.kernel.internal.tablefeatures.FeatureAutoEnabledByMetadata
-
Determine whether the feature must be supported and enabled because its metadata requirements are satisfied.
- MetricsReport - Interface in io.delta.kernel.metrics
-
Interface containing the metrics for a given operation
- MetricsReporter - Interface in io.delta.kernel.engine
-
Interface for reporting metrics for operations to a Delta table
- MetricsReportSerializers - Class in io.delta.kernel.internal.metrics
-
Defines JSON serializers for
MetricsReport
types - MICROS_PER_DAY - Static variable in class io.delta.kernel.internal.util.DateTimeConstants
- MICROS_PER_HOUR - Static variable in class io.delta.kernel.internal.util.DateTimeConstants
- MICROS_PER_MILLIS - Static variable in class io.delta.kernel.internal.util.DateTimeConstants
- MICROS_PER_MINUTE - Static variable in class io.delta.kernel.internal.util.DateTimeConstants
- MICROS_PER_SECOND - Static variable in class io.delta.kernel.internal.util.DateTimeConstants
- MICROSECONDS_PER_SECOND - Static variable in class io.delta.kernel.statistics.DataFileStatistics
- microsSinceEpoch(Timestamp) - Static method in class io.delta.kernel.internal.util.InternalUtils
-
Utility method to get the number of microseconds since the unix epoch for the given timestamp interpreted in UTC.
- MILLIS_PER_DAY - Static variable in class io.delta.kernel.internal.util.DateTimeConstants
- MILLIS_PER_HOUR - Static variable in class io.delta.kernel.internal.util.DateTimeConstants
- MILLIS_PER_MINUTE - Static variable in class io.delta.kernel.internal.util.DateTimeConstants
- MILLIS_PER_SECOND - Static variable in class io.delta.kernel.internal.util.DateTimeConstants
- MIN - Static variable in class io.delta.kernel.internal.skipping.StatsSchemaHelper
- minimumRequiredVersions(Set<TableFeature>) - Static method in class io.delta.kernel.internal.tablefeatures.TableFeatures
-
Returns the minimum reader/writer versions required to support all provided features.
- minReaderVersion() - Method in class io.delta.kernel.internal.tablefeatures.TableFeature
- MINUTES_PER_HOUR - Static variable in class io.delta.kernel.internal.util.DateTimeConstants
- minWriterVersion() - Method in class io.delta.kernel.internal.tablefeatures.TableFeature
- MISSING_ROW_ID_HIGH_WATERMARK - Static variable in class io.delta.kernel.internal.rowtracking.RowTrackingMetadataDomain
- missingCheckpoint(String, long) - Static method in class io.delta.kernel.internal.DeltaErrors
- missingNumRecordsStatsForRowTracking() - Static method in class io.delta.kernel.internal.DeltaErrors
- missingRemoveFileSizeDuringCommit() - Static method in class io.delta.kernel.internal.DeltaErrorsInternal
- mkdirs(String) - Method in interface io.delta.kernel.engine.FileSystemClient
-
Create a directory at the given path including parent directories.
- MONTHS_PER_YEAR - Static variable in class io.delta.kernel.internal.util.DateTimeConstants
- MULTI_PART - Enum constant in enum io.delta.kernel.internal.checkpoints.CheckpointInstance.CheckpointFormat
- MULTI_PART_CHECKPOINT_FILE_PATTERN - Static variable in class io.delta.kernel.internal.util.FileNames
-
Example: 00000000000000000001.checkpoint.0000000020.0000000060.parquet
- MULTIPART_CHECKPOINT - Enum constant in enum io.delta.kernel.internal.files.ParsedLogData.ParsedLogType
- MULTIPART_CHECKPOINT - Enum constant in enum io.delta.kernel.internal.replay.DeltaLogFile.LogType
- multiPartCheckpointFile(Path, long, int, int) - Static method in class io.delta.kernel.internal.util.FileNames
- multiPartCheckpointPartAndNumParts(Path) - Static method in class io.delta.kernel.internal.util.FileNames
- multiPartCheckpointPartAndNumParts(String) - Static method in class io.delta.kernel.internal.util.FileNames
N
- NAME - Enum constant in enum io.delta.kernel.internal.util.ColumnMapping.ColumnMappingMode
- NANOS_PER_MICROS - Static variable in class io.delta.kernel.internal.util.DateTimeConstants
- NANOS_PER_MILLIS - Static variable in class io.delta.kernel.internal.util.DateTimeConstants
- NANOS_PER_SECOND - Static variable in class io.delta.kernel.internal.util.DateTimeConstants
- NANOSECONDS_PER_MICROSECOND - Static variable in class io.delta.kernel.statistics.DataFileStatistics
- next() - Method in class io.delta.kernel.internal.replay.ActionsIterator
- next() - Method in class io.delta.kernel.internal.replay.ActiveAddFilesIterator
- next() - Method in class io.delta.kernel.internal.replay.CreateCheckpointIterator
- next() - Method in interface io.delta.kernel.utils.CloseableIterator
-
Returns the next element in the iteration.
- noCommitFilesFoundForVersionRange(String, long, long) - Static method in class io.delta.kernel.internal.DeltaErrors
- NONE - Enum constant in enum io.delta.kernel.internal.util.ColumnMapping.ColumnMappingMode
- NOOP - Static variable in interface io.delta.kernel.internal.metrics.Timer.Timed
- normalized() - Method in class io.delta.kernel.internal.actions.Protocol
-
Protocol normalization is the process of converting a table features protocol to the weakest possible form.
- NULL_COUNT - Static variable in class io.delta.kernel.internal.skipping.StatsSchemaHelper
- NUM_RECORDS - Static variable in class io.delta.kernel.internal.skipping.StatsSchemaHelper
- numParts - Variable in class io.delta.kernel.internal.checkpoints.CheckpointInstance
- numParts - Variable in class io.delta.kernel.internal.files.ParsedMultiPartCheckpointData
O
- of(Metadata, Protocol, String, String, String, String) - Static method in class io.delta.kernel.internal.data.ScanStateRow
- of(Metadata, String, int) - Static method in class io.delta.kernel.internal.data.TransactionStateRow
- of(String) - Static method in class io.delta.kernel.utils.FileStatus
-
Create a
FileStatus
with the given path with size and modification time set to 0. - of(String, long, long) - Static method in class io.delta.kernel.utils.FileStatus
-
Create a
FileStatus
with the given path, size and modification time. - ofBinary(byte[]) - Static method in class io.delta.kernel.expressions.Literal
-
Create a
binary
type literal expression. - ofBoolean(boolean) - Static method in class io.delta.kernel.expressions.Literal
-
Create a
boolean
type literal expression. - ofByte(byte) - Static method in class io.delta.kernel.expressions.Literal
-
Create a
byte
type literal expression. - ofDate(int) - Static method in class io.delta.kernel.expressions.Literal
-
Create a
date
type literal expression. - ofDecimal(BigDecimal, int, int) - Static method in class io.delta.kernel.expressions.Literal
-
Create a
decimal
type literal expression. - ofDouble(double) - Static method in class io.delta.kernel.expressions.Literal
-
Create a
double
type literal expression. - ofFloat(float) - Static method in class io.delta.kernel.expressions.Literal
-
Create a
float
type literal expression. - ofInt(int) - Static method in class io.delta.kernel.expressions.Literal
-
Create a
integer
type literal expression. - ofLong(long) - Static method in class io.delta.kernel.expressions.Literal
-
Create a
long
type literal expression. - ofNull(DataType) - Static method in class io.delta.kernel.expressions.Literal
-
Create
null
value literal. - ofShort(short) - Static method in class io.delta.kernel.expressions.Literal
-
Create a
short
type literal expression. - ofSideCar(FileStatus, long) - Static method in class io.delta.kernel.internal.replay.DeltaLogFile
- ofString(String) - Static method in class io.delta.kernel.expressions.Literal
-
Create a
string
type literal expression. - ofTimestamp(long) - Static method in class io.delta.kernel.expressions.Literal
-
Create a
timestamp
type literal expression. - ofTimestampNtz(long) - Static method in class io.delta.kernel.expressions.Literal
-
Create a
timestamp_ntz
type literal expression. - Operation - Enum in io.delta.kernel
-
An operation that can be performed on a Delta table.
- Or - Class in io.delta.kernel.expressions
-
OR
expression - Or(Predicate, Predicate) - Constructor for class io.delta.kernel.expressions.Or
- overlappingTablePropertiesSetAndUnset(Set<String>) - Static method in class io.delta.kernel.internal.DeltaErrors
P
- PARQUET_FIELD_ID_KEY - Static variable in class io.delta.kernel.internal.util.ColumnMapping
- PARQUET_FIELD_NESTED_IDS_METADATA_KEY - Static variable in class io.delta.kernel.internal.util.ColumnMapping
- ParquetHandler - Interface in io.delta.kernel.engine
-
Provides Parquet file related functionalities to Delta Kernel.
- ParsedCheckpointData - Class in io.delta.kernel.internal.files
- ParsedLogCompactionData - Class in io.delta.kernel.internal.files
- ParsedLogData - Class in io.delta.kernel.internal.files
-
Represents a Delta Log "file" - the actual content may be materialized to disk (with a file status) or stored inline (as a columnar batch).
- ParsedLogData.ParsedLogCategory - Enum in io.delta.kernel.internal.files
- ParsedLogData.ParsedLogType - Enum in io.delta.kernel.internal.files
- ParsedMultiPartCheckpointData - Class in io.delta.kernel.internal.files
- parseIntervalAsMicros(String) - Static method in class io.delta.kernel.internal.util.IntervalParserUtils
- parseJson(ColumnVector, StructType, Optional<ColumnVector>) - Method in interface io.delta.kernel.engine.JsonHandler
-
Parse the given json strings and return the fields requested by
outputSchema
as columns in aColumnarBatch
. - parseJSONKeyValueMap(String) - Static method in class io.delta.kernel.internal.util.JsonUtils
-
Parses the given JSON string into a map of key-value pairs.
- parseJsonStats(Engine, FilteredColumnarBatch, StructType) - Static method in class io.delta.kernel.internal.skipping.DataSkippingUtils
-
Given a
FilteredColumnarBatch
of scan files and the statistics schema to parse, return the parsed JSON stats from the scan files. - part - Variable in class io.delta.kernel.internal.files.ParsedMultiPartCheckpointData
- partition(List<T>, Predicate<? super T>) - Static method in class io.delta.kernel.internal.lang.ListUtils
- partitionColumnMissingInData(String, String) - Static method in class io.delta.kernel.internal.DeltaErrors
- partitionExists(Engine, Snapshot, Predicate) - Static method in class io.delta.kernel.utils.PartitionUtils
-
Check if a partition exists (i.e.
- PartitionUtils - Class in io.delta.kernel.internal.util
- PartitionUtils - Class in io.delta.kernel.utils
- PartitionValueExpression - Class in io.delta.kernel.expressions
-
Expression to decode the serialized partition value into partition type value according the Delta Protocol spec.
- PartitionValueExpression(Expression, DataType) - Constructor for class io.delta.kernel.expressions.PartitionValueExpression
-
Create
partition_value
expression. - parts - Variable in class io.delta.kernel.internal.checkpoints.CheckpointMetaData
- Path - Class in io.delta.kernel.internal.fs
-
Names a file or directory in a FileSystem.
- Path(Path, Path) - Constructor for class io.delta.kernel.internal.fs.Path
-
Create a new Path based on the child path resolved against the parent path.
- Path(Path, String) - Constructor for class io.delta.kernel.internal.fs.Path
-
Create a new Path based on the child path resolved against the parent path.
- Path(String) - Constructor for class io.delta.kernel.internal.fs.Path
-
Construct a path from a String.
- Path(String, Path) - Constructor for class io.delta.kernel.internal.fs.Path
-
Create a new Path based on the child path resolved against the parent path.
- Path(String, String) - Constructor for class io.delta.kernel.internal.fs.Path
-
Create a new Path based on the child path resolved against the parent path.
- Path(String, String, String) - Constructor for class io.delta.kernel.internal.fs.Path
-
Construct a Path from components.
- Path(URI) - Constructor for class io.delta.kernel.internal.fs.Path
-
Construct a path from a URI
- PATH_DV_MARKER - Static variable in class io.delta.kernel.internal.actions.DeletionVectorDescriptor
- physicalSchemaWithoutPartitionColumns(StructType, StructType, Set<String>) - Static method in class io.delta.kernel.internal.util.PartitionUtils
-
Utility method to remove the given columns (as
columnsToRemove
) from the givenphysicalSchema
. - populateDomainMetadataMap(ColumnVector, Map<String, DomainMetadata>) - Static method in class io.delta.kernel.internal.util.DomainMetadataUtils
-
Populate the map of domain metadata from actions.
- PostCommitHook - Interface in io.delta.kernel.hook
-
A hook for executing operation after a transaction commit.
- PostCommitHook.PostCommitHookType - Enum in io.delta.kernel.hook
- Preconditions - Class in io.delta.kernel.internal.util
-
Static convenience methods that help a method or constructor check whether it was invoked correctly (that is, whether its preconditions were met).
- Predicate - Class in io.delta.kernel.expressions
-
Defines predicate scalar expression which is an extension of
ScalarExpression
that evaluates to true, false, or null for each input row. - Predicate(String, Expression) - Constructor for class io.delta.kernel.expressions.Predicate
-
Constructor for a unary Predicate expression
- Predicate(String, Expression, Expression) - Constructor for class io.delta.kernel.expressions.Predicate
-
Constructor for a binary Predicate expression
- Predicate(String, List<Expression>) - Constructor for class io.delta.kernel.expressions.Predicate
- PredicateEvaluator - Interface in io.delta.kernel.expressions
-
Special interface for evaluating
Predicate
on input batch and return a selection vector containing one value for each row in input batch indicating whether the row has passed the predicate or not. - Protocol - Class in io.delta.kernel.internal.actions
- Protocol(int, int) - Constructor for class io.delta.kernel.internal.actions.Protocol
- Protocol(int, int, Set<String>, Set<String>) - Constructor for class io.delta.kernel.internal.actions.Protocol
- PROTOCOL - Enum constant in enum io.delta.kernel.internal.DeltaLogActionUtils.DeltaAction
- PROTOCOL_METADATA_READ_SCHEMA - Static variable in class io.delta.kernel.internal.replay.LogReplay
-
Read schema when searching for the latest Protocol and Metadata.
- protocolChangedException(long) - Static method in class io.delta.kernel.internal.DeltaErrors
- ProtocolChangedException - Exception in io.delta.kernel.exceptions
-
Thrown when the protocol of the Delta table has changed between the time of transaction start and the time of commit.
- ProtocolChangedException(long) - Constructor for exception io.delta.kernel.exceptions.ProtocolChangedException
- pruneStatsSchema(StructType, Set<Column>) - Static method in class io.delta.kernel.internal.skipping.DataSkippingUtils
-
Prunes the given schema to only include the referenced leaf columns.
- PUBLISHED_DELTA - Enum constant in enum io.delta.kernel.internal.files.ParsedLogData.ParsedLogType
- put(String, V) - Method in class io.delta.kernel.internal.util.CaseInsensitiveMap
- putAll(Map<? extends String, ? extends V>) - Method in class io.delta.kernel.internal.util.CaseInsensitiveMap
- putBoolean(String, boolean) - Method in class io.delta.kernel.types.FieldMetadata.Builder
- putBooleanArray(String, Boolean[]) - Method in class io.delta.kernel.types.FieldMetadata.Builder
- putDouble(String, double) - Method in class io.delta.kernel.types.FieldMetadata.Builder
- putDoubleArray(String, Double[]) - Method in class io.delta.kernel.types.FieldMetadata.Builder
- putFieldMetadata(String, FieldMetadata) - Method in class io.delta.kernel.types.FieldMetadata.Builder
- putFieldMetadataArray(String, FieldMetadata[]) - Method in class io.delta.kernel.types.FieldMetadata.Builder
- putLong(String, long) - Method in class io.delta.kernel.types.FieldMetadata.Builder
- putLongArray(String, Long[]) - Method in class io.delta.kernel.types.FieldMetadata.Builder
- putNull(String) - Method in class io.delta.kernel.types.FieldMetadata.Builder
- putString(String, String) - Method in class io.delta.kernel.types.FieldMetadata.Builder
- putStringArray(String, String[]) - Method in class io.delta.kernel.types.FieldMetadata.Builder
R
- RATIFIED_INLINE_COMMIT - Enum constant in enum io.delta.kernel.internal.files.ParsedLogData.ParsedLogType
- RATIFIED_STAGED_COMMIT - Enum constant in enum io.delta.kernel.internal.files.ParsedLogData.ParsedLogType
- READ_SCHEMA - Static variable in class io.delta.kernel.internal.actions.DeletionVectorDescriptor
- READ_SCHEMA - Static variable in class io.delta.kernel.internal.checkpoints.CheckpointMetaData
- READ_SCHEMA - Static variable in class io.delta.kernel.internal.checkpoints.SidecarFile
- readCommitFiles(Engine, List<FileStatus>, StructType) - Static method in class io.delta.kernel.internal.DeltaLogActionUtils
-
Read the given commitFiles and return the contents as an iterator of batches.
- ReaderWriterFeature(String, int, int) - Constructor for class io.delta.kernel.internal.tablefeatures.TableFeature.ReaderWriterFeature
- readFiles(CloseableIterator<FileReadRequest>) - Method in interface io.delta.kernel.engine.FileSystemClient
-
Return an iterator of byte streams one for each read request in
readRequests
. - readJsonFiles(CloseableIterator<FileStatus>, StructType, Optional<Predicate>) - Method in interface io.delta.kernel.engine.JsonHandler
-
Read and parse the JSON format file at given locations and return the data as a
ColumnarBatch
with the columns requested byphysicalSchema
. - readLastCheckpointFile(Engine) - Method in class io.delta.kernel.internal.checkpoints.Checkpointer
-
Returns information about the most recent checkpoint.
- readParquetFiles(CloseableIterator<FileStatus>, StructType, Optional<Predicate>) - Method in interface io.delta.kernel.engine.ParquetHandler
-
Read the Parquet format files at the given locations and return the data as a
ColumnarBatch
with the columns requested byphysicalSchema
. - record(long) - Method in class io.delta.kernel.internal.metrics.Timer
-
Records a custom amount.
- relativizePath(Path, URI) - Static method in class io.delta.kernel.internal.util.InternalUtils
-
Relativize the given child path with respect to the given root URI.
- remove(long) - Method in class io.delta.kernel.internal.stats.FileSizeHistogram
-
Removes a file size from the histogram, decrementing the appropriate bin's count and total bytes.
- remove(Object) - Method in class io.delta.kernel.internal.util.CaseInsensitiveMap
- REMOVE - Enum constant in enum io.delta.kernel.internal.DeltaLogActionUtils.DeltaAction
- REMOVE_FILE_DV_ORDINAL - Static variable in class io.delta.kernel.internal.replay.LogReplay
- REMOVE_FILE_ORDINAL - Static variable in class io.delta.kernel.internal.actions.SingleAction
- REMOVE_FILE_ORDINAL - Static variable in class io.delta.kernel.internal.replay.LogReplay
- REMOVE_FILE_PATH_ORDINAL - Static variable in class io.delta.kernel.internal.replay.LogReplay
- removed() - Method in class io.delta.kernel.internal.actions.DomainMetadata
- removeDomainMetadata(String) - Method in class io.delta.kernel.internal.TransactionImpl
- removeDomainMetadata(String) - Method in interface io.delta.kernel.Transaction
-
Mark the domain metadata with identifier
domain
as removed in this transaction. - removeDomainMetadataInternal(String) - Method in class io.delta.kernel.internal.TransactionImpl
- RemoveFile - Class in io.delta.kernel.internal.actions
-
Metadata about
remove
action in the Delta Log. - RemoveFile(Row) - Constructor for class io.delta.kernel.internal.actions.RemoveFile
-
Constructs an
RemoveFile
action from the given 'RemoveFile'Row
. - REMOVEFILE_FIELD_NAME - Static variable in class io.delta.kernel.internal.replay.LogReplay
- removeFilesFromDeltaFilesCounter - Variable in class io.delta.kernel.internal.metrics.ScanMetrics
- REPLACE_TABLE - Enum constant in enum io.delta.kernel.Operation
-
Recorded during REPLACE operation (may also be considered an overwrite)
- ReplaceTableTransactionBuilderImpl - Class in io.delta.kernel.internal
- ReplaceTableTransactionBuilderImpl(TableImpl, String) - Constructor for class io.delta.kernel.internal.ReplaceTableTransactionBuilderImpl
- report(MetricsReport) - Method in interface io.delta.kernel.engine.MetricsReporter
-
Indicates that an operation is done by reporting a
MetricsReport
- requiredFeatures() - Method in class io.delta.kernel.internal.tablefeatures.TableFeature
-
Set of table features that this table feature depends on.
- requireNonNull(ColumnVector, int, String) - Static method in class io.delta.kernel.internal.util.InternalUtils
- requireNonNull(Row, int, String) - Static method in class io.delta.kernel.internal.util.InternalUtils
- requireSchemaForReplaceTable() - Static method in class io.delta.kernel.internal.DeltaErrors
- requiresSchemaForNewTable(String) - Static method in class io.delta.kernel.internal.DeltaErrors
- reset() - Method in class io.delta.kernel.internal.metrics.Counter
-
Resets the current count to 0.
- resetActionMetricsForRetry() - Method in class io.delta.kernel.internal.metrics.TransactionMetrics
-
Resets any action metrics for a failed commit to prepare them for retrying.
- resolveConflicts(Engine) - Method in class io.delta.kernel.internal.replay.ConflictChecker
- resolveConflicts(Engine, SnapshotImpl, long, TransactionImpl, List<DomainMetadata>, CloseableIterable<Row>) - Static method in class io.delta.kernel.internal.replay.ConflictChecker
-
Resolve conflicts between the losing transaction and the winning transactions and return a rebase state that the losing transaction needs to rebase against before attempting the commit.
- resolvePath(String) - Method in interface io.delta.kernel.engine.FileSystemClient
-
Resolve the given path to a fully qualified path.
- rewritePartitionPredicateOnCheckpointFileSchema(Predicate, Map<String, StructField>) - Static method in class io.delta.kernel.internal.util.PartitionUtils
-
Rewrite the given predicate on partition columns on `partitionValues_parsed` in checkpoint schema.
- rewritePartitionPredicateOnScanFileSchema(Predicate, Map<String, StructField>) - Static method in class io.delta.kernel.internal.util.PartitionUtils
-
Utility method to rewrite the partition predicate referring to the table schema as predicate referring to the
partitionValues
in scan files read from Delta log. - RoaringBitmapArray - Class in io.delta.kernel.internal.deletionvectors
-
A 64-bit extension of [[RoaringBitmap]] that is optimized for cases that usually fit within a 32-bit bitmap, but may run over by a few bits on occasion.
- RoaringBitmapArray() - Constructor for class io.delta.kernel.internal.deletionvectors.RoaringBitmapArray
- Row - Interface in io.delta.kernel.data
-
Represent a single record
- ROW_COMMIT_VERSION - Static variable in class io.delta.kernel.internal.rowtracking.MaterializedRowTrackingColumn
-
Static instance for the materialized row commit version column.
- ROW_ID - Static variable in class io.delta.kernel.internal.rowtracking.MaterializedRowTrackingColumn
-
Static instance for the materialized row ID column.
- ROW_TRACKING_ENABLED - Static variable in class io.delta.kernel.internal.TableConfig
-
Indicates whether Row Tracking is enabled on the table.
- ROW_TRACKING_W_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- RowBackedAction - Class in io.delta.kernel.internal.actions
-
An abstract base class for Delta Log actions that are backed by a
Row
. - RowTracking - Class in io.delta.kernel.internal.rowtracking
-
A collection of helper methods for working with row tracking.
- RowTrackingMetadataDomain - Class in io.delta.kernel.internal.rowtracking
-
Represents the metadata domain for row tracking.
- RowTrackingMetadataDomain(long) - Constructor for class io.delta.kernel.internal.rowtracking.RowTrackingMetadataDomain
-
Constructs a RowTrackingMetadataDomain with the specified row ID high water mark.
- rowTrackingSupportedWithDomainMetadataUnsupported() - Static method in class io.delta.kernel.internal.DeltaErrors
S
- safeParseIntervalAsMillis(String) - Static method in class io.delta.kernel.internal.util.IntervalParserUtils
-
Parse the given interval string into milliseconds.
- ScalarExpression - Class in io.delta.kernel.expressions
-
Scalar SQL expressions which take zero or more inputs and for each input row generate one output value.
- ScalarExpression(String, List<Expression>) - Constructor for class io.delta.kernel.expressions.ScalarExpression
- Scan - Interface in io.delta.kernel
-
Represents a scan of a Delta table.
- SCAN_FILE_SCHEMA - Static variable in class io.delta.kernel.internal.InternalScanFileUtils
-
Schema of the returned scan files.
- SCAN_FILE_SCHEMA_WITH_STATS - Static variable in class io.delta.kernel.internal.InternalScanFileUtils
-
Schema of the returned scan files when
ScanImpl.getScanFiles(Engine, boolean)
is called withincludeStats=true
. - ScanBuilder - Interface in io.delta.kernel
-
Builder to construct
Scan
object. - ScanBuilderImpl - Class in io.delta.kernel.internal
-
Implementation of
ScanBuilder
. - ScanBuilderImpl(Path, Protocol, Metadata, StructType, LogReplay, SnapshotReport) - Constructor for class io.delta.kernel.internal.ScanBuilderImpl
- ScanImpl - Class in io.delta.kernel.internal
-
Implementation of
Scan
- ScanImpl(StructType, StructType, Protocol, Metadata, LogReplay, Optional<Predicate>, Path, SnapshotReport) - Constructor for class io.delta.kernel.internal.ScanImpl
- ScanMetrics - Class in io.delta.kernel.internal.metrics
-
Stores the metrics for an ongoing scan.
- ScanMetrics() - Constructor for class io.delta.kernel.internal.metrics.ScanMetrics
- ScanMetricsResult - Interface in io.delta.kernel.metrics
-
Stores the metrics results for a
ScanReport
- ScanReport - Interface in io.delta.kernel.metrics
-
Defines the metadata and metrics for a Scan
MetricsReport
- ScanReportImpl - Class in io.delta.kernel.internal.metrics
-
A basic POJO implementation of
ScanReport
for creating them - ScanReportImpl(String, long, StructType, UUID, Optional<Predicate>, StructType, Optional<Predicate>, Optional<Predicate>, boolean, ScanMetrics, Optional<Exception>) - Constructor for class io.delta.kernel.internal.metrics.ScanReportImpl
- ScanStateRow - Class in io.delta.kernel.internal.data
-
Encapsulate the scan state (common info for all scan files) as a
Row
- ScanStateRow(HashMap<Integer, Object>) - Constructor for class io.delta.kernel.internal.data.ScanStateRow
- schema - Variable in enum io.delta.kernel.internal.DeltaLogActionUtils.DeltaAction
- SCHEMA_WITH_STATS - Static variable in class io.delta.kernel.internal.actions.AddFile
- SCHEMA_WITHOUT_STATS - Static variable in class io.delta.kernel.internal.actions.AddFile
-
Schema of the
add
action in the Delta Log without stats. - SchemaUtils - Class in io.delta.kernel.internal.util
-
Utility methods for schema related operations such as validating the schema has no duplicate columns and the names contain only valid characters.
- SECONDS_PER_DAY - Static variable in class io.delta.kernel.internal.util.DateTimeConstants
- SECONDS_PER_HOUR - Static variable in class io.delta.kernel.internal.util.DateTimeConstants
- SECONDS_PER_MINUTE - Static variable in class io.delta.kernel.internal.util.DateTimeConstants
- SelectionColumnVector - Class in io.delta.kernel.internal.data
-
The selection vector for a columnar batch as a boolean
ColumnVector
. - SelectionColumnVector(RoaringBitmapArray, ColumnVector) - Constructor for class io.delta.kernel.internal.data.SelectionColumnVector
- SEPARATOR - Static variable in class io.delta.kernel.internal.fs.Path
-
The directory separator, a slash.
- SEPARATOR_CHAR - Static variable in class io.delta.kernel.internal.fs.Path
-
The directory separator, a slash, as a character.
- serializeAsJson(StructType) - Method in class io.delta.kernel.statistics.DataFileStatistics
-
Serializes the statistics as a JSON string.
- serializeDataType(DataType) - Static method in class io.delta.kernel.internal.types.DataTypeJsonSerDe
-
Serializes a
DataType
to a JSON string according to the Delta Protocol. - serializePartitionMap(Map<String, Literal>) - Static method in class io.delta.kernel.internal.util.PartitionUtils
-
Convert the given partition values to a
MapValue
that can be serialized to a Delta commit file. - serializeScanReport(ScanReport) - Static method in class io.delta.kernel.internal.metrics.MetricsReportSerializers
-
Serializes a
ScanReport
to a JSON string - serializeSnapshotReport(SnapshotReport) - Static method in class io.delta.kernel.internal.metrics.MetricsReportSerializers
-
Serializes a
SnapshotReport
to a JSON string - serializeStructType(StructType) - Static method in class io.delta.kernel.internal.types.DataTypeJsonSerDe
-
Serializes a
StructType
to a JSON string - serializeTransactionReport(TransactionReport) - Static method in class io.delta.kernel.internal.metrics.MetricsReportSerializers
-
Serializes a
TransactionReport
to a JSON string - SET_TABLE_FEATURE_SUPPORTED_PREFIX - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
-
The prefix for setting an override of a feature option in Metadata configuration.
- SET_TRANSACTION_READ_SCHEMA - Static variable in class io.delta.kernel.internal.replay.LogReplay
-
Read schema when searching for just the transaction identifiers
- setInCommitTimestamp(Optional<Long>) - Method in class io.delta.kernel.internal.actions.CommitInfo
- setTime(long) - Method in class io.delta.kernel.internal.util.ManualClock
- SetTransaction - Class in io.delta.kernel.internal.actions
-
Delta log action representing a transaction identifier action.
- SetTransaction(String, Long, Optional<Long>) - Constructor for class io.delta.kernel.internal.actions.SetTransaction
- setVersion(long) - Method in class io.delta.kernel.internal.metrics.SnapshotQueryContext
-
Updates the
version
stored in this snapshot context. - SHORT - Static variable in class io.delta.kernel.types.ShortType
- ShortType - Class in io.delta.kernel.types
-
The data type representing
short
type values. - shouldCompact(long, long) - Static method in class io.delta.kernel.internal.compaction.LogCompactionWriter
-
Utility to determine if log compaction should run for the given commit version.
- SIDECAR - Enum constant in enum io.delta.kernel.internal.replay.DeltaLogFile.LogType
- SIDECAR_DIRECTORY - Static variable in class io.delta.kernel.internal.util.FileNames
- SIDECAR_FIELD_NAME - Static variable in class io.delta.kernel.internal.replay.LogReplay
- sidecarFile(Path, String) - Static method in class io.delta.kernel.internal.util.FileNames
-
Example: /a/_sidecars/3a0d65cd-4056-49b8-937b-95f9e3ee90e5.parquet
- SidecarFile - Class in io.delta.kernel.internal.checkpoints
-
Action representing a SidecarFile in a top-level V2 checkpoint file.
- SidecarFile(String, long, long) - Constructor for class io.delta.kernel.internal.checkpoints.SidecarFile
- SingleAction - Class in io.delta.kernel.internal.actions
- SingleAction() - Constructor for class io.delta.kernel.internal.actions.SingleAction
- singletonCloseableIterator(T) - Static method in class io.delta.kernel.internal.util.Utils
-
Utility method to create a singleton
CloseableIterator
. - singletonStringColumnVector(String) - Static method in class io.delta.kernel.internal.util.InternalUtils
-
Utility method to create a singleton string
ColumnVector
- size - Variable in class io.delta.kernel.internal.checkpoints.CheckpointMetaData
- size() - Method in class io.delta.kernel.internal.util.CaseInsensitiveMap
- Snapshot - Interface in io.delta.kernel
-
Represents the snapshot of a Delta table.
- SnapshotHint - Class in io.delta.kernel.internal.snapshot
-
Contains summary information of a
Snapshot
. - SnapshotHint(long, Protocol, Metadata) - Constructor for class io.delta.kernel.internal.snapshot.SnapshotHint
- SnapshotImpl - Class in io.delta.kernel.internal
-
Implementation of
Snapshot
. - SnapshotImpl(Path, LogSegment, LogReplay, Protocol, Metadata, SnapshotQueryContext) - Constructor for class io.delta.kernel.internal.SnapshotImpl
- SnapshotManager - Class in io.delta.kernel.internal.snapshot
- SnapshotManager(Path) - Constructor for class io.delta.kernel.internal.snapshot.SnapshotManager
- SnapshotMetrics - Class in io.delta.kernel.internal.metrics
-
Stores the metrics for an ongoing snapshot construction.
- SnapshotMetrics() - Constructor for class io.delta.kernel.internal.metrics.SnapshotMetrics
- SnapshotMetricsResult - Interface in io.delta.kernel.metrics
-
Stores the metrics results for a
SnapshotReport
- SnapshotQueryContext - Class in io.delta.kernel.internal.metrics
-
Stores the context for a given Snapshot query.
- SnapshotReport - Interface in io.delta.kernel.metrics
-
Defines the metadata and metrics for a snapshot construction
MetricsReport
- SnapshotReportImpl - Class in io.delta.kernel.internal.metrics
-
A basic POJO implementation of
SnapshotReport
for creating them - spliterator() - Method in interface io.delta.kernel.utils.CloseableIterable
- splitMetadataAndDataPredicates(Predicate, Set<String>) - Static method in class io.delta.kernel.internal.util.PartitionUtils
-
Split the given predicate into predicate on partition columns and predicate on data columns.
- STAGED_COMMIT_DIRECTORY - Static variable in class io.delta.kernel.internal.util.FileNames
- stagedCommitFile(Path, long) - Static method in class io.delta.kernel.internal.util.FileNames
- start() - Method in class io.delta.kernel.internal.metrics.Timer
-
Starts the timer and returns a
Timer.Timed
instance. - startVersion - Variable in class io.delta.kernel.internal.files.ParsedLogCompactionData
- startVersionNotFound(String, long, Optional<Long>) - Static method in class io.delta.kernel.internal.DeltaErrors
- StatsSchemaHelper - Class in io.delta.kernel.internal.skipping
-
Provides information and utilities for statistics columns given a table schema.
- StatsSchemaHelper(StructType) - Constructor for class io.delta.kernel.internal.skipping.StatsSchemaHelper
- statsTypeMismatch(String, DataType, DataType) - Static method in class io.delta.kernel.internal.DeltaErrors
- StatsUtils - Class in io.delta.kernel.internal.util
-
Encapsulates various utility methods for statistics related operations.
- stop() - Method in interface io.delta.kernel.internal.metrics.Timer.Timed
-
Stops the timer and records the total duration up until
Timer.start()
was called. - STREAMING_UPDATE - Enum constant in enum io.delta.kernel.Operation
-
Recorded during streaming inserts.
- STRING - Static variable in class io.delta.kernel.types.StringType
- stringStringMapValue(Map<String, String>) - Static method in class io.delta.kernel.internal.util.VectorUtils
-
Creates a
MapValue
from map of string keys and string values. - StringType - Class in io.delta.kernel.types
-
The data type representing
string
type values. - StringType(CollationIdentifier) - Constructor for class io.delta.kernel.types.StringType
- StringType(String) - Constructor for class io.delta.kernel.types.StringType
- StructField - Class in io.delta.kernel.types
-
Represents a subfield of
StructType
with additional properties and metadata. - StructField(String, DataType, boolean) - Constructor for class io.delta.kernel.types.StructField
- StructField(String, DataType, boolean, FieldMetadata) - Constructor for class io.delta.kernel.types.StructField
- StructRow - Class in io.delta.kernel.internal.data
-
A
Row
abstraction for a struct type column vector and a specificrowId
. - StructType - Class in io.delta.kernel.types
-
Struct type which contains one or more columns.
- StructType() - Constructor for class io.delta.kernel.types.StructType
- StructType(List<StructField>) - Constructor for class io.delta.kernel.types.StructType
- supportsFeature(TableFeature) - Method in class io.delta.kernel.internal.actions.Protocol
-
Check if the protocol supports the given table feature
- supportsReaderFeatures(int) - Static method in class io.delta.kernel.internal.tablefeatures.TableFeatures
-
Does reader version supports explicitly specifying reader feature set in protocol?
- supportsWriterFeatures(int) - Static method in class io.delta.kernel.internal.tablefeatures.TableFeatures
-
Does writer version supports explicitly specifying writer feature set in protocol?
T
- Table - Interface in io.delta.kernel
-
Represents the Delta Lake table for a given path.
- TABLE_FEATURE_MAP - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- TABLE_FEATURES - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- TABLE_FEATURES_MIN_READER_VERSION - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
-
Min reader version that supports reader features.
- TABLE_FEATURES_MIN_WRITER_VERSION - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
-
Min reader version that supports writer features.
- TABLE_ROOT_STRUCT_FIELD - Static variable in class io.delta.kernel.internal.InternalScanFileUtils
- tableAlreadyExists(String, String) - Static method in class io.delta.kernel.internal.DeltaErrors
- TableAlreadyExistsException - Exception in io.delta.kernel.exceptions
-
Thrown when trying to create a Delta table at a location where a Delta table already exists.
- TableAlreadyExistsException(String) - Constructor for exception io.delta.kernel.exceptions.TableAlreadyExistsException
- TableAlreadyExistsException(String, String) - Constructor for exception io.delta.kernel.exceptions.TableAlreadyExistsException
- TableConfig<T> - Class in io.delta.kernel.internal
-
Represents the table properties.
- TableConfig.UniversalFormats - Class in io.delta.kernel.internal
- TableFeature - Class in io.delta.kernel.internal.tablefeatures
-
Base class for table features.
- TableFeature(String, int, int) - Constructor for class io.delta.kernel.internal.tablefeatures.TableFeature
-
Constructor.
- TableFeature.LegacyFeatureType - Interface in io.delta.kernel.internal.tablefeatures
-
An interface to indicate a feature is legacy, i.e., released before Table Features.
- TableFeature.LegacyReaderWriterFeature - Class in io.delta.kernel.internal.tablefeatures
-
A base class for all table legacy reader-writer features.
- TableFeature.LegacyWriterFeature - Class in io.delta.kernel.internal.tablefeatures
-
A base class for all table legacy writer-only features.
- TableFeature.ReaderWriterFeature - Class in io.delta.kernel.internal.tablefeatures
-
A base class for all non-legacy table reader-writer features.
- TableFeature.ReaderWriterFeatureType - Interface in io.delta.kernel.internal.tablefeatures
-
An interface to indicate a feature applies to readers and writers.
- TableFeature.WriterFeature - Class in io.delta.kernel.internal.tablefeatures
-
A base class for all non-legacy table writer features.
- TableFeatures - Class in io.delta.kernel.internal.tablefeatures
-
Contains utility methods related to the Delta table feature support in protocol.
- TableFeatures() - Constructor for class io.delta.kernel.internal.tablefeatures.TableFeatures
- TableImpl - Class in io.delta.kernel.internal
- TableImpl(String, Clock) - Constructor for class io.delta.kernel.internal.TableImpl
- TableNotFoundException - Exception in io.delta.kernel.exceptions
-
Thrown when there is no Delta table at the given location.
- TableNotFoundException(String) - Constructor for exception io.delta.kernel.exceptions.TableNotFoundException
- TableNotFoundException(String, String) - Constructor for exception io.delta.kernel.exceptions.TableNotFoundException
- takeWhile(Function<T, Boolean>) - Method in interface io.delta.kernel.utils.CloseableIterator
-
Returns a new
CloseableIterator
that includes elements from this iterator as long as the givenmapper
function returnstrue
. - threadSafeInvoke(Engine) - Method in interface io.delta.kernel.hook.PostCommitHook
-
Invokes the post commit operation whose implementation must be thread safe.
- threadSafeInvoke(Engine) - Method in class io.delta.kernel.internal.hook.CheckpointHook
- threadSafeInvoke(Engine) - Method in class io.delta.kernel.internal.hook.ChecksumFullHook
- threadSafeInvoke(Engine) - Method in class io.delta.kernel.internal.hook.ChecksumSimpleHook
- threadSafeInvoke(Engine) - Method in class io.delta.kernel.internal.hook.LogCompactionHook
- throwIfColumnNamesConflictWithSchema(Metadata) - Static method in class io.delta.kernel.internal.rowtracking.MaterializedRowTrackingColumn
-
Validates that the materialized column names for ROW_ID and ROW_COMMIT_VERSION do not conflict with any existing logical or physical column names in the table's schema.
- throwIfRowTrackingToggled(Metadata, Metadata) - Static method in class io.delta.kernel.internal.rowtracking.RowTracking
-
Throws an exception if row tracking enablement is toggled between the old and the new metadata.
- time(Runnable) - Method in class io.delta.kernel.internal.metrics.Timer
- time(Supplier<T>) - Method in class io.delta.kernel.internal.metrics.Timer
- timeCallable(Callable<T>) - Method in class io.delta.kernel.internal.metrics.Timer
- Timer - Class in io.delta.kernel.internal.metrics
-
A timer class for measuring the duration of operations in nanoseconds
- Timer() - Constructor for class io.delta.kernel.internal.metrics.Timer
- Timer.Timed - Interface in io.delta.kernel.internal.metrics
-
A timing sample that carries internal state about the Timer's start position.
- TIMESTAMP - Static variable in class io.delta.kernel.types.TimestampType
- TIMESTAMP_NTZ - Static variable in class io.delta.kernel.types.TimestampNTZType
- TIMESTAMP_NTZ_RW_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- timestampAfterLatestCommit(String, long, long, long) - Static method in class io.delta.kernel.internal.DeltaErrors
- timestampBeforeFirstAvailableCommit(String, long, long, long) - Static method in class io.delta.kernel.internal.DeltaErrors
- TimestampNTZType - Class in io.delta.kernel.types
-
The timestamp without time zone type represents a local time in microsecond precision, which is independent of time zone.
- timestampToVersionResolutionTimer - Variable in class io.delta.kernel.internal.metrics.SnapshotMetrics
- TimestampType - Class in io.delta.kernel.types
-
A timestamp type, supporting [0001-01-01T00:00:00.000000Z, 9999-12-31T23:59:59.999999Z] where the left/right-bound is a date and time of the proleptic Gregorian calendar in UTC+00:00.
- toArray() - Method in class io.delta.kernel.internal.deletionvectors.RoaringBitmapArray
- toCloseableIterator(Iterator<T>) - Static method in class io.delta.kernel.internal.util.Utils
-
Convert a
Iterator
toCloseableIterator
. - toDomainMetadata() - Method in class io.delta.kernel.internal.metadatadomain.JsonMetadataDomain
-
Generate a
DomainMetadata
action from this metadata domain. - toInMemoryList() - Method in interface io.delta.kernel.utils.CloseableIterator
-
Collects all elements from this
CloseableIterator
into aList
. - toJavaList(ArrayValue) - Static method in class io.delta.kernel.internal.util.VectorUtils
-
Converts an
ArrayValue
to a Java list. - toJavaMap(MapValue) - Static method in class io.delta.kernel.internal.util.VectorUtils
-
Converts a
MapValue
to a Java map. - toJson() - Method in class io.delta.kernel.types.StructType
-
Convert the struct type to Delta protocol specified serialization format.
- toJsonConfiguration() - Method in class io.delta.kernel.internal.metadatadomain.JsonMetadataDomain
-
Serializes this object into a JSON string.
- toLowerCaseSet(Collection<String>) - Static method in class io.delta.kernel.internal.util.InternalUtils
- TOMBSTONE_RETENTION - Static variable in class io.delta.kernel.internal.TableConfig
-
The shortest duration we have to keep logically deleted data files around before deleting them physically.
- topLevelV2CheckpointFile(Path, long, String, String) - Static method in class io.delta.kernel.internal.util.FileNames
-
Returns the path for a top-level V2 checkpoint file up to the given version with a given UUID and filetype (JSON or Parquet).
- toRemoveFileRow(boolean, Optional<Long>) - Method in class io.delta.kernel.internal.actions.AddFile
-
Utility to generate a 'RemoveFile' action from this AddFile action.
- toRow() - Method in class io.delta.kernel.internal.actions.CommitInfo
-
Encode as a
Row
object with the schemaCommitInfo.FULL_SCHEMA
. - toRow() - Method in class io.delta.kernel.internal.actions.DomainMetadata
-
Encode as a
Row
object with the schemaDomainMetadata.FULL_SCHEMA
. - toRow() - Method in class io.delta.kernel.internal.actions.Format
-
Encode as a
Row
object with the schemaFormat.FULL_SCHEMA
. - toRow() - Method in class io.delta.kernel.internal.actions.Metadata
-
Encode as a
Row
object with the schemaMetadata.FULL_SCHEMA
. - toRow() - Method in class io.delta.kernel.internal.actions.Protocol
-
Encode as a
Row
object with the schemaProtocol.FULL_SCHEMA
. - toRow() - Method in class io.delta.kernel.internal.actions.RowBackedAction
-
Returns the underlying
Row
that represents this action. - toRow() - Method in class io.delta.kernel.internal.actions.SetTransaction
-
Encode as a
Row
object with the schemaSetTransaction.FULL_SCHEMA
. - toRow() - Method in class io.delta.kernel.internal.checkpoints.CheckpointMetaData
- toRow() - Method in class io.delta.kernel.internal.checksum.CRCInfo
-
Encode as a
Row
object with the schemaCRCInfo.CRC_FILE_SCHEMA
. - toRow() - Method in class io.delta.kernel.internal.stats.FileSizeHistogram
-
Encode as a
Row
object with the schemaFileSizeHistogram.FULL_SCHEMA
. - toString() - Method in class io.delta.kernel.expressions.Column
- toString() - Method in class io.delta.kernel.expressions.Literal
- toString() - Method in class io.delta.kernel.expressions.PartitionValueExpression
- toString() - Method in class io.delta.kernel.expressions.Predicate
- toString() - Method in class io.delta.kernel.expressions.ScalarExpression
- toString() - Method in class io.delta.kernel.internal.actions.AddFile
- toString() - Method in class io.delta.kernel.internal.actions.DeletionVectorDescriptor
- toString() - Method in class io.delta.kernel.internal.actions.DomainMetadata
- toString() - Method in class io.delta.kernel.internal.actions.Format
- toString() - Method in class io.delta.kernel.internal.actions.Metadata
- toString() - Method in class io.delta.kernel.internal.actions.Protocol
- toString() - Method in class io.delta.kernel.internal.actions.RemoveFile
- toString() - Method in class io.delta.kernel.internal.checkpoints.CheckpointInstance
- toString() - Method in class io.delta.kernel.internal.checkpoints.CheckpointMetaData
- toString() - Method in class io.delta.kernel.internal.files.ParsedLogData
- toString() - Method in class io.delta.kernel.internal.fs.Path
- toString() - Method in class io.delta.kernel.internal.metrics.Counter
- toString() - Method in class io.delta.kernel.internal.metrics.ScanMetrics
- toString() - Method in class io.delta.kernel.internal.metrics.SnapshotMetrics
- toString() - Method in class io.delta.kernel.internal.metrics.SnapshotQueryContext
- toString() - Method in class io.delta.kernel.internal.metrics.Timer
- toString() - Method in class io.delta.kernel.internal.metrics.TransactionMetrics
- toString() - Method in class io.delta.kernel.internal.rowtracking.RowTrackingMetadataDomain
- toString() - Method in class io.delta.kernel.internal.snapshot.LogSegment
- toString() - Method in enum io.delta.kernel.internal.util.ColumnMapping.ColumnMappingMode
- toString() - Method in class io.delta.kernel.statistics.DataFileStatistics
- toString() - Method in class io.delta.kernel.types.ArrayType
- toString() - Method in class io.delta.kernel.types.BasePrimitiveType
- toString() - Method in class io.delta.kernel.types.CollationIdentifier
- toString() - Method in class io.delta.kernel.types.DataType
- toString() - Method in class io.delta.kernel.types.DecimalType
- toString() - Method in class io.delta.kernel.types.FieldMetadata
- toString() - Method in class io.delta.kernel.types.MapType
- toString() - Method in class io.delta.kernel.types.StructField
- toString() - Method in class io.delta.kernel.types.StructType
- toString() - Method in class io.delta.kernel.utils.DataFileStatus
- toString() - Method in class io.delta.kernel.utils.FileStatus
- toStringWithoutVersion() - Method in class io.delta.kernel.types.CollationIdentifier
- totalActionsCounter - Variable in class io.delta.kernel.internal.metrics.TransactionMetrics
- totalCommitTimer - Variable in class io.delta.kernel.internal.metrics.TransactionMetrics
- totalDurationIfRecorded() - Method in class io.delta.kernel.internal.metrics.Timer
- totalDurationMs() - Method in class io.delta.kernel.internal.metrics.Timer
- totalDurationNs() - Method in class io.delta.kernel.internal.metrics.Timer
- totalPlanningTimer - Variable in class io.delta.kernel.internal.metrics.ScanMetrics
- toUri() - Method in class io.delta.kernel.internal.fs.Path
-
Convert this Path to a URI.
- Transaction - Interface in io.delta.kernel
-
Represents a transaction to mutate a Delta table.
- TransactionBuilder - Interface in io.delta.kernel
-
Builder for creating a
Transaction
to mutate a Delta table. - TransactionBuilderImpl - Class in io.delta.kernel.internal
- TransactionBuilderImpl(TableImpl, String, Operation) - Constructor for class io.delta.kernel.internal.TransactionBuilderImpl
- TransactionCommitResult - Class in io.delta.kernel
-
Contains the result of a successful transaction commit.
- TransactionCommitResult(long, List<PostCommitHook>, TransactionMetricsResult) - Constructor for class io.delta.kernel.TransactionCommitResult
- TransactionImpl - Class in io.delta.kernel.internal
- TransactionImpl(boolean, Path, Path, SnapshotImpl, String, Operation, Protocol, Metadata, Optional<SetTransaction>, Optional<List<Column>>, boolean, boolean, int, int, Clock) - Constructor for class io.delta.kernel.internal.TransactionImpl
- TransactionMetrics - Class in io.delta.kernel.internal.metrics
-
Stores the metrics for an ongoing transaction.
- TransactionMetricsResult - Interface in io.delta.kernel.metrics
-
Stores the metrics results for a
TransactionReport
- TransactionRebaseState(long, long, CloseableIterable<Row>, List<DomainMetadata>, Optional<CRCInfo>) - Constructor for class io.delta.kernel.internal.replay.ConflictChecker.TransactionRebaseState
- TransactionReport - Interface in io.delta.kernel.metrics
-
Defines the metadata and metrics for a transaction
MetricsReport
- TransactionReportImpl - Class in io.delta.kernel.internal.metrics
-
A basic POJO implementation of
TransactionReport
for creating them - TransactionReportImpl(String, String, String, Optional<Long>, TransactionMetrics, SnapshotReport, Optional<Exception>) - Constructor for class io.delta.kernel.internal.metrics.TransactionReportImpl
- TransactionStateRow - Class in io.delta.kernel.internal.data
- transformLogicalData(Engine, Row, CloseableIterator<FilteredColumnarBatch>, Map<String, Literal>) - Static method in interface io.delta.kernel.Transaction
-
Given the logical data that needs to be written to the table, convert it into the required physical data depending upon the table Delta protocol and features enabled on the table.
- transformPhysicalData(Engine, Row, Row, CloseableIterator<ColumnarBatch>) - Static method in interface io.delta.kernel.Scan
-
Transform the physical data read from the table data file into the logical data that expected out of the Delta table.
- tryExtractInCommitTimestamp(ColumnarBatch) - Static method in class io.delta.kernel.internal.util.InCommitTimestampUtils
-
Tries to extract the inCommitTimestamp from the commitInfo action in the given ColumnarBatch.
- Tuple2<K,
V> - Class in io.delta.kernel.internal.util -
Represents tuple of objects.
- Tuple2(K, V) - Constructor for class io.delta.kernel.internal.util.Tuple2
- type - Variable in class io.delta.kernel.internal.files.ParsedLogData
- TYPE_WIDENING_ENABLED - Static variable in class io.delta.kernel.internal.TableConfig
-
Whether widening the type of an existing column or field is allowed, either manually using ALTER TABLE CHANGE COLUMN or automatically if automatic schema evolution is enabled.
- TYPE_WIDENING_RW_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- TYPE_WIDENING_RW_PREVIEW_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- TypeWideningChecker - Class in io.delta.kernel.internal.types
-
Utility class for checking type widening compatibility according to the Delta protocol.
U
- UNIVERSAL_FORMAT_ENABLED_FORMATS - Static variable in class io.delta.kernel.internal.TableConfig
-
Table config that allows for translation of Delta metadata to other table formats metadata.
- UniversalFormats() - Constructor for class io.delta.kernel.internal.TableConfig.UniversalFormats
- unknownConfigurationException(String) - Static method in class io.delta.kernel.internal.DeltaErrors
- UnknownConfigurationException - Exception in io.delta.kernel.exceptions
-
Thrown when an unknown configuration key is specified.
- UnknownConfigurationException(String) - Constructor for exception io.delta.kernel.exceptions.UnknownConfigurationException
- unsupportedDataType(DataType) - Static method in class io.delta.kernel.internal.DeltaErrors
- unsupportedPartitionDataType(String, DataType) - Static method in class io.delta.kernel.internal.DeltaErrors
- unsupportedReaderFeatures(String, Set<String>) - Static method in class io.delta.kernel.internal.DeltaErrors
- unsupportedReaderProtocol(String, int) - Static method in class io.delta.kernel.internal.DeltaErrors
- unsupportedStatsDataType(DataType) - Static method in class io.delta.kernel.internal.DeltaErrors
- unsupportedTableFeature(String) - Static method in class io.delta.kernel.internal.DeltaErrors
- unsupportedWriterFeatures(String, Set<String>) - Static method in class io.delta.kernel.internal.DeltaErrors
- unsupportedWriterProtocol(String, int) - Static method in class io.delta.kernel.internal.DeltaErrors
- updateColumnMappingMetadataIfNeeded(Metadata, boolean) - Static method in class io.delta.kernel.internal.util.ColumnMapping
-
Updates the column mapping metadata if needed based on the column mapping mode and whether the icebergCompatV2 is enabled.
- updateForAddFile(long) - Method in class io.delta.kernel.internal.metrics.TransactionMetrics
-
Updates the metrics for a seen AddFile with size
addFileSize
. - updateForRemoveFile(long) - Method in class io.delta.kernel.internal.metrics.TransactionMetrics
-
Updates the metrics for a seen RemoveFile with size
removeFileSize
. - updateRowIdHighWatermarkIfNeeded(SnapshotImpl, Protocol, Optional<Long>, CloseableIterable<Row>, List<DomainMetadata>) - Static method in class io.delta.kernel.internal.rowtracking.RowTracking
-
Inserts or updates the
DomainMetadata
action reflecting the new row ID high watermark when this transaction adds rows and pushed it higher. - USER_DEFAULT - Static variable in class io.delta.kernel.types.DecimalType
- usesSidecars() - Method in enum io.delta.kernel.internal.checkpoints.CheckpointInstance.CheckpointFormat
- Utils - Class in io.delta.kernel.internal.util
-
Various utility methods to help the connectors work with data objects returned by Kernel
- Utils() - Constructor for class io.delta.kernel.internal.util.Utils
- UUID_DV_MARKER - Static variable in class io.delta.kernel.internal.actions.DeletionVectorDescriptor
V
- V2 - Enum constant in enum io.delta.kernel.internal.checkpoints.CheckpointInstance.CheckpointFormat
- V2_CHECKPOINT - Enum constant in enum io.delta.kernel.internal.files.ParsedLogData.ParsedLogType
- V2_CHECKPOINT_MANIFEST - Enum constant in enum io.delta.kernel.internal.replay.DeltaLogFile.LogType
- v2CheckpointSidecarFile(Path, String) - Static method in class io.delta.kernel.internal.util.FileNames
-
Returns the path for a V2 sidecar file with a given UUID.
- VACUUM_PROTOCOL_CHECK_RW_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- validate(Metadata) - Static method in class io.delta.kernel.internal.icebergcompat.IcebergUniversalFormatMetadataValidatorAndUpdater
-
Ensures the metadata is consistent with the enabled Universal output targets.
- validateAndSanitizePartitionValues(StructType, List<String>, Map<String, Literal>) - Static method in class io.delta.kernel.internal.util.PartitionUtils
-
Validate
partitionValues
contains values for every partition column in the table and the type of the value is correct. - validateAndUpdateIcebergCompatV2Metadata(boolean, Metadata, Protocol) - Static method in class io.delta.kernel.internal.icebergcompat.IcebergCompatV2MetadataValidatorAndUpdater
-
Validate and update the given Iceberg V2 metadata.
- validateAndUpdateIcebergWriterCompatV1Metadata(boolean, Metadata, Protocol) - Static method in class io.delta.kernel.internal.icebergcompat.IcebergWriterCompatV1MetadataValidatorAndUpdater
-
Validate and update the given Iceberg Writer Compat V1 metadata.
- validateDataFileStatus(DataFileStatus) - Static method in class io.delta.kernel.internal.icebergcompat.IcebergCompatV2MetadataValidatorAndUpdater
-
Validate the given
DataFileStatus
that is being added as aadd
action to Delta Log. - validateDeltaProperties(Map<String, String>) - Static method in class io.delta.kernel.internal.TableConfig
-
Validates that the given new properties that the txn is trying to update in table.
- validateDomainMetadatas(List<DomainMetadata>, Protocol) - Static method in class io.delta.kernel.internal.util.DomainMetadataUtils
-
Validates the list of domain metadata actions before committing them.
- validateIcebergWriterCompatV1Change(Map<String, String>, Map<String, String>, boolean) - Static method in class io.delta.kernel.internal.icebergcompat.IcebergWriterCompatV1MetadataValidatorAndUpdater
-
Validates that any change to property
TableConfig.ICEBERG_WRITER_COMPAT_V1_ENABLED
is valid. - validateKernelCanReadTheTable(Protocol, String) - Static method in class io.delta.kernel.internal.tablefeatures.TableFeatures
-
Utility method to check if the table with given protocol is readable by the Kernel.
- validateKernelCanWriteToTable(Protocol, Metadata, String) - Static method in class io.delta.kernel.internal.tablefeatures.TableFeatures
-
Utility method to check if the table with given protocol and metadata is writable by the Kernel.
- validateObject() - Method in class io.delta.kernel.internal.fs.Path
-
Validate the contents of a deserialized Path, so as to defend against malicious object streams.
- validatePartitionColumns(StructType, List<String>) - Static method in class io.delta.kernel.internal.util.SchemaUtils
-
Verify the partition columns exists in the table schema and a supported data type for a partition column.
- validatePredicateOnlyOnPartitionColumns(Predicate, Set<String>) - Static method in class io.delta.kernel.internal.util.PartitionUtils
-
Validate that the given predicate references only (and at least one) partition columns.
- validateRowTrackingConfigsNotMissing(Metadata, String) - Static method in class io.delta.kernel.internal.rowtracking.MaterializedRowTrackingColumn
-
Validates that materialized column names for ROW_ID and ROW_COMMIT_VERSION are not missing when row tracking is enabled.
- validateSchema(StructType, boolean) - Static method in class io.delta.kernel.internal.util.SchemaUtils
-
Validate the schema.
- validateUpdatedSchema(Metadata, Metadata, Set<String>, boolean) - Static method in class io.delta.kernel.internal.util.SchemaUtils
-
Performs the following validations on an updated table schema using the current schema as a base for validation.
- value - Variable in enum io.delta.kernel.internal.util.ColumnMapping.ColumnMappingMode
- value() - Method in class io.delta.kernel.internal.metrics.Counter
-
Reports the current count.
- valueOf(String) - Static method in enum io.delta.kernel.hook.PostCommitHook.PostCommitHookType
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum io.delta.kernel.internal.checkpoints.CheckpointInstance.CheckpointFormat
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum io.delta.kernel.internal.DeltaLogActionUtils.DeltaAction
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum io.delta.kernel.internal.files.ParsedLogData.ParsedLogCategory
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum io.delta.kernel.internal.files.ParsedLogData.ParsedLogType
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum io.delta.kernel.internal.replay.DeltaLogFile.LogType
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum io.delta.kernel.internal.util.ColumnMapping.ColumnMappingMode
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum io.delta.kernel.internal.util.FileNames.DeltaLogFileType
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum io.delta.kernel.Operation
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum io.delta.kernel.utils.CloseableIterator.BreakableFilterResult
-
Returns the enum constant of this type with the specified name.
- values() - Static method in enum io.delta.kernel.hook.PostCommitHook.PostCommitHookType
-
Returns an array containing the constants of this enum type, in the order they are declared.
- values() - Static method in enum io.delta.kernel.internal.checkpoints.CheckpointInstance.CheckpointFormat
-
Returns an array containing the constants of this enum type, in the order they are declared.
- values() - Static method in enum io.delta.kernel.internal.DeltaLogActionUtils.DeltaAction
-
Returns an array containing the constants of this enum type, in the order they are declared.
- values() - Static method in enum io.delta.kernel.internal.files.ParsedLogData.ParsedLogCategory
-
Returns an array containing the constants of this enum type, in the order they are declared.
- values() - Static method in enum io.delta.kernel.internal.files.ParsedLogData.ParsedLogType
-
Returns an array containing the constants of this enum type, in the order they are declared.
- values() - Static method in enum io.delta.kernel.internal.replay.DeltaLogFile.LogType
-
Returns an array containing the constants of this enum type, in the order they are declared.
- values() - Method in class io.delta.kernel.internal.util.CaseInsensitiveMap
- values() - Static method in enum io.delta.kernel.internal.util.ColumnMapping.ColumnMappingMode
-
Returns an array containing the constants of this enum type, in the order they are declared.
- values() - Static method in enum io.delta.kernel.internal.util.FileNames.DeltaLogFileType
-
Returns an array containing the constants of this enum type, in the order they are declared.
- values() - Static method in enum io.delta.kernel.Operation
-
Returns an array containing the constants of this enum type, in the order they are declared.
- values() - Static method in enum io.delta.kernel.utils.CloseableIterator.BreakableFilterResult
-
Returns an array containing the constants of this enum type, in the order they are declared.
- VARIANT - Static variable in class io.delta.kernel.types.VariantType
- VARIANT_RW_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- VARIANT_RW_PREVIEW_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- VARIANT_SHREDDING_ENABLED - Static variable in class io.delta.kernel.internal.TableConfig
-
Table property that enables modifying the table in accordance with the Delta-Variant Shredding Preview protocol.
- VARIANT_SHREDDING_PREVIEW_RW_FEATURE - Static variable in class io.delta.kernel.internal.tablefeatures.TableFeatures
- VariantType - Class in io.delta.kernel.types
-
A logical variant type.
- VectorUtils - Class in io.delta.kernel.internal.util
- verifyColumnMappingChange(Map<String, String>, Map<String, String>, boolean) - Static method in class io.delta.kernel.internal.util.ColumnMapping
- verifyDeltaVersionsContiguous(List<Long>, Path) - Static method in class io.delta.kernel.internal.snapshot.SnapshotManager
-
Verify that a list of delta versions is contiguous.
- version - Variable in class io.delta.kernel.internal.checkpoints.CheckpointInstance
- version - Variable in class io.delta.kernel.internal.checkpoints.CheckpointMetaData
- version - Variable in class io.delta.kernel.internal.files.ParsedLogData
- versionBeforeFirstAvailableCommit(String, long, long) - Static method in class io.delta.kernel.internal.DeltaErrors
- versionToLoadAfterLatestCommit(String, long, long) - Static method in class io.delta.kernel.internal.DeltaErrors
- VisibleForTesting - Annotation Type in io.delta.kernel.internal.annotation
-
Indicates that the visibility of a program element (such as a field, method, or class) is intentionally wider than necessary for testing purposes.
- voidTypeEncountered() - Static method in class io.delta.kernel.internal.DeltaErrors
W
- WINDOWS - Static variable in class io.delta.kernel.internal.fs.Path
-
Whether the current host is a Windows machine.
- withClusteringColumns(Engine, List<Column>) - Method in class io.delta.kernel.internal.TransactionBuilderImpl
-
There are three possible cases when handling clustering columns via `withClusteringColumns`: Clustering columns are not set (i.e., `withClusteringColumns` is not called): No changes are made related to clustering.
- withClusteringColumns(Engine, List<Column>) - Method in interface io.delta.kernel.TransactionBuilder
-
Set the list of clustering columns when create a new clustered table.
- withConfigurationKeysUnset(Set<String>) - Method in class io.delta.kernel.internal.actions.Metadata
-
Returns a new metadata object that has a new configuration which does not contain any of the keys provided in
keysToUnset
. - withDeletedColumnAt(int) - Method in interface io.delta.kernel.data.ColumnarBatch
-
Return a copy of this
ColumnarBatch
with the column at givenordinal
removed. - withDomainMetadataSupported() - Method in class io.delta.kernel.internal.TransactionBuilderImpl
- withDomainMetadataSupported() - Method in interface io.delta.kernel.TransactionBuilder
-
Enables support for Domain Metadata on this table if it is not supported already.
- withExistingTableFileSizeHistogram(Optional<FileSizeHistogram>) - Static method in class io.delta.kernel.internal.metrics.TransactionMetrics
- withFeature(TableFeature) - Method in class io.delta.kernel.internal.actions.Protocol
-
Get a new Protocol object that has `feature` supported.
- withFeatures(Iterable<TableFeature>) - Method in class io.delta.kernel.internal.actions.Protocol
-
Create a new
Protocol
object with the givenTableFeature
supported. - withFilter(Predicate) - Method in class io.delta.kernel.internal.ScanBuilderImpl
- withFilter(Predicate) - Method in interface io.delta.kernel.ScanBuilder
-
Apply the given filter expression to prune any files that do not possibly contain the data that satisfies the given filter.
- withLogCompactionInverval(int) - Method in class io.delta.kernel.internal.TransactionBuilderImpl
- withLogCompactionInverval(int) - Method in interface io.delta.kernel.TransactionBuilder
-
Set the number of commits between log compactions.
- withMaxRetries(int) - Method in class io.delta.kernel.internal.TransactionBuilderImpl
- withMaxRetries(int) - Method in interface io.delta.kernel.TransactionBuilder
-
Set the maximum number of times to retry a transaction if a concurrent write is detected.
- withMergedConfiguration(Map<String, String>) - Method in class io.delta.kernel.internal.actions.Metadata
-
Returns a new metadata object that has a new configuration which is the combination of its current configuration and
configuration
. - withNewBaseRowId(long) - Method in class io.delta.kernel.internal.actions.AddFile
-
Returns a new
AddFile
with the provided baseRowId. - withNewColumn(int, StructField, ColumnVector) - Method in interface io.delta.kernel.data.ColumnarBatch
-
Return a copy of the
ColumnarBatch
with given new column vector inserted at the givencolumnVector
at givenordinal
. - withNewDefaultRowCommitVersion(long) - Method in class io.delta.kernel.internal.actions.AddFile
-
Returns a new
AddFile
with the provided defaultRowCommitVersion. - withNewMetadata(FieldMetadata) - Method in class io.delta.kernel.types.StructField
- withNewSchema(StructType) - Method in interface io.delta.kernel.data.ColumnarBatch
-
Generate a copy of this
ColumnarBatch
with the givennewSchema
. - withNewSchema(StructType) - Method in class io.delta.kernel.internal.actions.Metadata
- withPartitionColumns(Engine, List<String>) - Method in class io.delta.kernel.internal.TransactionBuilderImpl
- withPartitionColumns(Engine, List<String>) - Method in interface io.delta.kernel.TransactionBuilder
-
Set the list of partitions columns when create a new partitioned table.
- withPartitionColumns(ExpressionHandler, ColumnarBatch, Map<String, String>, StructType) - Static method in class io.delta.kernel.internal.util.PartitionUtils
- withReadSchema(StructType) - Method in class io.delta.kernel.internal.ScanBuilderImpl
- withReadSchema(StructType) - Method in interface io.delta.kernel.ScanBuilder
-
Apply the given readSchema.
- withReplacedConfiguration(Map<String, String>) - Method in class io.delta.kernel.internal.actions.Metadata
-
Returns a new Metadata object with the configuration provided with newConfiguration (any prior configuration is replaced).
- withSchema(Engine, StructType) - Method in class io.delta.kernel.internal.TransactionBuilderImpl
- withSchema(Engine, StructType) - Method in interface io.delta.kernel.TransactionBuilder
-
Set the schema of the table.
- withSidecarFileSchema(StructType) - Static method in class io.delta.kernel.internal.replay.LogReplay
- withTableProperties(Engine, Map<String, String>) - Method in class io.delta.kernel.internal.TransactionBuilderImpl
- withTableProperties(Engine, Map<String, String>) - Method in interface io.delta.kernel.TransactionBuilder
-
Set the table properties for the table.
- withTablePropertiesRemoved(Set<String>) - Method in class io.delta.kernel.internal.TransactionBuilderImpl
- withTablePropertiesRemoved(Set<String>) - Method in interface io.delta.kernel.TransactionBuilder
-
Unset the provided table properties on the table.
- withTransactionId(Engine, String, long) - Method in class io.delta.kernel.internal.TransactionBuilderImpl
- withTransactionId(Engine, String, long) - Method in interface io.delta.kernel.TransactionBuilder
-
Set the transaction identifier for idempotent writes.
- withUpdatedMetadata(Metadata) - Method in class io.delta.kernel.internal.icebergcompat.IcebergCompatMetadataValidatorAndUpdater.IcebergCompatInputContext
- wrapEngineException(Supplier<T>, String, Object...) - Static method in class io.delta.kernel.internal.DeltaErrors
- wrapEngineExceptionThrowsIO(DeltaErrors.SupplierWithIOException<T>, String, Object...) - Static method in class io.delta.kernel.internal.DeltaErrors
- write(JsonGenerator, T) - Method in interface io.delta.kernel.internal.util.JsonUtils.JsonValueWriter
- WRITE - Enum constant in enum io.delta.kernel.Operation
-
Recorded during batch inserts.
- writeCheckSum(Engine, CRCInfo) - Method in class io.delta.kernel.internal.checksum.ChecksumWriter
-
Writes a checksum file
- writeJsonFileAtomically(String, CloseableIterator<Row>, boolean) - Method in interface io.delta.kernel.engine.JsonHandler
-
Serialize each
Row
in the iterator as JSON and write as a separate line in destination file. - writeLastCheckpointFile(Engine, CheckpointMetaData) - Method in class io.delta.kernel.internal.checkpoints.Checkpointer
-
Write the given data to last checkpoint metadata file.
- writeLogCompactionFile(Engine) - Method in class io.delta.kernel.internal.compaction.LogCompactionWriter
- writeParquetFileAtomically(String, CloseableIterator<FilteredColumnarBatch>) - Method in interface io.delta.kernel.engine.ParquetHandler
-
Write the given data as a Parquet file.
- writeParquetFiles(String, CloseableIterator<FilteredColumnarBatch>, List<Column>) - Method in interface io.delta.kernel.engine.ParquetHandler
-
Write the given data batches to a Parquet files.
- WriterFeature(String, int) - Constructor for class io.delta.kernel.internal.tablefeatures.TableFeature.WriterFeature
_
- _1 - Variable in class io.delta.kernel.internal.util.Tuple2
- _2 - Variable in class io.delta.kernel.internal.util.Tuple2
All Classes and Interfaces|All Packages|Constant Field Values|Serialized Form