Package io.cdap.cdap.api.dataset.lib.partitioned

Implementation of classes which use partitioned datasets (such as partition consumers) in CDAP.

See: Description

Interface Summary

Interface	Description
ConsumablePartition	Represents a Partition available for consuming.
PartitionAcceptor	Defines whether to accept PartitionDetails, while iterating over a collection of them.
PartitionBatchInput.BatchPartitionCommitter	Used to persist the state of the partition consumer.
PartitionConsumer	Incrementally consumes new/unprocessed Partitions of a PartitionedFileSet.
StatePersistor	Defines how some state will be persisted/managed.

Class Summary

Class	Description
AbstractPartitionConsumer	Abstract implementation of PartitionConsumer, which manages state persistence and serialization/deserialization before delegating to the abstract methods.
ComparableCodec	Responsible for serialization and deserialization of Comparable.
ConcurrentPartitionConsumer	A PartitionConsumer that supports multiple instances consuming the same set of partitions by using a working set of partitions, and keeping track of their progress state during processing of those partitions.
ConsumerConfiguration	Configuration parameters to be used by a PartitionConsumer.
ConsumerConfiguration.Builder	A Builder to construct ConsumerConfiguration instances.
ConsumerWorkingSet	Keeps track of a list of partitions that are either available for consuming or are currently being consumed.
DefaultConsumablePartition	Default implementation of ConsumablePartition.
DelegatingStatePersistor	An implementation of StatePersistor that uses a DatasetContext to read and persist state.
KVTableStatePersistor	An implementation of DatasetStatePersistor that uses a row of a KeyValueTable for persistence.
PartitionAcceptor.Limit	An implementation of PartitionAcceptor which limits the number of accepted partitions to a given value.
PartitionBatchInput	A utility class for batch processing (i.e.
PartitionConsumerResult	Returns access to a list of partitions for consuming, as well as a list of PartitionKeys corresponding to partitions that have previously failed processing the configured number of tries.
PartitionKeyCodec	Codec used to serialize and deserialize PartitionKeys.
TransactionalPartitionConsumer	An implementation of PartitionConsumer that uses a Transactional to execute its methods in a new transaction.

Enum Summary

Enum	Description
PartitionAcceptor.Return	Return value, determining what to do with a Partition.
ProcessState	Represents the process/consumed state of the partition.

Package io.cdap.cdap.api.dataset.lib.partitioned Description

Implementation of classes which use partitioned datasets (such as partition consumers) in CDAP.