Incremental partitioned parquet writer Provides incremental writing for partitioned puckets Any error will return with the minimum checkpoint of the current set of writers
Incremental functional writer class for parquet
Partitioned parquet writer Allows a pucket to be written to with a partitioning scheme
Partitioned writer mixin provides partitioning to a writer type
Partitioned writer mixin provides partitioning to a writer type
This trait provides common partitioning functions to implementing classes. It maintains a cache of open writer instances with a configurable size.
type of data to be written
type of error
implementing type to be used when creating new instances
Trait for functional wrappers of parquet writers
Trait for functional wrappers of parquet writers
type of data to be written
type of error returned if write fails
Factory object for com.intenthq.pucket.writer.IncrementalPartitionedWriter
Factory object for com.intenthq.pucket.writer.IncrementalWriter
Factory object for com.intenthq.pucket.writer.PartitionedWriter
Incremental functional writer class for parquet
Overview
Allows large files to be written to parquet in smaller incements. Closes the underlying writer and creates a new file at a defined writer count.
Also keeps a checkpoint of when the last file was sucessfully flushed to the filesystem so in the event of an error processing can be restarted from that point.
type of data to be written