Package org.apache.accumulo.hadoopImpl.mapreduce

Class FileOutputFormatBuilderImpl<T>

java.lang.Object

org.apache.accumulo.hadoopImpl.mapreduce.FileOutputFormatBuilderImpl<T>

All Implemented Interfaces:

FileOutputFormatBuilder, FileOutputFormatBuilder.OutputOptions<T>, FileOutputFormatBuilder.PathParams<T>

public class FileOutputFormatBuilderImpl<T>
extends Object
implements FileOutputFormatBuilder, FileOutputFormatBuilder.PathParams<T>, FileOutputFormatBuilder.OutputOptions<T>

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.accumulo.hadoop.mapreduce.FileOutputFormatBuilder

FileOutputFormatBuilder.OutputOptions<T>, FileOutputFormatBuilder.PathParams<T>

Constructor Summary

Constructors

Constructor	Description
FileOutputFormatBuilderImpl(Class<?> callingClass)

Method Summary

Modifier and Type	Method	Description
FileOutputFormatBuilder.OutputOptions<T>	compression(String compressionType)	Sets the compression type to use for data blocks, overriding the default.
FileOutputFormatBuilder.OutputOptions<T>	dataBlockSize(long dataBlockSize)	Sets the size for data blocks within each file. Data blocks are a span of key/value pairs stored in the file that are compressed and indexed as a group.
FileOutputFormatBuilder.OutputOptions<T>	fileBlockSize(long fileBlockSize)	Sets the size for file blocks in the file system; file blocks are managed, and replicated, by the underlying file system.
FileOutputFormatBuilder.OutputOptions<T>	indexBlockSize(long indexBlockSize)	Sets the size for index blocks within each file; smaller blocks means a deeper index hierarchy within the file, while larger blocks mean a more shallow index hierarchy within the file.
FileOutputFormatBuilder.OutputOptions<T>	outputPath(org.apache.hadoop.fs.Path path)	Set the Path of the output directory for the map-reduce job.
FileOutputFormatBuilder.OutputOptions<T>	replication(int replication)	Sets the file system replication factor for the resulting file, overriding the file system default.
FileOutputFormatBuilder.OutputOptions<T>	sampler(SamplerConfiguration samplerConfig)	Specify a sampler to be used when writing out data.
void	store(T j)	Finish configuring, verify and serialize options into the Job or JobConf
FileOutputFormatBuilder.OutputOptions<T>	summarizers(SummarizerConfiguration... summarizerConfigs)	Specifies a list of summarizer configurations to create summary data in the output file.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

FileOutputFormatBuilderImpl

public FileOutputFormatBuilderImpl(Class<?> callingClass)

Method Details

outputPath

public FileOutputFormatBuilder.OutputOptions<T> outputPath(org.apache.hadoop.fs.Path path)

Description copied from interface: FileOutputFormatBuilder.PathParams

Set the Path of the output directory for the map-reduce job.

Specified by:

outputPath in interface FileOutputFormatBuilder.PathParams<T>

compression

public FileOutputFormatBuilder.OutputOptions<T> compression(String compressionType)

Description copied from interface: FileOutputFormatBuilder.OutputOptions

Sets the compression type to use for data blocks, overriding the default. Specifying a compression may require additional libraries to be available to your Job.

Specified by:

compression in interface FileOutputFormatBuilder.OutputOptions<T>

Parameters:

compressionType - one of "none", "gz", "bzip2", "lzo", "lz4", "snappy", or "zstd"

dataBlockSize

public FileOutputFormatBuilder.OutputOptions<T> dataBlockSize(long dataBlockSize)

Description copied from interface: FileOutputFormatBuilder.OutputOptions

Sets the size for data blocks within each file.
Data blocks are a span of key/value pairs stored in the file that are compressed and indexed as a group.

Making this value smaller may increase seek performance, but at the cost of increasing the size of the indexes (which can also affect seek performance).

Specified by:

dataBlockSize in interface FileOutputFormatBuilder.OutputOptions<T>

Parameters:

dataBlockSize - the block size, in bytes

fileBlockSize

public FileOutputFormatBuilder.OutputOptions<T> fileBlockSize(long fileBlockSize)

Description copied from interface: FileOutputFormatBuilder.OutputOptions

Sets the size for file blocks in the file system; file blocks are managed, and replicated, by the underlying file system.

Specified by:

fileBlockSize in interface FileOutputFormatBuilder.OutputOptions<T>

Parameters:

fileBlockSize - the block size, in bytes

indexBlockSize

public FileOutputFormatBuilder.OutputOptions<T> indexBlockSize(long indexBlockSize)

Description copied from interface: FileOutputFormatBuilder.OutputOptions

Sets the size for index blocks within each file; smaller blocks means a deeper index hierarchy within the file, while larger blocks mean a more shallow index hierarchy within the file. This can affect the performance of queries.

Specified by:

indexBlockSize in interface FileOutputFormatBuilder.OutputOptions<T>

Parameters:

indexBlockSize - the block size, in bytes

replication

public FileOutputFormatBuilder.OutputOptions<T> replication(int replication)

Description copied from interface: FileOutputFormatBuilder.OutputOptions

Sets the file system replication factor for the resulting file, overriding the file system default.

Specified by:

replication in interface FileOutputFormatBuilder.OutputOptions<T>

Parameters:

replication - the number of replicas for produced files

sampler

public FileOutputFormatBuilder.OutputOptions<T> sampler(SamplerConfiguration samplerConfig)

Description copied from interface: FileOutputFormatBuilder.OutputOptions

Specify a sampler to be used when writing out data. This will result in the output file having sample data.

Specified by:

sampler in interface FileOutputFormatBuilder.OutputOptions<T>

Parameters:

samplerConfig - The configuration for creating sample data in the output file.

summarizers

public FileOutputFormatBuilder.OutputOptions<T> summarizers(SummarizerConfiguration... summarizerConfigs)

Description copied from interface: FileOutputFormatBuilder.OutputOptions

Specifies a list of summarizer configurations to create summary data in the output file. Each Key Value written will be passed to the configured Summarizer's.

Specified by:

summarizers in interface FileOutputFormatBuilder.OutputOptions<T>

Parameters:

summarizerConfigs - summarizer configurations

store

public void store(T j)

Description copied from interface: FileOutputFormatBuilder.OutputOptions

Finish configuring, verify and serialize options into the Job or JobConf

Specified by:

store in interface FileOutputFormatBuilder.OutputOptions<T>