org.apache.hadoop.io.compress

Interface SplittableCompressionCodec

All Superinterfaces:

CompressionCodec

All Known Implementing Classes:

BZip2Codec

@InterfaceAudience.Public
@InterfaceStability.Evolving
public interface SplittableCompressionCodec
extends CompressionCodec

This interface is meant to be implemented by those compression codecs which are capable to compress / de-compress a stream starting at any arbitrary position. Especially the process of de-compressing a stream starting at some arbitrary position is challenging. Most of the codecs are only able to successfully de-compress a stream, if they start from the very beginning till the end. One of the reasons is the stored state at the beginning of the stream which is crucial for de-compression. Yet there are few codecs which do not save the whole state at the beginning of the stream and hence can be used to de-compress stream starting at any arbitrary points. This interface is meant to be used by such codecs. Such codecs are highly valuable, especially in the context of Hadoop, because an input compressed file can be split and hence can be worked on by multiple machines in parallel.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	SplittableCompressionCodec.READ_MODE During decompression, data can be read off from the decompressor in two modes, namely continuous and blocked.

Method Summary

Methods

Modifier and Type	Method and Description
SplitCompressionInputStream	createInputStream(InputStream seekableIn, Decompressor decompressor, long start, long end, SplittableCompressionCodec.READ_MODE readMode) Create a stream as dictated by the readMode.

Methods inherited from interface org.apache.hadoop.io.compress.CompressionCodec

createCompressor, createDecompressor, createInputStream, createInputStream, createOutputStream, createOutputStream, getCompressorType, getDecompressorType, getDefaultExtension

Method Detail

createInputStream

SplitCompressionInputStream createInputStream(InputStream seekableIn,
                                            Decompressor decompressor,
                                            long start,
                                            long end,
                                            SplittableCompressionCodec.READ_MODE readMode)
                                              throws IOException

Create a stream as dictated by the readMode. This method is used when the codecs wants the ability to work with the underlying stream positions.

Parameters:

seekableIn - The seekable input stream (seeks in compressed data)

start - The start offset into the compressed stream. May be changed by the underlying codec.

end - The end offset into the compressed stream. May be changed by the underlying codec.

readMode - Controls whether stream position is reported continuously from the compressed stream only only at block boundaries.

Returns:

a stream to read uncompressed bytes from

Throws:

IOException