org.apache.hadoop.io.file.tfile
Class TFile

java.lang.Object
  extended by org.apache.hadoop.io.file.tfile.TFile

@InterfaceAudience.Public
@InterfaceStability.Evolving
public class TFile
extends Object

A TFile is a container of key-value pairs. Both keys and values are type-less bytes. Keys are restricted to 64KB, value length is not restricted (practically limited to the available disk storage). TFile further provides the following features:

The memory footprint of a TFile includes the following:

The behavior of TFile can be customized by the following variables through Configuration:

Suggestions on performance optimization.

Some design rationale behind TFile can be found at Hadoop-3315.


Nested Class Summary
static class TFile.Reader
          TFile Reader.
static class TFile.Writer
          TFile Writer.
 
Field Summary
static String COMPARATOR_JCLASS
          comparator prefix: java class
static String COMPARATOR_MEMCMP
          comparator: memcmp
static String COMPRESSION_GZ
          compression: gzip
static String COMPRESSION_LZO
          compression: lzo
static String COMPRESSION_NONE
          compression: none
 
Method Summary
static String[] getSupportedCompressionAlgorithms()
          Get names of supported compression algorithms.
static void main(String[] args)
          Dumping the TFile information.
static Comparator<RawComparable> makeComparator(String name)
          Make a raw comparator from a string name.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

COMPRESSION_GZ

public static final String COMPRESSION_GZ
compression: gzip

See Also:
Constant Field Values

COMPRESSION_LZO

public static final String COMPRESSION_LZO
compression: lzo

See Also:
Constant Field Values

COMPRESSION_NONE

public static final String COMPRESSION_NONE
compression: none

See Also:
Constant Field Values

COMPARATOR_MEMCMP

public static final String COMPARATOR_MEMCMP
comparator: memcmp

See Also:
Constant Field Values

COMPARATOR_JCLASS

public static final String COMPARATOR_JCLASS
comparator prefix: java class

See Also:
Constant Field Values
Method Detail

makeComparator

public static Comparator<RawComparable> makeComparator(String name)
Make a raw comparator from a string name.

Parameters:
name - Comparator name
Returns:
A RawComparable comparator.

getSupportedCompressionAlgorithms

public static String[] getSupportedCompressionAlgorithms()
Get names of supported compression algorithms. The names are acceptable by TFile.Writer.

Returns:
Array of strings, each represents a supported compression algorithm. Currently, the following compression algorithms are supported.
  • "none" - No compression.
  • "lzo" - LZO compression.
  • "gz" - GZIP compression.

main

public static void main(String[] args)
Dumping the TFile information.

Parameters:
args - A list of TFile paths.


Copyright © 2013 Apache Software Foundation. All Rights Reserved.