org.apache.hadoop.mapreduce.lib.partition

Class KeyFieldBasedComparator<K,V>

java.lang.Object

org.apache.hadoop.io.WritableComparator

org.apache.hadoop.mapreduce.lib.partition.KeyFieldBasedComparator<K,V>

All Implemented Interfaces:

Comparator, org.apache.hadoop.conf.Configurable, org.apache.hadoop.io.RawComparator

Direct Known Subclasses:

KeyFieldBasedComparator

@InterfaceAudience.Public
@InterfaceStability.Stable
public class KeyFieldBasedComparator<K,V>
extends org.apache.hadoop.io.WritableComparator
implements org.apache.hadoop.conf.Configurable

This comparator implementation provides a subset of the features provided by the Unix/GNU Sort. In particular, the supported features are: -n, (Sort numerically) -r, (Reverse the result of comparison) -k pos1[,pos2], where pos is of the form f[.c][opts], where f is the number of the field to use, and c is the number of the first character from the beginning of the field. Fields and character posns are numbered starting with 1; a character position of zero in pos2 indicates the field's last character. If '.c' is omitted from pos1, it defaults to 1 (the beginning of the field); if omitted from pos2, it defaults to 0 (the end of the field). opts are ordering options (any of 'nr' as described above). We assume that the fields in the key are separated by MRJobConfig.MAP_OUTPUT_KEY_FIELD_SEPERATOR.

Field Summary

Fields

Modifier and Type	Field and Description
static String	COMPARATOR_OPTIONS

Constructor Summary

Constructors

Constructor and Description
KeyFieldBasedComparator()

Method Summary

Methods

Modifier and Type	Method and Description
int	compare(byte[] b1, int s1, int l1, byte[] b2, int s2, int l2)
org.apache.hadoop.conf.Configuration	getConf()
static String	getKeyFieldComparatorOption(JobContext job) Get the KeyFieldBasedComparator options
void	setConf(org.apache.hadoop.conf.Configuration conf)
static void	setKeyFieldComparatorOptions(Job job, String keySpec) Set the KeyFieldBasedComparator options used to compare keys.

Methods inherited from class org.apache.hadoop.io.WritableComparator

compare, compare, compareBytes, define, get, getKeyClass, hashBytes, hashBytes, newKey, readDouble, readFloat, readInt, readLong, readUnsignedShort, readVInt, readVLong

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Comparator

equals

Field Detail

COMPARATOR_OPTIONS

public static String COMPARATOR_OPTIONS

Constructor Detail

KeyFieldBasedComparator

public KeyFieldBasedComparator()

Method Detail

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

compare

public int compare(byte[] b1,
          int s1,
          int l1,
          byte[] b2,
          int s2,
          int l2)

Specified by:

compare in interface org.apache.hadoop.io.RawComparator

Overrides:

compare in class org.apache.hadoop.io.WritableComparator

setKeyFieldComparatorOptions

public static void setKeyFieldComparatorOptions(Job job,
                                String keySpec)

Set the KeyFieldBasedComparator options used to compare keys.

Parameters:

keySpec - the key specification of the form -k pos1[,pos2], where, pos is of the form f[.c][opts], where f is the number of the key field to use, and c is the number of the first character from the beginning of the field. Fields and character posns are numbered starting with 1; a character position of zero in pos2 indicates the field's last character. If '.c' is omitted from pos1, it defaults to 1 (the beginning of the field); if omitted from pos2, it defaults to 0 (the end of the field). opts are ordering options. The supported options are: -n, (Sort numerically) -r, (Reverse the result of comparison)

getKeyFieldComparatorOption

public static String getKeyFieldComparatorOption(JobContext job)

Get the KeyFieldBasedComparator options