org.apache.hadoop.mapred

Class SequenceFileInputFilter.MD5Filter

java.lang.Object

org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFilter.FilterBase

org.apache.hadoop.mapred.SequenceFileInputFilter.FilterBase

org.apache.hadoop.mapred.SequenceFileInputFilter.MD5Filter

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, SequenceFileInputFilter.Filter

Enclosing class:

SequenceFileInputFilter<K,V>

public static class SequenceFileInputFilter.MD5Filter
extends SequenceFileInputFilter.FilterBase

This class returns a set of records by examing the MD5 digest of its key against a filtering frequency f. The filtering criteria is MD5(key) % f == 0.

Field Summary

Fields

Modifier and Type	Field and Description
static int	MD5_LEN

Constructor Summary

Constructors

Constructor and Description
MD5Filter()

Method Summary

Modifier and Type	Method and Description
boolean	accept(Object key) Filtering method If MD5(key) % frequency==0, return true; otherwise return false
void	setConf(org.apache.hadoop.conf.Configuration conf) configure the filter according to configuration
static void	setFrequency(org.apache.hadoop.conf.Configuration conf, int frequency) set the filtering frequency in configuration

Methods inherited from class org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFilter.FilterBase

getConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf

Field Detail

MD5_LEN

public static final int MD5_LEN

See Also:

Constant Field Values

Constructor Detail

MD5Filter

public MD5Filter()

Method Detail

setFrequency

public static void setFrequency(org.apache.hadoop.conf.Configuration conf,
                                int frequency)

set the filtering frequency in configuration

Parameters:

conf - configuration

frequency - filtering frequency

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

configure the filter according to configuration

Parameters:

conf - configuration

accept

public boolean accept(Object key)

Filtering method If MD5(key) % frequency==0, return true; otherwise return false

Parameters:

key - record key

Returns:

true if a record is accepted; return false otherwise

See Also:

SequenceFileInputFilter.Filter.accept(Object)