org.apache.hadoop.mapreduce.lib.join

Class MultiFilterRecordReader<K extends org.apache.hadoop.io.WritableComparable<?>,V extends org.apache.hadoop.io.Writable>

java.lang.Object

org.apache.hadoop.mapreduce.RecordReader<K,V>

org.apache.hadoop.mapreduce.lib.join.ComposableRecordReader<K,X>

org.apache.hadoop.mapreduce.lib.join.CompositeRecordReader<K,V,V>

org.apache.hadoop.mapreduce.lib.join.MultiFilterRecordReader<K,V>

All Implemented Interfaces:

Closeable, AutoCloseable, Comparable<ComposableRecordReader<K,?>>, org.apache.hadoop.conf.Configurable

Direct Known Subclasses:

OverrideRecordReader

@InterfaceAudience.Public
@InterfaceStability.Stable
public abstract class MultiFilterRecordReader<K extends org.apache.hadoop.io.WritableComparable<?>,V extends org.apache.hadoop.io.Writable>
extends CompositeRecordReader<K,V,V>

Base class for Composite join returning values derived from multiple sources, but generally not tuples.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	MultiFilterRecordReader.MultiFilterDelegationIterator Proxy the JoinCollector, but include callback to emit.

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.lib.join.CompositeRecordReader

CompositeRecordReader.JoinCollector

Field Summary

Fields inherited from class org.apache.hadoop.mapreduce.lib.join.CompositeRecordReader

conf, jc, key, keyclass, kids, value

Constructor Summary

Constructors

Constructor and Description
MultiFilterRecordReader(int id, org.apache.hadoop.conf.Configuration conf, int capacity, Class<? extends org.apache.hadoop.io.WritableComparator> cmpcl)

Method Summary

Methods

Modifier and Type	Method and Description
protected boolean	combine(Object[] srcs, TupleWritable dst) Default implementation offers emit(org.apache.hadoop.mapreduce.lib.join.TupleWritable) every Tuple from the collector (the outer join of child RRs).
protected abstract V	emit(TupleWritable dst) For each tuple emitted, return a value (typically one of the values in the tuple).
protected ResetableIterator<V>	getDelegate() Return an iterator returning a single value from the tuple.
void	initialize(InputSplit split, TaskAttemptContext context) Called once at initialization.
boolean	nextKeyValue() Read the next key, value pair.

Methods inherited from class org.apache.hadoop.mapreduce.lib.join.CompositeRecordReader

accept, add, close, compareTo, createKey, createTupleWritable, fillJoinCollector, getComparator, getConf, getCurrentKey, getCurrentValue, getProgress, getRecordReaderQueue, hasNext, id, key, key, setConf, skip

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MultiFilterRecordReader

public MultiFilterRecordReader(int id,
                       org.apache.hadoop.conf.Configuration conf,
                       int capacity,
                       Class<? extends org.apache.hadoop.io.WritableComparator> cmpcl)
                        throws IOException

Throws:

IOException

Method Detail

emit

protected abstract V emit(TupleWritable dst)
                                                         throws IOException

For each tuple emitted, return a value (typically one of the values in the tuple). Modifying the Writables in the tuple is permitted and unlikely to affect join behavior in most cases, but it is not recommended. It's safer to clone first.

Throws:

IOException

combine

protected boolean combine(Object[] srcs,
              TupleWritable dst)

Default implementation offers emit(org.apache.hadoop.mapreduce.lib.join.TupleWritable) every Tuple from the collector (the outer join of child RRs).

Specified by:

combine in class CompositeRecordReader<K extends org.apache.hadoop.io.WritableComparable<?>,V extends org.apache.hadoop.io.Writable,V extends org.apache.hadoop.io.Writable>

nextKeyValue

public boolean nextKeyValue()
                     throws IOException,
                            InterruptedException

Read the next key, value pair.

Specified by:

nextKeyValue in class RecordReader<K extends org.apache.hadoop.io.WritableComparable<?>,V extends org.apache.hadoop.io.Writable>

Returns:

true if a key/value pair was read

Throws:

IOException

InterruptedException

initialize

public void initialize(InputSplit split,
              TaskAttemptContext context)
                throws IOException,
                       InterruptedException

Description copied from class: RecordReader

Called once at initialization.

Overrides:

initialize in class CompositeRecordReader<K extends org.apache.hadoop.io.WritableComparable<?>,V extends org.apache.hadoop.io.Writable,V extends org.apache.hadoop.io.Writable>

Parameters:

split - the split that defines the range of records to read

context - the information about the task

Throws:

IOException

InterruptedException

getDelegate

protected ResetableIterator<V> getDelegate()

Return an iterator returning a single value from the tuple.

Specified by:

getDelegate in class CompositeRecordReader<K extends org.apache.hadoop.io.WritableComparable<?>,V extends org.apache.hadoop.io.Writable,V extends org.apache.hadoop.io.Writable>

See Also:

MultiFilterRecordReader.MultiFilterDelegationIterator