org.apache.hadoop.mapreduce.task.reduce

Interface MergeManager<K,V>

All Known Implementing Classes:

MergeManagerImpl

@InterfaceAudience.Private
@InterfaceStability.Unstable
public interface MergeManager<K,V>

An interface for a reduce side merge that works with the default Shuffle implementation.

Method Summary

Methods

Modifier and Type	Method and Description
RawKeyValueIterator	close() Called at the end of shuffle.
MapOutput<K,V>	reserve(TaskAttemptID mapId, long requestedSize, int fetcher) To reserve resources for data to be shuffled.
void	waitForResource() To wait until merge has some freed resources available so that it can accept shuffled data.

Method Detail

waitForResource

void waitForResource()
                     throws InterruptedException

To wait until merge has some freed resources available so that it can accept shuffled data. This will be called before a network connection is established to get the map output.

Throws:

InterruptedException

reserve

MapOutput<K,V> reserve(TaskAttemptID mapId,
                     long requestedSize,
                     int fetcher)
                       throws IOException

To reserve resources for data to be shuffled. This will be called after a network connection is made to shuffle the data.

Parameters:

mapId - mapper from which data will be shuffled.

requestedSize - size in bytes of data that will be shuffled.

fetcher - id of the map output fetcher that will shuffle the data.

Returns:

a MapOutput object that can be used by shuffle to shuffle data. If required resources cannot be reserved immediately, a null can be returned.

Throws:

IOException

close

RawKeyValueIterator close()
                          throws Throwable

Called at the end of shuffle.

Returns:

a key value iterator object.

Throws:

Throwable