org.apache.hadoop.mapreduce.task.reduce
Class ShuffleSchedulerImpl<K,V>
java.lang.Object
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl<K,V>
- All Implemented Interfaces:
- ShuffleScheduler<K,V>
@InterfaceAudience.Private
@InterfaceStability.Unstable
public class ShuffleSchedulerImpl<K,V>
- extends Object
- implements ShuffleScheduler<K,V>
Method Summary |
void |
addKnownMapOutput(String hostName,
String hostUrl,
TaskAttemptID mapId)
|
void |
close()
|
void |
copyFailed(TaskAttemptID mapId,
MapHost host,
boolean readError,
boolean connectExcpt)
|
void |
copySucceeded(TaskAttemptID mapId,
MapHost host,
long bytes,
long millis,
MapOutput<K,V> output)
|
void |
freeHost(MapHost host)
|
MapHost |
getHost()
|
List<TaskAttemptID> |
getMapsForHost(MapHost host)
|
void |
obsoleteMapOutput(TaskAttemptID mapId)
|
void |
putBackKnownMapOutput(MapHost host,
TaskAttemptID mapId)
|
void |
reportLocalError(IOException ioe)
|
void |
resetKnownMaps()
|
void |
resolve(TaskCompletionEvent event)
Interpret a TaskCompletionEvent from the event stream. |
void |
tipFailed(TaskID taskId)
|
boolean |
waitUntilDone(int millis)
Wait until the shuffle finishes or until the timeout. |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
ShuffleSchedulerImpl
public ShuffleSchedulerImpl(JobConf job,
TaskStatus status,
TaskAttemptID reduceId,
ExceptionReporter reporter,
org.apache.hadoop.util.Progress progress,
Counters.Counter shuffledMapsCounter,
Counters.Counter reduceShuffleBytes,
Counters.Counter failedShuffleCounter)
resolve
public void resolve(TaskCompletionEvent event)
- Description copied from interface:
ShuffleScheduler
- Interpret a
TaskCompletionEvent
from the event stream.
- Specified by:
resolve
in interface ShuffleScheduler<K,V>
- Parameters:
event
- Intermediate output metadata
copySucceeded
public void copySucceeded(TaskAttemptID mapId,
MapHost host,
long bytes,
long millis,
MapOutput<K,V> output)
throws IOException
- Throws:
IOException
copyFailed
public void copyFailed(TaskAttemptID mapId,
MapHost host,
boolean readError,
boolean connectExcpt)
reportLocalError
public void reportLocalError(IOException ioe)
tipFailed
public void tipFailed(TaskID taskId)
addKnownMapOutput
public void addKnownMapOutput(String hostName,
String hostUrl,
TaskAttemptID mapId)
obsoleteMapOutput
public void obsoleteMapOutput(TaskAttemptID mapId)
putBackKnownMapOutput
public void putBackKnownMapOutput(MapHost host,
TaskAttemptID mapId)
getHost
public MapHost getHost()
throws InterruptedException
- Throws:
InterruptedException
getMapsForHost
public List<TaskAttemptID> getMapsForHost(MapHost host)
freeHost
public void freeHost(MapHost host)
resetKnownMaps
public void resetKnownMaps()
waitUntilDone
public boolean waitUntilDone(int millis)
throws InterruptedException
- Wait until the shuffle finishes or until the timeout.
- Specified by:
waitUntilDone
in interface ShuffleScheduler<K,V>
- Parameters:
millis
- maximum wait time
- Returns:
- true if the shuffle is done
- Throws:
InterruptedException
close
public void close()
throws InterruptedException
- Specified by:
close
in interface ShuffleScheduler<K,V>
- Throws:
InterruptedException
Copyright © 2013 Apache Software Foundation. All Rights Reserved.