org.apache.hadoop.mapred

Class TaskLog

java.lang.Object

org.apache.hadoop.mapred.TaskLog

@InterfaceAudience.Private
public class TaskLog
extends Object

A simple logger to handle the task-specific user logs. This class uses the system property hadoop.log.dir.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TaskLog.LogName The filter for userlogs.
static class	TaskLog.Reader

Constructor Summary

Constructors

Constructor and Description
TaskLog()

Method Summary

Methods

Modifier and Type	Method and Description
static String	addCommand(List<String> cmd, boolean isExecutable) Add quotes to each of the command strings and return as a single string
static List<String>	captureOutAndError(List<String> setup, List<String> cmd, File stdoutFilename, File stderrFilename, long tailLength, boolean useSetsid) Wrap a command in a shell to capture stdout and stderr to files.
static ScheduledExecutorService	createLogSyncer()
static File	getJobDir(JobID jobid) Get the user log directory for the job jobid.
static String	getMRv2LogDir()
static File	getTaskLogFile(TaskAttemptID taskid, boolean isCleanup, TaskLog.LogName filter)
static long	getTaskLogLength(JobConf conf) Get the desired maximum length of task's logs.
static void	syncLogs()
static void	syncLogs(String logLocation, TaskAttemptID taskid, boolean isCleanup)
static void	syncLogsShutdown(ScheduledExecutorService scheduler)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TaskLog

public TaskLog()

Method Detail

getMRv2LogDir

public static String getMRv2LogDir()

getTaskLogFile

public static File getTaskLogFile(TaskAttemptID taskid,
                  boolean isCleanup,
                  TaskLog.LogName filter)

syncLogs

public static void syncLogs(String logLocation,
            TaskAttemptID taskid,
            boolean isCleanup)
                     throws IOException

Throws:

IOException

syncLogsShutdown

public static void syncLogsShutdown(ScheduledExecutorService scheduler)

syncLogs

public static void syncLogs()

createLogSyncer

public static ScheduledExecutorService createLogSyncer()

getTaskLogLength

public static long getTaskLogLength(JobConf conf)

Get the desired maximum length of task's logs.

Parameters:

conf - the job to look in

Returns:

the number of bytes to cap the log files at

captureOutAndError

public static List<String> captureOutAndError(List<String> setup,
                              List<String> cmd,
                              File stdoutFilename,
                              File stderrFilename,
                              long tailLength,
                              boolean useSetsid)
                                       throws IOException

Wrap a command in a shell to capture stdout and stderr to files. Setup commands such as setting memory limit can be passed which will be executed before exec. If the tailLength is 0, the entire output will be saved.

Parameters:

setup - The setup commands for the execed process.

cmd - The command and the arguments that should be run

stdoutFilename - The filename that stdout should be saved to

stderrFilename - The filename that stderr should be saved to

tailLength - The length of the tail to be saved.

useSetsid - Should setsid be used in the command or not.

Returns:

the modified command that should be run

Throws:

IOException

addCommand

public static String addCommand(List<String> cmd,
                boolean isExecutable)
                         throws IOException

Add quotes to each of the command strings and return as a single string

Parameters:

cmd - The command to be quoted

isExecutable - makes shell path if the first argument is executable

Returns:

returns The quoted string.

Throws:

IOException

getJobDir

public static File getJobDir(JobID jobid)

Get the user log directory for the job jobid.

Parameters:

jobid -

Returns:

user log directory for the job