org.apache.hadoop.mapreduce

Class JobSubmissionFiles

java.lang.Object

org.apache.hadoop.mapreduce.JobSubmissionFiles

@InterfaceAudience.Private
public class JobSubmissionFiles
extends Object

A utility to manage job submission files.

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.hadoop.fs.permission.FsPermission	JOB_DIR_PERMISSION
static org.apache.hadoop.fs.permission.FsPermission	JOB_FILE_PERMISSION

Constructor Summary

Constructors

Constructor and Description
JobSubmissionFiles()

Method Summary

Methods

Modifier and Type	Method and Description
static org.apache.hadoop.fs.Path	getJobConfPath(org.apache.hadoop.fs.Path jobSubmitDir) Get the job conf path.
static org.apache.hadoop.fs.Path	getJobDistCacheArchives(org.apache.hadoop.fs.Path jobSubmitDir) Get the job distributed cache archives path.
static org.apache.hadoop.fs.Path	getJobDistCacheFiles(org.apache.hadoop.fs.Path jobSubmitDir) Get the job distributed cache files path.
static org.apache.hadoop.fs.Path	getJobDistCacheLibjars(org.apache.hadoop.fs.Path jobSubmitDir) Get the job distributed cache libjars path.
static org.apache.hadoop.fs.Path	getJobJar(org.apache.hadoop.fs.Path jobSubmitDir) Get the job jar path.
static org.apache.hadoop.fs.Path	getJobLog4jFile(org.apache.hadoop.fs.Path jobSubmitDir) Get the job distributed cache path for log4j properties.
static org.apache.hadoop.fs.Path	getJobSplitFile(org.apache.hadoop.fs.Path jobSubmissionDir)
static org.apache.hadoop.fs.Path	getJobSplitMetaFile(org.apache.hadoop.fs.Path jobSubmissionDir)
static org.apache.hadoop.fs.Path	getStagingDir(Cluster cluster, org.apache.hadoop.conf.Configuration conf) Initializes the staging directory and returns the path.
static org.apache.hadoop.fs.Path	getStagingDir(Cluster cluster, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.security.UserGroupInformation realUser) Initializes the staging directory and returns the path.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

JOB_DIR_PERMISSION

public static final org.apache.hadoop.fs.permission.FsPermission JOB_DIR_PERMISSION

JOB_FILE_PERMISSION

public static final org.apache.hadoop.fs.permission.FsPermission JOB_FILE_PERMISSION

Constructor Detail

JobSubmissionFiles

public JobSubmissionFiles()

Method Detail

getJobSplitFile

public static org.apache.hadoop.fs.Path getJobSplitFile(org.apache.hadoop.fs.Path jobSubmissionDir)

getJobSplitMetaFile

public static org.apache.hadoop.fs.Path getJobSplitMetaFile(org.apache.hadoop.fs.Path jobSubmissionDir)

getJobConfPath

public static org.apache.hadoop.fs.Path getJobConfPath(org.apache.hadoop.fs.Path jobSubmitDir)

Get the job conf path.

getJobJar

public static org.apache.hadoop.fs.Path getJobJar(org.apache.hadoop.fs.Path jobSubmitDir)

Get the job jar path.

getJobDistCacheFiles

public static org.apache.hadoop.fs.Path getJobDistCacheFiles(org.apache.hadoop.fs.Path jobSubmitDir)

Get the job distributed cache files path.

Parameters:

jobSubmitDir -

getJobLog4jFile

public static org.apache.hadoop.fs.Path getJobLog4jFile(org.apache.hadoop.fs.Path jobSubmitDir)

Get the job distributed cache path for log4j properties.

Parameters:

jobSubmitDir -

getJobDistCacheArchives

public static org.apache.hadoop.fs.Path getJobDistCacheArchives(org.apache.hadoop.fs.Path jobSubmitDir)

Get the job distributed cache archives path.

Parameters:

jobSubmitDir -

getJobDistCacheLibjars

public static org.apache.hadoop.fs.Path getJobDistCacheLibjars(org.apache.hadoop.fs.Path jobSubmitDir)

Get the job distributed cache libjars path.

Parameters:

jobSubmitDir -

getStagingDir

public static org.apache.hadoop.fs.Path getStagingDir(Cluster cluster,
                                      org.apache.hadoop.conf.Configuration conf)
                                               throws IOException,
                                                      InterruptedException

Initializes the staging directory and returns the path. It also keeps track of all necessary ownership and permissions

Parameters:

cluster -

conf -

Throws:

IOException

InterruptedException

getStagingDir

public static org.apache.hadoop.fs.Path getStagingDir(Cluster cluster,
                                      org.apache.hadoop.conf.Configuration conf,
                                      org.apache.hadoop.security.UserGroupInformation realUser)
                                               throws IOException,
                                                      InterruptedException

Initializes the staging directory and returns the path. It also keeps track of all necessary ownership and permissions. It is kept for unit testing.

Parameters:

cluster - Information about the map/reduce cluster

conf - Configuration object

realUser - UserGroupInformation of login user

Returns:

staging dir path object

Throws:

IOException - when ownership of staging area directory does not match the login user or current user.

InterruptedException - when getting the staging area directory path