org.apache.hadoop.mapred

Class QueueManager

java.lang.Object

org.apache.hadoop.mapred.QueueManager

@InterfaceAudience.Private
public class QueueManager
extends Object

Class that exposes information about queues maintained by the Hadoop Map/Reduce framework.

The Map/Reduce framework can be configured with one or more queues, depending on the scheduler it is configured with. While some schedulers work only with one queue, some schedulers support multiple queues. Some schedulers also support the notion of queues within queues - a feature called hierarchical queues.

Queue names are unique, and used as a key to lookup queues. Hierarchical queues are named by a 'fully qualified name' such as q1:q2:q3, where q2 is a child queue of q1 and q3 is a child queue of q2.

Leaf level queues are queues that contain no queues within them. Jobs can be submitted only to leaf level queues.

Queues can be configured with various properties. Some of these properties are common to all schedulers, and those are handled by this class. Schedulers might also associate several custom properties with queues. These properties are parsed and maintained per queue by the framework. If schedulers need more complicated structure to maintain configuration per queue, they are free to not use the facilities provided by the framework, but define their own mechanisms. In such cases, it is likely that the name of the queue will be used to relate the common properties of a queue with scheduler specific properties.

Information related to a queue, such as its name, properties, scheduling information and children are exposed by this class via a serializable class called JobQueueInfo.

Queues are configured in the configuration file mapred-queues.xml. To support backwards compatibility, queues can also be configured in mapred-site.xml. However, when configured in the latter, there is no support for hierarchical queues.

Field Summary

Fields

Modifier and Type	Field and Description
static String	QUEUE_CONF_FILE_NAME

Constructor Summary

Constructors

Constructor and Description
QueueManager(org.apache.hadoop.conf.Configuration clusterConf) Construct a new QueueManager using configuration specified in the passed in Configuration object.

Method Summary

Methods

Modifier and Type	Method and Description
Set<String>	getLeafQueueNames() Return the set of leaf level queues configured in the system to which jobs are submitted.
Object	getSchedulerInfo(String queueName) Return the scheduler information configured for this queue.
boolean	hasAccess(String queueName, QueueACL qACL, org.apache.hadoop.security.UserGroupInformation ugi) Return true if the given user is part of the ACL for the given QueueACL name for the given queue.
void	setSchedulerInfo(String queueName, Object queueInfo) Set a generic Object that represents scheduling information relevant to a queue.
static String	toFullPropertyName(String queue, String property)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

QUEUE_CONF_FILE_NAME

public static final String QUEUE_CONF_FILE_NAME

See Also:

Constant Field Values

Constructor Detail

QueueManager

public QueueManager(org.apache.hadoop.conf.Configuration clusterConf)

Construct a new QueueManager using configuration specified in the passed in Configuration object.

This instance supports queue configuration specified in mapred-site.xml, but without support for hierarchical queues. If no queue configuration is found in mapred-site.xml, it will then look for site configuration in mapred-queues.xml supporting hierarchical queues.

Parameters:

clusterConf - mapreduce cluster configuration

Method Detail

getLeafQueueNames

public Set<String> getLeafQueueNames()

Return the set of leaf level queues configured in the system to which jobs are submitted.

The number of queues configured should be dependent on the Scheduler configured. Note that some schedulers work with only one queue, whereas others can support multiple queues.

Returns:

Set of queue names.

hasAccess

public boolean hasAccess(String queueName,
                QueueACL qACL,
                org.apache.hadoop.security.UserGroupInformation ugi)

Return true if the given user is part of the ACL for the given QueueACL name for the given queue.

An operation is allowed if all users are provided access for this operation, or if either the user or any of the groups specified is provided access.

Parameters:

queueName - Queue on which the operation needs to be performed.

qACL - The queue ACL name to be checked

ugi - The user and groups who wish to perform the operation.

Returns:

true if the operation is allowed, false otherwise.

setSchedulerInfo

public void setSchedulerInfo(String queueName,
                    Object queueInfo)

Set a generic Object that represents scheduling information relevant to a queue.

A string representation of this Object will be used by the framework to display in user facing applications like the JobTracker web UI and the hadoop CLI.

Parameters:

queueName - queue for which the scheduling information is to be set.

queueInfo - scheduling information for this queue.

getSchedulerInfo

public Object getSchedulerInfo(String queueName)

Return the scheduler information configured for this queue.

Parameters:

queueName - queue for which the scheduling information is required.

Returns:

The scheduling information for this queue.

toFullPropertyName

public static final String toFullPropertyName(String queue,
                        String property)