|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object org.apache.accumulo.core.client.mapreduce.lib.util.ConfiguratorBase org.apache.accumulo.core.client.mapreduce.lib.util.InputConfigurator
public class InputConfigurator
Nested Class Summary | |
---|---|
static class |
InputConfigurator.Features
Configuration keys for various features. |
static class |
InputConfigurator.ScanOpts
Configuration keys for Scanner . |
Nested classes/interfaces inherited from class org.apache.accumulo.core.client.mapreduce.lib.util.ConfiguratorBase |
---|
ConfiguratorBase.ConnectorInfo, ConfiguratorBase.GeneralOpts, ConfiguratorBase.InstanceOpts |
Constructor Summary | |
---|---|
InputConfigurator()
|
Method Summary | |
---|---|
static void |
addIterator(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf,
IteratorSetting cfg)
Encode an iterator on the input for this job. |
static void |
fetchColumns(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf,
Collection<Pair<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>> columnFamilyColumnQualifierPairs)
Restricts the columns that will be mapped over for this job. |
static Boolean |
getAutoAdjustRanges(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf)
Determines whether a configuration has auto-adjust ranges enabled. |
static Set<Pair<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>> |
getFetchedColumns(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf)
Gets the columns to be mapped over from this job. |
static String |
getInputTableName(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf)
Gets the table name from the configuration. |
static List<IteratorSetting> |
getIterators(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf)
Gets a list of the iterator settings (for iterators to apply to a scanner) from this configuration. |
static List<Range> |
getRanges(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf)
Gets the ranges to scan over from a job. |
static Authorizations |
getScanAuthorizations(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf)
Gets the authorizations to set for the scans from the configuration. |
static TabletLocator |
getTabletLocator(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf)
Initializes an Accumulo TabletLocator based on the configuration. |
static Boolean |
isIsolated(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf)
Determines whether a configuration has isolation enabled. |
static Boolean |
isOfflineScan(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf)
Determines whether a configuration has the offline table scan feature enabled. |
static void |
setAutoAdjustRanges(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf,
boolean enableFeature)
Controls the automatic adjustment of ranges for this job. |
static void |
setInputTableName(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf,
String tableName)
Sets the name of the input table, over which this job will scan. |
static void |
setLocalIterators(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf,
boolean enableFeature)
Controls the use of the ClientSideIteratorScanner in this job. |
static void |
setOfflineTableScan(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf,
boolean enableFeature)
Enable reading offline tables. |
static void |
setRanges(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf,
Collection<Range> ranges)
Sets the input ranges to scan for this job. |
static void |
setScanAuthorizations(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf,
Authorizations auths)
Sets the Authorizations used to scan. |
static void |
setScanIsolation(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf,
boolean enableFeature)
Controls the use of the IsolatedScanner in this job. |
static Boolean |
usesLocalIterators(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf)
Determines whether a configuration uses local iterators. |
static void |
validateOptions(Class<?> implementingClass,
org.apache.hadoop.conf.Configuration conf)
Check whether a configuration is fully configured to be used with an Accumulo InputFormat . |
Methods inherited from class org.apache.accumulo.core.client.mapreduce.lib.util.ConfiguratorBase |
---|
enumToConfKey, getInstance, getLogLevel, getPrincipal, getToken, getTokenClass, isConnectorInfoSet, setConnectorInfo, setLogLevel, setMockInstance, setZooKeeperInstance |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public InputConfigurator()
Method Detail |
---|
public static void setInputTableName(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf, String tableName)
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configuretableName
- the table to use when the tablename is null in the write callpublic static String getInputTableName(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf)
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configure
setInputTableName(Class, Configuration, String)
public static void setScanAuthorizations(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf, Authorizations auths)
Authorizations
used to scan. Must be a subset of the user's authorization. Defaults to the empty set.
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configureauths
- the user's authorizationspublic static Authorizations getScanAuthorizations(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf)
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configure
setScanAuthorizations(Class, Configuration, Authorizations)
public static void setRanges(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf, Collection<Range> ranges)
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configureranges
- the ranges that will be mapped overpublic static List<Range> getRanges(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf) throws IOException
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configure
IOException
- if the ranges have been encoded improperlysetRanges(Class, Configuration, Collection)
public static void fetchColumns(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf, Collection<Pair<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>> columnFamilyColumnQualifierPairs)
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configurecolumnFamilyColumnQualifierPairs
- a pair of Text
objects corresponding to column family and column qualifier. If the column qualifier is null, the entire column family is
selected. An empty set is the default and is equivalent to scanning the all columns.public static Set<Pair<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>> getFetchedColumns(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf)
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configure
fetchColumns(Class, Configuration, Collection)
public static void addIterator(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf, IteratorSetting cfg)
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configurecfg
- the configuration of the iteratorpublic static List<IteratorSetting> getIterators(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf)
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configure
addIterator(Class, Configuration, IteratorSetting)
public static void setAutoAdjustRanges(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf, boolean enableFeature)
By default, this feature is enabled.
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configureenableFeature
- the feature is enabled if true, disabled otherwisesetRanges(Class, Configuration, Collection)
public static Boolean getAutoAdjustRanges(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf)
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configure
setAutoAdjustRanges(Class, Configuration, boolean)
public static void setScanIsolation(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf, boolean enableFeature)
IsolatedScanner
in this job.
By default, this feature is disabled.
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configureenableFeature
- the feature is enabled if true, disabled otherwisepublic static Boolean isIsolated(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf)
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configure
setScanIsolation(Class, Configuration, boolean)
public static void setLocalIterators(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf, boolean enableFeature)
ClientSideIteratorScanner
in this job. Enabling this feature will cause the iterator stack to be constructed within the Map
task, rather than within the Accumulo TServer. To use this feature, all classes needed for those iterators must be available on the classpath for the task.
By default, this feature is disabled.
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configureenableFeature
- the feature is enabled if true, disabled otherwisepublic static Boolean usesLocalIterators(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf)
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configure
setLocalIterators(Class, Configuration, boolean)
public static void setOfflineTableScan(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf, boolean enableFeature)
Enable reading offline tables. By default, this feature is disabled and only online tables are scanned. This will make the map reduce job directly read the table's files. If the table is not offline, then the job will fail. If the table comes online during the map reduce job, it is likely that the job will fail.
To use this option, the map reduce user will need access to read the Accumulo directory in HDFS.
Reading the offline table will create the scan time iterator stack in the map process. So any iterators that are configured for the table will need to be on the mapper's classpath. The accumulo-site.xml may need to be on the mapper's classpath if HDFS or the Accumulo directory in HDFS are non-standard.
One way to use this feature is to clone a table, take the clone offline, and use the clone as the input table for a map reduce job. If you plan to map reduce over the data many times, it may be better to the compact the table, clone it, take it offline, and use the clone for all map reduce jobs. The reason to do this is that compaction will reduce each tablet in the table to one file, and it is faster to read from one file.
There are two possible advantages to reading a tables file directly out of HDFS. First, you may see better read performance. Second, it will support speculative execution better. When reading an online table speculative execution can put more load on an already slow tablet server.
By default, this feature is disabled.
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configureenableFeature
- the feature is enabled if true, disabled otherwisepublic static Boolean isOfflineScan(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf)
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configure
setOfflineTableScan(Class, Configuration, boolean)
public static TabletLocator getTabletLocator(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf) throws TableNotFoundException
TabletLocator
based on the configuration.
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configure
TableNotFoundException
- if the table name set on the configuration doesn't existpublic static void validateOptions(Class<?> implementingClass, org.apache.hadoop.conf.Configuration conf) throws IOException
InputFormat
.
implementingClass
- the class whose name will be used as a prefix for the property configuration keyconf
- the Hadoop configuration object to configure
IOException
- if the context is improperly configured
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |