org.locationtech.geomesa.bigtable.spark

Class BigtableInputFormatBase

java.lang.Object

org.apache.hadoop.mapreduce.InputFormat<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>

org.locationtech.geomesa.bigtable.spark.BigtableInputFormatBase

public abstract class BigtableInputFormatBase
extends org.apache.hadoop.mapreduce.InputFormat<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>

Adapted from HBase MultiTableInputFormatBase

Constructor Summary

Constructors

Constructor and Description
BigtableInputFormatBase()

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>	createRecordReader(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context) Builds a TableRecordReader.
protected List<org.apache.hadoop.hbase.client.Scan>	getScans() Allows subclasses to get the list of Scan objects.
List<org.apache.hadoop.mapreduce.InputSplit>	getSplits(org.apache.hadoop.mapreduce.JobContext context) Calculates the splits that will serve as input for the map tasks.
static String	scanToString(com.google.cloud.bigtable.hbase.BigtableExtendedScan scan)
void	setName(org.apache.hadoop.hbase.TableName name)
protected void	setScans(List<org.apache.hadoop.hbase.client.Scan> scans) Allows subclasses to set the list of Scan objects.
protected void	setTableRecordReader(BigtableTableRecordReader tableRecordReader) Allows subclasses to set the TableRecordReader.
static com.google.cloud.bigtable.hbase.BigtableExtendedScan	stringToScan(String encoded)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BigtableInputFormatBase

public BigtableInputFormatBase()

Method Detail

setName

public void setName(org.apache.hadoop.hbase.TableName name)

createRecordReader

public org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result> createRecordReader(org.apache.hadoop.mapreduce.InputSplit split,
                                                                                                                                                            org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                                                                                                     throws IOException,
                                                                                                                                                            InterruptedException

Builds a TableRecordReader. If no TableRecordReader was provided, uses the default.

Specified by:

createRecordReader in class org.apache.hadoop.mapreduce.InputFormat<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>

Parameters:

split - The split to work with.

context - The current context.

Returns:

The newly created record reader.

Throws:

IOException - When creating the reader fails.

InterruptedException - when record reader initialization fails

See Also:

InputFormat.createRecordReader( org.apache.hadoop.mapreduce.InputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext)

getSplits

public List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext context)
                                                       throws IOException

Calculates the splits that will serve as input for the map tasks. The number of splits matches the number of regions in a table.

Specified by:

getSplits in class org.apache.hadoop.mapreduce.InputFormat<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.client.Result>

Parameters:

context - The current job context.

Returns:

The list of input splits.

Throws:

IOException - When creating the list of splits fails.

See Also:

InputFormat.getSplits(org.apache.hadoop.mapreduce.JobContext)

getScans

protected List<org.apache.hadoop.hbase.client.Scan> getScans()

Allows subclasses to get the list of Scan objects.

setScans

protected void setScans(List<org.apache.hadoop.hbase.client.Scan> scans)

Allows subclasses to set the list of Scan objects.

Parameters:

scans - The list of Scan used to define the input

setTableRecordReader

protected void setTableRecordReader(BigtableTableRecordReader tableRecordReader)

Allows subclasses to set the TableRecordReader.

Parameters:

tableRecordReader - A different TableRecordReader implementation.

scanToString

public static String scanToString(com.google.cloud.bigtable.hbase.BigtableExtendedScan scan)
                           throws IOException

Throws:

IOException

stringToScan

public static com.google.cloud.bigtable.hbase.BigtableExtendedScan stringToScan(String encoded)
                                                                         throws IOException

Throws:

IOException