com.google.cloud.hadoop.io.bigquery.mapred

Class BigQueryMapredInputFormat

java.lang.Object

com.google.cloud.hadoop.io.bigquery.mapred.BigQueryMapredInputFormat

All Implemented Interfaces:

org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.LongWritable,com.google.gson.JsonObject>

public class BigQueryMapredInputFormat
extends java.lang.Object
implements org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.LongWritable,com.google.gson.JsonObject>

InputFormat that uses the old mapred API so that we can do streaming input with our BigQuery connector.

Constructor Summary

Constructors

Constructor and Description
BigQueryMapredInputFormat()

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,com.google.gson.JsonObject>	getRecordReader(org.apache.hadoop.mapred.InputSplit inputSplit, org.apache.hadoop.mapred.JobConf conf, org.apache.hadoop.mapred.Reporter reporter) Get a RecordReader by calling through to AbstractBigQueryInputFormat.createRecordReader(org.apache.hadoop.mapreduce.InputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext).
org.apache.hadoop.mapred.InputSplit[]	getSplits(org.apache.hadoop.mapred.JobConf job, int numSplits) Calls through to AbstractBigQueryInputFormat.getSplits(org.apache.hadoop.mapreduce.JobContext).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BigQueryMapredInputFormat

public BigQueryMapredInputFormat()

Method Detail

getSplits

public org.apache.hadoop.mapred.InputSplit[] getSplits(org.apache.hadoop.mapred.JobConf job,
                                                       int numSplits)
                                                throws java.io.IOException

Calls through to AbstractBigQueryInputFormat.getSplits(org.apache.hadoop.mapreduce.JobContext).

Specified by:

getSplits in interface org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.LongWritable,com.google.gson.JsonObject>

Parameters:

job - The config passed to us from the streaming package.

numSplits - We ignore this parameter.

Throws:

java.io.IOException

getRecordReader

public org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,com.google.gson.JsonObject> getRecordReader(org.apache.hadoop.mapred.InputSplit inputSplit,
                                                                                                                           org.apache.hadoop.mapred.JobConf conf,
                                                                                                                           org.apache.hadoop.mapred.Reporter reporter)
                                                                                                                    throws java.io.IOException

Get a RecordReader by calling through to AbstractBigQueryInputFormat.createRecordReader(org.apache.hadoop.mapreduce.InputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext).

Specified by:

getRecordReader in interface org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.LongWritable,com.google.gson.JsonObject>

Throws:

java.io.IOException