org.apache.hadoop.mapred.lib.db

Class DBInputFormat.DBRecordReader

java.lang.Object

org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,T>

org.apache.hadoop.mapreduce.lib.db.DBRecordReader<T>

org.apache.hadoop.mapred.lib.db.DBInputFormat.DBRecordReader

All Implemented Interfaces:

Closeable, AutoCloseable, RecordReader<org.apache.hadoop.io.LongWritable,T>

Enclosing class:

DBInputFormat<T extends DBWritable>

protected class DBInputFormat.DBRecordReader
extends DBRecordReader<T>
implements RecordReader<org.apache.hadoop.io.LongWritable,T>

A RecordReader that reads records from a SQL table. Emits LongWritables containing the record number as key and DBWritables as value.

Field Summary

Fields inherited from class org.apache.hadoop.mapreduce.lib.db.DBRecordReader

statement

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	DBInputFormat.DBRecordReader(DBInputFormat.DBInputSplit split, Class<T> inputClass, JobConf job) The constructor is kept to be compatible with M/R 1.x
protected	DBInputFormat.DBRecordReader(DBInputFormat.DBInputSplit split, Class<T> inputClass, JobConf job, Connection conn, DBConfiguration dbConfig, String cond, String[] fields, String table)

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.hadoop.io.LongWritable	createKey() Create an object of the appropriate type to be used as a key.
T	createValue() Create an object of the appropriate type to be used as a value.
long	getPos() Returns the current position in the input.
boolean	next(org.apache.hadoop.io.LongWritable key, T value) Reads the next key/value pair from the input for processing.

Methods inherited from class org.apache.hadoop.mapreduce.lib.db.DBRecordReader

close, executeQuery, getConditions, getConnection, getCurrentKey, getCurrentValue, getDBConf, getFieldNames, getProgress, getSelectQuery, getSplit, getStatement, getTableName, initialize, nextKeyValue, setStatement

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.mapred.RecordReader

close, getProgress

Constructor Detail

DBInputFormat.DBRecordReader

protected DBInputFormat.DBRecordReader(DBInputFormat.DBInputSplit split,
                            Class<T> inputClass,
                            JobConf job)
                                throws SQLException

The constructor is kept to be compatible with M/R 1.x

Parameters:

split - The InputSplit to read data for

Throws:

SQLException

DBInputFormat.DBRecordReader

protected DBInputFormat.DBRecordReader(DBInputFormat.DBInputSplit split,
                            Class<T> inputClass,
                            JobConf job,
                            Connection conn,
                            DBConfiguration dbConfig,
                            String cond,
                            String[] fields,
                            String table)
                                throws SQLException

Parameters:

split - The InputSplit to read data for

Throws:

SQLException

Method Detail

createKey

public org.apache.hadoop.io.LongWritable createKey()

Create an object of the appropriate type to be used as a key.

Specified by:

createKey in interface RecordReader<org.apache.hadoop.io.LongWritable,T extends DBWritable>

Returns:

a new key object.

createValue

public T createValue()

Create an object of the appropriate type to be used as a value.

Specified by:

createValue in interface RecordReader<org.apache.hadoop.io.LongWritable,T extends DBWritable>

Overrides:

createValue in class DBRecordReader<T extends DBWritable>

Returns:

a new value object.

getPos

public long getPos()
            throws IOException

Description copied from interface: RecordReader

Returns the current position in the input.

Specified by:

getPos in interface RecordReader<org.apache.hadoop.io.LongWritable,T extends DBWritable>

Overrides:

getPos in class DBRecordReader<T extends DBWritable>

Returns:

the current position in the input.

Throws:

IOException

next

public boolean next(org.apache.hadoop.io.LongWritable key,
           T value)
             throws IOException

Reads the next key/value pair from the input for processing.

Specified by:

next in interface RecordReader<org.apache.hadoop.io.LongWritable,T extends DBWritable>

Overrides:

next in class DBRecordReader<T extends DBWritable>

Parameters:

key - the key to read data into

value - the value to read data into

Returns:

true iff a key/value was read, false if at EOF

Throws:

IOException