org.elasticsearch.hadoop.hive

Class EsHiveInputFormat

java.lang.Object

org.apache.hadoop.mapreduce.InputFormat<K,V>

org.elasticsearch.hadoop.mr.EsInputFormat<org.apache.hadoop.io.Text,java.util.Map<org.apache.hadoop.io.Writable,org.apache.hadoop.io.Writable>>

org.elasticsearch.hadoop.hive.EsHiveInputFormat

All Implemented Interfaces:

org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.Text,java.util.Map<org.apache.hadoop.io.Writable,org.apache.hadoop.io.Writable>>

public class EsHiveInputFormat
extends EsInputFormat<org.apache.hadoop.io.Text,java.util.Map<org.apache.hadoop.io.Writable,org.apache.hadoop.io.Writable>>

Hive specific InputFormat. Since Hive code base makes a lot of assumptions about the tables being actual files in HDFS (using instanceof checks without proper else) this class tries to 'fix' this by adding a dummy FileInputFormat to ESInputFormat.

Nested Class Summary

Nested classes/interfaces inherited from class org.elasticsearch.hadoop.mr.EsInputFormat

EsInputFormat.ShardInputSplit, EsInputFormat.ShardRecordReader<K,V>, EsInputFormat.WritableShardRecordReader

Constructor Summary

Constructors

Constructor and Description
EsHiveInputFormat()

Method Summary

Methods

Modifier and Type	Method and Description
EsInputFormat.WritableShardRecordReader	getRecordReader(org.apache.hadoop.mapred.InputSplit split, org.apache.hadoop.mapred.JobConf job, org.apache.hadoop.mapred.Reporter reporter)
org.apache.hadoop.mapred.FileSplit[]	getSplits(org.apache.hadoop.mapred.JobConf job, int numSplits)

Methods inherited from class org.elasticsearch.hadoop.mr.EsInputFormat

createRecordReader, getSplits

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

EsHiveInputFormat

public EsHiveInputFormat()

Method Detail

getSplits

public org.apache.hadoop.mapred.FileSplit[] getSplits(org.apache.hadoop.mapred.JobConf job,
                                             int numSplits)
                                               throws java.io.IOException

Specified by:

getSplits in interface org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.Text,java.util.Map<org.apache.hadoop.io.Writable,org.apache.hadoop.io.Writable>>

Overrides:

getSplits in class EsInputFormat<org.apache.hadoop.io.Text,java.util.Map<org.apache.hadoop.io.Writable,org.apache.hadoop.io.Writable>>

Throws:

java.io.IOException

getRecordReader

public EsInputFormat.WritableShardRecordReader getRecordReader(org.apache.hadoop.mapred.InputSplit split,
                                                      org.apache.hadoop.mapred.JobConf job,
                                                      org.apache.hadoop.mapred.Reporter reporter)

Specified by:

getRecordReader in interface org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.Text,java.util.Map<org.apache.hadoop.io.Writable,org.apache.hadoop.io.Writable>>

Overrides:

getRecordReader in class EsInputFormat<org.apache.hadoop.io.Text,java.util.Map<org.apache.hadoop.io.Writable,org.apache.hadoop.io.Writable>>