org.elasticsearch.hadoop.mr

Class EsInputFormat<K,V>

java.lang.Object

org.apache.hadoop.mapreduce.InputFormat<K,V>

org.elasticsearch.hadoop.mr.EsInputFormat<K,V>

All Implemented Interfaces:

InputFormat<K,V>

Direct Known Subclasses:

EsHiveInputFormat, EsPigInputFormat

public class EsInputFormat<K,V>
extends InputFormat<K,V>
implements InputFormat<K,V>

ElasticSearch InputFormat for streaming data (typically based on a query) from ElasticSearch. Returns the document ID as key and its content as value.

This class implements both the "old" (org.apache.hadoop.mapred) and the "new" (org.apache.hadoop.mapreduce) API.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	EsInputFormat.AbstractWritableShardRecordReader<V>
protected static class	EsInputFormat.JsonWritableShardRecordReader
protected static class	EsInputFormat.ShardInputSplit
protected static class	EsInputFormat.ShardRecordReader<K,V>
protected static class	EsInputFormat.WritableShardRecordReader

Constructor Summary

Constructors

Constructor and Description
EsInputFormat()

Method Summary

Methods

Modifier and Type	Method and Description
EsInputFormat.ShardRecordReader<K,V>	createRecordReader(InputSplit split, TaskAttemptContext context)
EsInputFormat.ShardRecordReader<K,V>	getRecordReader(InputSplit split, JobConf job, Reporter reporter)
InputSplit[]	getSplits(JobConf job, int numSplits)
List<InputSplit>	getSplits(JobContext context)
protected boolean	isOutputAsJson(Configuration cfg)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

EsInputFormat

public EsInputFormat()

Method Detail

getSplits

public List<InputSplit> getSplits(JobContext context)
                           throws IOException

Specified by:

getSplits in class InputFormat<K,V>

Throws:

IOException

createRecordReader

public EsInputFormat.ShardRecordReader<K,V> createRecordReader(InputSplit split,
                                                      TaskAttemptContext context)

Specified by:

createRecordReader in class InputFormat<K,V>

getSplits

public InputSplit[] getSplits(JobConf job,
                     int numSplits)
                       throws IOException

Specified by:

getSplits in interface InputFormat<K,V>

Throws:

IOException

getRecordReader

public EsInputFormat.ShardRecordReader<K,V> getRecordReader(InputSplit split,
                                                   JobConf job,
                                                   Reporter reporter)

Specified by:

getRecordReader in interface InputFormat<K,V>

isOutputAsJson

protected boolean isOutputAsJson(Configuration cfg)