Package com.yahoo.vespa.hadoop.mapreduce

Class VespaOutputFormat

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat

com.yahoo.vespa.hadoop.mapreduce.VespaOutputFormat

public class VespaOutputFormat
extends org.apache.hadoop.mapreduce.OutputFormat

An output specification for writing to Vespa instances in a Map-Reduce job. Mainly returns an instance of a LegacyVespaRecordWriter that does the actual feeding to Vespa.

Author:

lesters

Constructor Summary

Constructors

Constructor	Description
VespaOutputFormat()
VespaOutputFormat(Properties configOverride)

Method Summary

Modifier and Type	Method	Description
void	checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext jobContext)
org.apache.hadoop.mapreduce.OutputCommitter	getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext taskAttemptContext)
org.apache.hadoop.mapreduce.RecordWriter	getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

VespaOutputFormat

public VespaOutputFormat()

VespaOutputFormat

public VespaOutputFormat(Properties configOverride)

Method Details

getRecordWriter

public org.apache.hadoop.mapreduce.RecordWriter getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                         throws IOException

Specified by:

getRecordWriter in class org.apache.hadoop.mapreduce.OutputFormat

Throws:

IOException

getOutputCommitter

public org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext taskAttemptContext)
                                                               throws IOException,
InterruptedException

Specified by:

getOutputCommitter in class org.apache.hadoop.mapreduce.OutputFormat

Throws:

IOException

InterruptedException

checkOutputSpecs

public void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext jobContext)
                      throws IOException,
InterruptedException

Specified by:

checkOutputSpecs in class org.apache.hadoop.mapreduce.OutputFormat

Throws:

IOException

InterruptedException