org.apache.cassandra.hadoop

Class ColumnFamilyOutputFormat

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<K,Y>

org.apache.cassandra.hadoop.AbstractColumnFamilyOutputFormat<java.nio.ByteBuffer,java.util.List<org.apache.cassandra.thrift.Mutation>>

org.apache.cassandra.hadoop.ColumnFamilyOutputFormat

All Implemented Interfaces:

org.apache.hadoop.mapred.OutputFormat<java.nio.ByteBuffer,java.util.List<org.apache.cassandra.thrift.Mutation>>

public class ColumnFamilyOutputFormat
extends AbstractColumnFamilyOutputFormat<java.nio.ByteBuffer,java.util.List<org.apache.cassandra.thrift.Mutation>>

The ColumnFamilyOutputFormat acts as a Hadoop-specific OutputFormat that allows reduce tasks to store keys (and corresponding values) as Cassandra rows (and respective columns) in a given ColumnFamily.

As is the case with the ColumnFamilyInputFormat, you need to set the Keyspace and ColumnFamily in your Hadoop job Configuration. The ConfigHelper class, through its ConfigHelper.setOutputColumnFamily(org.apache.hadoop.conf.Configuration, java.lang.String) method, is provided to make this simple.

For the sake of performance, this class employs a lazy write-back caching mechanism, where its record writer batches mutations created based on the reduce's inputs (in a task-specific map), and periodically makes the changes official by sending a batch mutate request to Cassandra.

Field Summary

Fields inherited from class org.apache.cassandra.hadoop.AbstractColumnFamilyOutputFormat

BATCH_THRESHOLD, QUEUE_SIZE

Constructor Summary

Constructors

Constructor and Description
ColumnFamilyOutputFormat()

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.cassandra.hadoop.ColumnFamilyRecordWriter	getRecordWriter(org.apache.hadoop.fs.FileSystem filesystem, org.apache.hadoop.mapred.JobConf job, java.lang.String name, org.apache.hadoop.util.Progressable progress) Deprecated.
org.apache.cassandra.hadoop.ColumnFamilyRecordWriter	getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context) Get the RecordWriter for the given task.

Methods inherited from class org.apache.cassandra.hadoop.AbstractColumnFamilyOutputFormat

checkOutputSpecs, checkOutputSpecs, checkOutputSpecs, createAuthenticatedClient, getOutputCommitter

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ColumnFamilyOutputFormat

public ColumnFamilyOutputFormat()

Method Detail

getRecordWriter

@Deprecated
public org.apache.cassandra.hadoop.ColumnFamilyRecordWriter getRecordWriter(org.apache.hadoop.fs.FileSystem filesystem,
                                                                              org.apache.hadoop.mapred.JobConf job,
                                                                              java.lang.String name,
                                                                              org.apache.hadoop.util.Progressable progress)

Deprecated.

Fills the deprecated OutputFormat interface for streaming.

getRecordWriter

public org.apache.cassandra.hadoop.ColumnFamilyRecordWriter getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                     throws java.lang.InterruptedException

Get the RecordWriter for the given task.

Specified by:

getRecordWriter in class org.apache.hadoop.mapreduce.OutputFormat<java.nio.ByteBuffer,java.util.List<org.apache.cassandra.thrift.Mutation>>

Parameters:

context - the information about the current task.

Returns:

a RecordWriter to write the output for the job.

Throws:

java.io.IOException

java.lang.InterruptedException