org.apache.cassandra.hadoop

Class ColumnFamilyOutputFormat

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<K,Y>

org.apache.cassandra.hadoop.AbstractColumnFamilyOutputFormat<java.nio.ByteBuffer,java.util.List<org.apache.cassandra.thrift.Mutation>>

org.apache.cassandra.hadoop.ColumnFamilyOutputFormat

All Implemented Interfaces:

org.apache.hadoop.mapred.OutputFormat<java.nio.ByteBuffer,java.util.List<org.apache.cassandra.thrift.Mutation>>

public class ColumnFamilyOutputFormat
extends AbstractColumnFamilyOutputFormat<java.nio.ByteBuffer,java.util.List<org.apache.cassandra.thrift.Mutation>>

The ColumnFamilyOutputFormat acts as a Hadoop-specific OutputFormat that allows reduce tasks to store keys (and corresponding values) as Cassandra rows (and respective columns) in a given ColumnFamily.

As is the case with the ColumnFamilyInputFormat, you need to set the Keyspace and ColumnFamily in your Hadoop job Configuration. The ConfigHelper class, through its ConfigHelper.setOutputColumnFamily(org.apache.hadoop.conf.Configuration, java.lang.String) method, is provided to make this simple.

For the sake of performance, this class employs a lazy write-back caching mechanism, where its record writer batches mutations created based on the reduce's inputs (in a task-specific map), and periodically makes the changes official by sending a batch mutate request to Cassandra.

Field Summary

Fields inherited from class org.apache.cassandra.hadoop.AbstractColumnFamilyOutputFormat

BATCH_THRESHOLD, QUEUE_SIZE

Constructor Summary

Constructors

Constructor and Description
ColumnFamilyOutputFormat()

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.cassandra.hadoop.ColumnFamilyRecordWriter	getRecordWriter(org.apache.hadoop.fs.FileSystem filesystem, org.apache.hadoop.mapred.JobConf job, java.lang.String name, org.apache.hadoop.util.Progressable progress) Deprecated.
org.apache.cassandra.hadoop.ColumnFamilyRecordWriter	getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context) Get the RecordWriter for the given task.

Methods inherited from class org.apache.cassandra.hadoop.AbstractColumnFamilyOutputFormat

checkOutputSpecs, checkOutputSpecs, checkOutputSpecs, createAuthenticatedClient, getOutputCommitter, login

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ColumnFamilyOutputFormat

public ColumnFamilyOutputFormat()

Method Detail

getRecordWriter

@Deprecated
public org.apache.cassandra.hadoop.ColumnFamilyRecordWriter getRecordWriter(org.apache.hadoop.fs.FileSystem filesystem,
                                                                              org.apache.hadoop.mapred.JobConf job,
                                                                              java.lang.String name,
                                                                              org.apache.hadoop.util.Progressable progress)

Deprecated.

Fills the deprecated OutputFormat interface for streaming.

getRecordWriter

public org.apache.cassandra.hadoop.ColumnFamilyRecordWriter getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                     throws java.lang.InterruptedException

Get the RecordWriter for the given task.

Specified by:

getRecordWriter in class org.apache.hadoop.mapreduce.OutputFormat<java.nio.ByteBuffer,java.util.List<org.apache.cassandra.thrift.Mutation>>

Parameters:

context - the information about the current task.

Returns:

a RecordWriter to write the output for the job.

Throws:

java.io.IOException

java.lang.InterruptedException