org.apache.cassandra.hadoop.cql3

Class CqlOutputFormat

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<K,Y>

org.apache.cassandra.hadoop.AbstractColumnFamilyOutputFormat<java.util.Map<java.lang.String,java.nio.ByteBuffer>,java.util.List<java.nio.ByteBuffer>>

org.apache.cassandra.hadoop.cql3.CqlOutputFormat

All Implemented Interfaces:

org.apache.hadoop.mapred.OutputFormat<java.util.Map<java.lang.String,java.nio.ByteBuffer>,java.util.List<java.nio.ByteBuffer>>

public class CqlOutputFormat
extends AbstractColumnFamilyOutputFormat<java.util.Map<java.lang.String,java.nio.ByteBuffer>,java.util.List<java.nio.ByteBuffer>>

The ColumnFamilyOutputFormat acts as a Hadoop-specific OutputFormat that allows reduce tasks to store keys (and corresponding bound variable values) as CQL rows (and respective columns) in a given ColumnFamily.

As is the case with the ColumnFamilyInputFormat, you need to set the prepared statement in your Hadoop job Configuration. The CqlConfigHelper class, through its ConfigHelper#setOutputPreparedStatement method, is provided to make this simple. you need to set the Keyspace. The ConfigHelper class, through its ConfigHelper.setOutputColumnFamily(org.apache.hadoop.conf.Configuration, java.lang.String) method, is provided to make this simple.

For the sake of performance, this class employs a lazy write-back caching mechanism, where its record writer prepared statement binded variable values created based on the reduce's inputs (in a task-specific map), and periodically makes the changes official by sending a execution of prepared statement request to Cassandra.

Field Summary

Fields inherited from class org.apache.cassandra.hadoop.AbstractColumnFamilyOutputFormat

BATCH_THRESHOLD, QUEUE_SIZE

Constructor Summary

Constructors

Constructor and Description
CqlOutputFormat()

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.cassandra.hadoop.cql3.CqlRecordWriter	getRecordWriter(org.apache.hadoop.fs.FileSystem filesystem, org.apache.hadoop.mapred.JobConf job, java.lang.String name, org.apache.hadoop.util.Progressable progress) Deprecated.
org.apache.cassandra.hadoop.cql3.CqlRecordWriter	getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context) Get the RecordWriter for the given task.

Methods inherited from class org.apache.cassandra.hadoop.AbstractColumnFamilyOutputFormat

checkOutputSpecs, checkOutputSpecs, checkOutputSpecs, createAuthenticatedClient, getOutputCommitter, login

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CqlOutputFormat

public CqlOutputFormat()

Method Detail

getRecordWriter

@Deprecated
public org.apache.cassandra.hadoop.cql3.CqlRecordWriter getRecordWriter(org.apache.hadoop.fs.FileSystem filesystem,
                                                                          org.apache.hadoop.mapred.JobConf job,
                                                                          java.lang.String name,
                                                                          org.apache.hadoop.util.Progressable progress)
                                                                 throws java.io.IOException

Deprecated.

Fills the deprecated OutputFormat interface for streaming.

Throws:

java.io.IOException

getRecordWriter

public org.apache.cassandra.hadoop.cql3.CqlRecordWriter getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                 throws java.io.IOException,
                                                                        java.lang.InterruptedException

Get the RecordWriter for the given task.

Specified by:

getRecordWriter in class org.apache.hadoop.mapreduce.OutputFormat<java.util.Map<java.lang.String,java.nio.ByteBuffer>,java.util.List<java.nio.ByteBuffer>>

Parameters:

context - the information about the current task.

Returns:

a RecordWriter to write the output for the job.

Throws:

java.io.IOException

java.lang.InterruptedException