org.apache.cassandra.hadoop

Class ColumnFamilyOutputFormat

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<java.nio.ByteBuffer,java.util.List<org.apache.cassandra.thrift.Mutation>>

org.apache.cassandra.hadoop.ColumnFamilyOutputFormat

All Implemented Interfaces:

org.apache.hadoop.mapred.OutputFormat<java.nio.ByteBuffer,java.util.List<org.apache.cassandra.thrift.Mutation>>

Deprecated.

@Deprecated
public class ColumnFamilyOutputFormat
extends org.apache.hadoop.mapreduce.OutputFormat<java.nio.ByteBuffer,java.util.List<org.apache.cassandra.thrift.Mutation>>
implements org.apache.hadoop.mapred.OutputFormat<java.nio.ByteBuffer,java.util.List<org.apache.cassandra.thrift.Mutation>>

The ColumnFamilyOutputFormat acts as a Hadoop-specific OutputFormat that allows reduce tasks to store keys (and corresponding values) as Cassandra rows (and respective columns) in a given ColumnFamily.

As is the case with the ColumnFamilyInputFormat, you need to set the Keyspace and ColumnFamily in your Hadoop job Configuration. The ConfigHelper class, through its ConfigHelper.setOutputColumnFamily(org.apache.hadoop.conf.Configuration, java.lang.String) method, is provided to make this simple.

For the sake of performance, this class employs a lazy write-back caching mechanism, where its record writer batches mutations created based on the reduce's inputs (in a task-specific map), and periodically makes the changes official by sending a batch mutate request to Cassandra.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	BATCH_THRESHOLD Deprecated.
static java.lang.String	QUEUE_SIZE Deprecated.

Constructor Summary

Constructors

Constructor and Description
ColumnFamilyOutputFormat() Deprecated.

Method Summary

Modifier and Type	Method and Description
protected void	checkOutputSpecs(org.apache.hadoop.conf.Configuration conf) Deprecated.
void	checkOutputSpecs(org.apache.hadoop.fs.FileSystem filesystem, org.apache.hadoop.mapred.JobConf job) Deprecated.
void	checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context) Deprecated. Check for validity of the output-specification for the job.
static org.apache.cassandra.thrift.Cassandra.Client	createAuthenticatedClient(java.lang.String host, int port, org.apache.hadoop.conf.Configuration conf) Deprecated. Connects to the given server:port and returns a client based on the given socket that points to the configured keyspace, and is logged in with the configured credentials.
org.apache.hadoop.mapreduce.OutputCommitter	getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context) Deprecated. The OutputCommitter for this format does not write any data to the DFS.
org.apache.cassandra.hadoop.ColumnFamilyRecordWriter	getRecordWriter(org.apache.hadoop.fs.FileSystem filesystem, org.apache.hadoop.mapred.JobConf job, java.lang.String name, org.apache.hadoop.util.Progressable progress) Deprecated.
org.apache.cassandra.hadoop.ColumnFamilyRecordWriter	getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context) Deprecated. Get the RecordWriter for the given task.
static void	login(java.lang.String user, java.lang.String password, org.apache.cassandra.thrift.Cassandra.Client client) Deprecated.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

BATCH_THRESHOLD

public static final java.lang.String BATCH_THRESHOLD

Deprecated.

See Also:

Constant Field Values

QUEUE_SIZE

public static final java.lang.String QUEUE_SIZE

Deprecated.

See Also:

Constant Field Values

Constructor Detail

ColumnFamilyOutputFormat

public ColumnFamilyOutputFormat()

Deprecated.

Method Detail

getOutputCommitter

public org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                               throws java.io.IOException,
                                                                      java.lang.InterruptedException

Deprecated.

The OutputCommitter for this format does not write any data to the DFS.

Specified by:

getOutputCommitter in class org.apache.hadoop.mapreduce.OutputFormat<java.nio.ByteBuffer,java.util.List<org.apache.cassandra.thrift.Mutation>>

Parameters:

context - the task context

Returns:

an output committer

Throws:

java.io.IOException

java.lang.InterruptedException

checkOutputSpecs

public void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context)

Deprecated.

Check for validity of the output-specification for the job.

Specified by:

checkOutputSpecs in class org.apache.hadoop.mapreduce.OutputFormat<java.nio.ByteBuffer,java.util.List<org.apache.cassandra.thrift.Mutation>>

Parameters:

context - information about the job

checkOutputSpecs

protected void checkOutputSpecs(org.apache.hadoop.conf.Configuration conf)

Deprecated.

checkOutputSpecs

@Deprecated
public void checkOutputSpecs(org.apache.hadoop.fs.FileSystem filesystem,
                                         org.apache.hadoop.mapred.JobConf job)
                                  throws java.io.IOException

Deprecated.

Fills the deprecated OutputFormat interface for streaming.

Specified by:

checkOutputSpecs in interface org.apache.hadoop.mapred.OutputFormat<java.nio.ByteBuffer,java.util.List<org.apache.cassandra.thrift.Mutation>>

Throws:

java.io.IOException

createAuthenticatedClient

public static org.apache.cassandra.thrift.Cassandra.Client createAuthenticatedClient(java.lang.String host,
                                                                                     int port,
                                                                                     org.apache.hadoop.conf.Configuration conf)
                                                                              throws java.lang.Exception

Deprecated.

Connects to the given server:port and returns a client based on the given socket that points to the configured keyspace, and is logged in with the configured credentials.

Parameters:

host - fully qualified host name to connect to

port - RPC port of the server

conf - a job configuration

Returns:

a cassandra client

Throws:

java.lang.Exception - set of thrown exceptions may be implementation defined, depending on the used transport factory

login

public static void login(java.lang.String user,
                         java.lang.String password,
                         org.apache.cassandra.thrift.Cassandra.Client client)
                  throws java.lang.Exception

Deprecated.

Throws:

java.lang.Exception

getRecordWriter

@Deprecated
public org.apache.cassandra.hadoop.ColumnFamilyRecordWriter getRecordWriter(org.apache.hadoop.fs.FileSystem filesystem,
                                                                                        org.apache.hadoop.mapred.JobConf job,
                                                                                        java.lang.String name,
                                                                                        org.apache.hadoop.util.Progressable progress)

Deprecated.

Fills the deprecated OutputFormat interface for streaming.

Specified by:

getRecordWriter in interface org.apache.hadoop.mapred.OutputFormat<java.nio.ByteBuffer,java.util.List<org.apache.cassandra.thrift.Mutation>>

getRecordWriter

public org.apache.cassandra.hadoop.ColumnFamilyRecordWriter getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                     throws java.lang.InterruptedException

Deprecated.

Get the RecordWriter for the given task.

Specified by:

getRecordWriter in class org.apache.hadoop.mapreduce.OutputFormat<java.nio.ByteBuffer,java.util.List<org.apache.cassandra.thrift.Mutation>>

Parameters:

context - the information about the current task.

Returns:

a RecordWriter to write the output for the job.

Throws:

java.lang.InterruptedException