org.apache.cassandra.hadoop

Class ColumnFamilyInputFormat

java.lang.Object

org.apache.hadoop.mapreduce.InputFormat<K,Y>

org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat<java.nio.ByteBuffer,java.util.SortedMap<java.nio.ByteBuffer,ColumnFamilyRecordReader.Column>>

org.apache.cassandra.hadoop.ColumnFamilyInputFormat

All Implemented Interfaces:

org.apache.hadoop.mapred.InputFormat<java.nio.ByteBuffer,java.util.SortedMap<java.nio.ByteBuffer,ColumnFamilyRecordReader.Column>>

Deprecated.

@Deprecated
public class ColumnFamilyInputFormat
extends AbstractColumnFamilyInputFormat<java.nio.ByteBuffer,java.util.SortedMap<java.nio.ByteBuffer,ColumnFamilyRecordReader.Column>>

Hadoop InputFormat allowing map/reduce against Cassandra rows within one ColumnFamily. At minimum, you need to set the CF and predicate (description of columns to extract from each row) in your Hadoop job Configuration. The ConfigHelper class is provided to make this simple: ConfigHelper.setInputColumnFamily ConfigHelper.setInputSlicePredicate You can also configure the number of rows per InputSplit with ConfigHelper.setInputSplitSize This should be "as big as possible, but no bigger." Each InputSplit is read from Cassandra with multiple get_slice_range queries, and the per-call overhead of get_slice_range is high, so larger split sizes are better -- but if it is too large, you will run out of memory. The default split size is 64k rows.

Field Summary

Fields inherited from class org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat

CASSANDRA_HADOOP_MAX_KEY_SIZE, CASSANDRA_HADOOP_MAX_KEY_SIZE_DEFAULT, MAPRED_TASK_ID

Constructor Summary

Constructors

Constructor and Description
ColumnFamilyInputFormat() Deprecated.

Method Summary

Modifier and Type	Method and Description
static org.apache.cassandra.thrift.Cassandra.Client	createAuthenticatedClient(java.lang.String location, int port, org.apache.hadoop.conf.Configuration conf) Deprecated.
org.apache.hadoop.mapreduce.RecordReader<java.nio.ByteBuffer,java.util.SortedMap<java.nio.ByteBuffer,ColumnFamilyRecordReader.Column>>	createRecordReader(org.apache.hadoop.mapreduce.InputSplit inputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext taskAttemptContext) Deprecated.
org.apache.hadoop.mapred.RecordReader<java.nio.ByteBuffer,java.util.SortedMap<java.nio.ByteBuffer,ColumnFamilyRecordReader.Column>>	getRecordReader(org.apache.hadoop.mapred.InputSplit split, org.apache.hadoop.mapred.JobConf jobConf, org.apache.hadoop.mapred.Reporter reporter) Deprecated.
protected void	validateConfiguration(org.apache.hadoop.conf.Configuration conf) Deprecated.

Methods inherited from class org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat

getSplits, getSplits

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ColumnFamilyInputFormat

public ColumnFamilyInputFormat()

Deprecated.

Method Detail

createAuthenticatedClient

public static org.apache.cassandra.thrift.Cassandra.Client createAuthenticatedClient(java.lang.String location,
                                                                                     int port,
                                                                                     org.apache.hadoop.conf.Configuration conf)
                                                                              throws java.lang.Exception

Deprecated.

Throws:

java.lang.Exception

createRecordReader

public org.apache.hadoop.mapreduce.RecordReader<java.nio.ByteBuffer,java.util.SortedMap<java.nio.ByteBuffer,ColumnFamilyRecordReader.Column>> createRecordReader(org.apache.hadoop.mapreduce.InputSplit inputSplit,
                                                                                                                                                                 org.apache.hadoop.mapreduce.TaskAttemptContext taskAttemptContext)
                                                                                                                                                          throws java.io.IOException,
                                                                                                                                                                 java.lang.InterruptedException

Deprecated.

Specified by:

createRecordReader in class org.apache.hadoop.mapreduce.InputFormat<java.nio.ByteBuffer,java.util.SortedMap<java.nio.ByteBuffer,ColumnFamilyRecordReader.Column>>

Throws:

java.io.IOException

java.lang.InterruptedException

getRecordReader

public org.apache.hadoop.mapred.RecordReader<java.nio.ByteBuffer,java.util.SortedMap<java.nio.ByteBuffer,ColumnFamilyRecordReader.Column>> getRecordReader(org.apache.hadoop.mapred.InputSplit split,
                                                                                                                                                           org.apache.hadoop.mapred.JobConf jobConf,
                                                                                                                                                           org.apache.hadoop.mapred.Reporter reporter)
                                                                                                                                                    throws java.io.IOException

Deprecated.

Throws:

java.io.IOException

validateConfiguration

protected void validateConfiguration(org.apache.hadoop.conf.Configuration conf)

Deprecated.

Overrides:

validateConfiguration in class AbstractColumnFamilyInputFormat<java.nio.ByteBuffer,java.util.SortedMap<java.nio.ByteBuffer,ColumnFamilyRecordReader.Column>>