org.apache.cassandra.hadoop.pig

Class CqlStorage

java.lang.Object

org.apache.pig.LoadFunc

org.apache.cassandra.hadoop.pig.AbstractCassandraStorage

org.apache.cassandra.hadoop.pig.CqlStorage

All Implemented Interfaces:

org.apache.pig.LoadMetadata, org.apache.pig.StoreFuncInterface

public class CqlStorage
extends AbstractCassandraStorage

A LoadStoreFunc for retrieving data from and storing data to Cassandra A row from a standard CF will be returned as nested tuples: (((key1, value1), (key2, value2)), ((name1, val1), (name2, val2))).

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.cassandra.hadoop.pig.AbstractCassandraStorage

AbstractCassandraStorage.MarshallerType

Field Summary

Fields inherited from class org.apache.cassandra.hadoop.pig.AbstractCassandraStorage

column_family, conf, DEFAULT_INPUT_FORMAT, DEFAULT_OUTPUT_FORMAT, inputFormatClass, keyspace, loadSignature, outputFormatClass, PARTITION_FILTER_SIGNATURE, partitionerClass, password, PIG_INITIAL_ADDRESS, PIG_INPUT_FORMAT, PIG_INPUT_INITIAL_ADDRESS, PIG_INPUT_PARTITIONER, PIG_INPUT_RPC_PORT, PIG_INPUT_SPLIT_SIZE, PIG_OUTPUT_FORMAT, PIG_OUTPUT_INITIAL_ADDRESS, PIG_OUTPUT_PARTITIONER, PIG_OUTPUT_RPC_PORT, PIG_PARTITIONER, PIG_RPC_PORT, splitSize, storeSignature, usePartitionFilter, username

Constructor Summary

Constructors

Constructor and Description
CqlStorage()
CqlStorage(int pageSize)

Method Summary

Methods

Modifier and Type	Method and Description
protected java.util.List<org.apache.cassandra.thrift.ColumnDef>	getColumnMetadata(org.apache.cassandra.thrift.Cassandra.Client client, boolean cql3Table) include key columns
org.apache.pig.data.Tuple	getNext() get next row
org.apache.pig.ResourceSchema	getSchema(java.lang.String location, org.apache.hadoop.mapreduce.Job job) schema: (value, value, value) where keys are in the front.
void	prepareToRead(org.apache.hadoop.mapreduce.RecordReader reader, org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSplit split)
void	prepareToWrite(org.apache.hadoop.mapreduce.RecordWriter writer)
void	putNext(org.apache.pig.data.Tuple t) output: (((name, value), (name, value)), (value ...
void	setLocation(java.lang.String location, org.apache.hadoop.mapreduce.Job job) set read configuration settings
void	setPartitionFilter(org.apache.pig.Expression partitionFilter)
void	setStoreLocation(java.lang.String location, org.apache.hadoop.mapreduce.Job job) set store configuration settings

Methods inherited from class org.apache.cassandra.hadoop.pig.AbstractCassandraStorage

cfdefFromString, cfdefToString, checkSchema, cleanupOnFailure, columnToTuple, composeComposite, getCfDef, getCfDef, getColumnMeta, getDefaultMarshallers, getFullyQualifiedClassName, getIndexes, getIndexType, getInputFormat, getKeysMeta, getOutputFormat, getPartitionKeys, getPigType, getQueryMap, getStatistics, getValidatorMap, initSchema, objToBB, parseType, relativeToAbsolutePath, relToAbsPathForStoreLocation, setConnectionInformation, setStoreFuncUDFContextSignature, setTupleValue, setUDFContextSignature

Methods inherited from class org.apache.pig.LoadFunc

getAbsolutePath, getLoadCaster, getPathStrings, join, warn

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CqlStorage

public CqlStorage()

CqlStorage

public CqlStorage(int pageSize)

Parameters:

pageSize - limit number of CQL rows to fetch in a thrift request

Method Detail

prepareToRead

public void prepareToRead(org.apache.hadoop.mapreduce.RecordReader reader,
                 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSplit split)

Specified by:

prepareToRead in class org.apache.pig.LoadFunc

getNext

public org.apache.pig.data.Tuple getNext()
                                  throws java.io.IOException

get next row

Specified by:

getNext in class org.apache.pig.LoadFunc

Throws:

java.io.IOException

setLocation

public void setLocation(java.lang.String location,
               org.apache.hadoop.mapreduce.Job job)
                 throws java.io.IOException

set read configuration settings

Specified by:

setLocation in class org.apache.pig.LoadFunc

Throws:

java.io.IOException

setStoreLocation

public void setStoreLocation(java.lang.String location,
                    org.apache.hadoop.mapreduce.Job job)
                      throws java.io.IOException

set store configuration settings

Throws:

java.io.IOException

getSchema

public org.apache.pig.ResourceSchema getSchema(java.lang.String location,
                                      org.apache.hadoop.mapreduce.Job job)
                                        throws java.io.IOException

schema: (value, value, value) where keys are in the front.

Throws:

java.io.IOException

setPartitionFilter

public void setPartitionFilter(org.apache.pig.Expression partitionFilter)

prepareToWrite

public void prepareToWrite(org.apache.hadoop.mapreduce.RecordWriter writer)

putNext

public void putNext(org.apache.pig.data.Tuple t)
             throws java.io.IOException

output: (((name, value), (name, value)), (value ... value), (value...value))

Throws:

java.io.IOException

getColumnMetadata

protected java.util.List<org.apache.cassandra.thrift.ColumnDef> getColumnMetadata(org.apache.cassandra.thrift.Cassandra.Client client,
                                                                      boolean cql3Table)
                                                                           throws org.apache.cassandra.thrift.InvalidRequestException,
                                                                                  org.apache.cassandra.thrift.UnavailableException,
                                                                                  org.apache.cassandra.thrift.TimedOutException,
                                                                                  org.apache.cassandra.thrift.SchemaDisagreementException,
                                                                                  org.apache.thrift.TException,
                                                                                  java.nio.charset.CharacterCodingException

include key columns

Specified by:

getColumnMetadata in class AbstractCassandraStorage

Throws:

org.apache.cassandra.thrift.InvalidRequestException

org.apache.cassandra.thrift.UnavailableException

org.apache.cassandra.thrift.TimedOutException

org.apache.cassandra.thrift.SchemaDisagreementException

org.apache.thrift.TException

java.nio.charset.CharacterCodingException