org.apache.cassandra.hadoop.pig

Class CassandraStorage

java.lang.Object

org.apache.pig.LoadFunc

org.apache.cassandra.hadoop.pig.AbstractCassandraStorage

org.apache.cassandra.hadoop.pig.CassandraStorage

All Implemented Interfaces:

org.apache.pig.LoadMetadata, org.apache.pig.StoreFuncInterface

public class CassandraStorage
extends AbstractCassandraStorage

A LoadStoreFunc for retrieving data from and storing data to Cassandra A row from a standard CF will be returned as nested tuples: (key, ((name1, val1), (name2, val2))).

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.cassandra.hadoop.pig.AbstractCassandraStorage

AbstractCassandraStorage.MarshallerType

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	PIG_ALLOW_DELETES
static java.lang.String	PIG_USE_SECONDARY
static java.lang.String	PIG_WIDEROW_INPUT

Fields inherited from class org.apache.cassandra.hadoop.pig.AbstractCassandraStorage

column_family, conf, DEFAULT_INPUT_FORMAT, DEFAULT_OUTPUT_FORMAT, inputFormatClass, keyspace, loadSignature, outputFormatClass, PARTITION_FILTER_SIGNATURE, partitionerClass, password, PIG_INITIAL_ADDRESS, PIG_INPUT_FORMAT, PIG_INPUT_INITIAL_ADDRESS, PIG_INPUT_PARTITIONER, PIG_INPUT_RPC_PORT, PIG_INPUT_SPLIT_SIZE, PIG_OUTPUT_FORMAT, PIG_OUTPUT_INITIAL_ADDRESS, PIG_OUTPUT_PARTITIONER, PIG_OUTPUT_RPC_PORT, PIG_PARTITIONER, PIG_RPC_PORT, splitSize, storeSignature, usePartitionFilter, username

Constructor Summary

Constructors

Constructor and Description
CassandraStorage()
CassandraStorage(int limit)

Method Summary

Methods

Modifier and Type	Method and Description
protected java.util.List<org.apache.cassandra.thrift.ColumnDef>	getColumnMetadata(org.apache.cassandra.thrift.Cassandra.Client client, boolean cql3Table) get a list of column for the column family
int	getLimit()
org.apache.pig.data.Tuple	getNext()
org.apache.pig.data.Tuple	getNextWide() read wide row
org.apache.pig.ResourceSchema	getSchema(java.lang.String location, org.apache.hadoop.mapreduce.Job job) define the schema
void	prepareToRead(org.apache.hadoop.mapreduce.RecordReader reader, org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSplit split)
void	prepareToWrite(org.apache.hadoop.mapreduce.RecordWriter writer) prepare writer
void	putNext(org.apache.pig.data.Tuple t) write next row
protected void	setConnectionInformation() set hadoop cassandra connection settings
void	setLocation(java.lang.String location, org.apache.hadoop.mapreduce.Job job) set read configuration settings
void	setPartitionFilter(org.apache.pig.Expression partitionFilter) set partition filter
void	setStoreLocation(java.lang.String location, org.apache.hadoop.mapreduce.Job job) set store configuration settings

Methods inherited from class org.apache.cassandra.hadoop.pig.AbstractCassandraStorage

cfdefFromString, cfdefToString, checkSchema, cleanupOnFailure, columnToTuple, composeComposite, getCfDef, getCfDef, getColumnMeta, getDefaultMarshallers, getFullyQualifiedClassName, getIndexes, getIndexType, getInputFormat, getKeysMeta, getOutputFormat, getPartitionKeys, getPigType, getQueryMap, getStatistics, getValidatorMap, initSchema, objToBB, parseType, relativeToAbsolutePath, relToAbsPathForStoreLocation, setStoreFuncUDFContextSignature, setTupleValue, setUDFContextSignature

Methods inherited from class org.apache.pig.LoadFunc

getAbsolutePath, getLoadCaster, getPathStrings, join, warn

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PIG_ALLOW_DELETES

public static final java.lang.String PIG_ALLOW_DELETES

See Also:

Constant Field Values

PIG_WIDEROW_INPUT

public static final java.lang.String PIG_WIDEROW_INPUT

See Also:

Constant Field Values

PIG_USE_SECONDARY

public static final java.lang.String PIG_USE_SECONDARY

See Also:

Constant Field Values

Constructor Detail

CassandraStorage

public CassandraStorage()

CassandraStorage

public CassandraStorage(int limit)

Parameters:

limit - number of columns to fetch in a slice

Method Detail

getLimit

public int getLimit()

prepareToRead

public void prepareToRead(org.apache.hadoop.mapreduce.RecordReader reader,
                 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSplit split)

Specified by:

prepareToRead in class org.apache.pig.LoadFunc

getNextWide

public org.apache.pig.data.Tuple getNextWide()
                                      throws java.io.IOException

read wide row

Throws:

java.io.IOException

getNext

public org.apache.pig.data.Tuple getNext()
                                  throws java.io.IOException

Specified by:

getNext in class org.apache.pig.LoadFunc

Throws:

java.io.IOException

setConnectionInformation

protected void setConnectionInformation()
                                 throws java.io.IOException

set hadoop cassandra connection settings

Overrides:

setConnectionInformation in class AbstractCassandraStorage

Throws:

java.io.IOException

setLocation

public void setLocation(java.lang.String location,
               org.apache.hadoop.mapreduce.Job job)
                 throws java.io.IOException

set read configuration settings

Specified by:

setLocation in class org.apache.pig.LoadFunc

Throws:

java.io.IOException

setStoreLocation

public void setStoreLocation(java.lang.String location,
                    org.apache.hadoop.mapreduce.Job job)
                      throws java.io.IOException

set store configuration settings

Throws:

java.io.IOException

getSchema

public org.apache.pig.ResourceSchema getSchema(java.lang.String location,
                                      org.apache.hadoop.mapreduce.Job job)
                                        throws java.io.IOException

define the schema

Throws:

java.io.IOException

setPartitionFilter

public void setPartitionFilter(org.apache.pig.Expression partitionFilter)

set partition filter

prepareToWrite

public void prepareToWrite(org.apache.hadoop.mapreduce.RecordWriter writer)

prepare writer

putNext

public void putNext(org.apache.pig.data.Tuple t)
             throws java.io.IOException

write next row

Throws:

java.io.IOException

getColumnMetadata

protected java.util.List<org.apache.cassandra.thrift.ColumnDef> getColumnMetadata(org.apache.cassandra.thrift.Cassandra.Client client,
                                                                      boolean cql3Table)
                                                                           throws org.apache.cassandra.thrift.InvalidRequestException,
                                                                                  org.apache.cassandra.thrift.UnavailableException,
                                                                                  org.apache.cassandra.thrift.TimedOutException,
                                                                                  org.apache.cassandra.thrift.SchemaDisagreementException,
                                                                                  org.apache.thrift.TException,
                                                                                  java.nio.charset.CharacterCodingException

get a list of column for the column family

Specified by:

getColumnMetadata in class AbstractCassandraStorage

Throws:

org.apache.cassandra.thrift.InvalidRequestException

org.apache.cassandra.thrift.UnavailableException

org.apache.cassandra.thrift.TimedOutException

org.apache.cassandra.thrift.SchemaDisagreementException

org.apache.thrift.TException

java.nio.charset.CharacterCodingException