org.apache.cassandra.hadoop.pig

Class CqlNativeStorage

java.lang.Object

org.apache.pig.LoadFunc

org.apache.cassandra.hadoop.pig.AbstractCassandraStorage

org.apache.cassandra.hadoop.pig.CqlNativeStorage

All Implemented Interfaces:

org.apache.pig.LoadMetadata, org.apache.pig.StoreFuncInterface

Direct Known Subclasses:

CqlStorage

public class CqlNativeStorage
extends AbstractCassandraStorage

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.cassandra.hadoop.pig.AbstractCassandraStorage

AbstractCassandraStorage.CfInfo, AbstractCassandraStorage.MarshallerType

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	BULK_OUTPUT_FORMAT

Fields inherited from class org.apache.cassandra.hadoop.pig.AbstractCassandraStorage

column_family, conf, DEFAULT_INPUT_FORMAT, DEFAULT_OUTPUT_FORMAT, initHostAddress, inputFormatClass, keyspace, loadSignature, nativeProtocolVersion, outputFormatClass, PARTITION_FILTER_SIGNATURE, partitionerClass, password, PIG_INITIAL_ADDRESS, PIG_INPUT_FORMAT, PIG_INPUT_INITIAL_ADDRESS, PIG_INPUT_PARTITIONER, PIG_INPUT_RPC_PORT, PIG_INPUT_SPLIT_SIZE, PIG_OUTPUT_FORMAT, PIG_OUTPUT_INITIAL_ADDRESS, PIG_OUTPUT_PARTITIONER, PIG_OUTPUT_RPC_PORT, PIG_PARTITIONER, PIG_RPC_PORT, rpcPort, splitSize, storeSignature, usePartitionFilter, username

Constructor Summary

Constructors

Constructor and Description
CqlNativeStorage()
CqlNativeStorage(int pageSize)

Method Summary

Methods

Modifier and Type	Method and Description
protected java.util.List<org.apache.cassandra.thrift.ColumnDef>	getColumnMetadata(org.apache.cassandra.thrift.Cassandra.Client client) include key columns
org.apache.pig.data.Tuple	getNext() get next row
org.apache.pig.ResourceSchema	getSchema(java.lang.String location, org.apache.hadoop.mapreduce.Job job) schema: (value, value, value) where keys are in the front.
java.nio.ByteBuffer	nullToBB() Thrift API can't handle null, so use empty byte array
void	prepareToRead(org.apache.hadoop.mapreduce.RecordReader reader, org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSplit split)
void	prepareToWrite(org.apache.hadoop.mapreduce.RecordWriter writer)
void	putNext(org.apache.pig.data.Tuple t) output: (((name, value), (name, value)), (value ...
void	setLocation(java.lang.String location, org.apache.hadoop.mapreduce.Job job) set read configuration settings
void	setPartitionFilter(org.apache.pig.Expression partitionFilter)
void	setStoreLocation(java.lang.String location, org.apache.hadoop.mapreduce.Job job) set store configuration settings

Methods inherited from class org.apache.cassandra.hadoop.pig.AbstractCassandraStorage

cassandraToObj, cfdefFromString, cfdefToString, checkSchema, cleanupOnFailure, cleanupOnSuccess, columnToTuple, composeComposite, getCfInfo, getCfInfo, getCFMetaData, getColumnMeta, getDefaultMarshallers, getFullyQualifiedClassName, getIndexes, getIndexType, getInputFormat, getOutputFormat, getPartitionKeys, getPigType, getQueryMap, getStatistics, getValidatorMap, initSchema, objToBB, parseType, relativeToAbsolutePath, relToAbsPathForStoreLocation, setConnectionInformation, setStoreFuncUDFContextSignature, setTupleValue, setUDFContextSignature

Methods inherited from class org.apache.pig.LoadFunc

getAbsolutePath, getLoadCaster, getPathStrings, join, warn

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

BULK_OUTPUT_FORMAT

public static java.lang.String BULK_OUTPUT_FORMAT

Constructor Detail

CqlNativeStorage

public CqlNativeStorage()

CqlNativeStorage

public CqlNativeStorage(int pageSize)

Parameters:

pageSize - limit number of CQL rows to fetch in a thrift request

Method Detail

prepareToRead

public void prepareToRead(org.apache.hadoop.mapreduce.RecordReader reader,
                 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSplit split)

Specified by:

prepareToRead in class org.apache.pig.LoadFunc

prepareToWrite

public void prepareToWrite(org.apache.hadoop.mapreduce.RecordWriter writer)

getNext

public org.apache.pig.data.Tuple getNext()
                                  throws java.io.IOException

get next row

Specified by:

getNext in class org.apache.pig.LoadFunc

Throws:

java.io.IOException

getColumnMetadata

protected java.util.List<org.apache.cassandra.thrift.ColumnDef> getColumnMetadata(org.apache.cassandra.thrift.Cassandra.Client client)
                                                                           throws org.apache.cassandra.thrift.InvalidRequestException,
                                                                                  org.apache.cassandra.thrift.UnavailableException,
                                                                                  org.apache.cassandra.thrift.TimedOutException,
                                                                                  org.apache.cassandra.thrift.SchemaDisagreementException,
                                                                                  org.apache.thrift.TException,
                                                                                  java.nio.charset.CharacterCodingException,
                                                                                  InvalidRequestException,
                                                                                  ConfigurationException,
                                                                                  org.apache.cassandra.thrift.NotFoundException

include key columns

Specified by:

getColumnMetadata in class AbstractCassandraStorage

Throws:

org.apache.cassandra.thrift.InvalidRequestException

org.apache.cassandra.thrift.UnavailableException

org.apache.cassandra.thrift.TimedOutException

org.apache.cassandra.thrift.SchemaDisagreementException

org.apache.thrift.TException

java.nio.charset.CharacterCodingException

ConfigurationException

org.apache.cassandra.thrift.NotFoundException

getSchema

public org.apache.pig.ResourceSchema getSchema(java.lang.String location,
                                      org.apache.hadoop.mapreduce.Job job)
                                        throws java.io.IOException

schema: (value, value, value) where keys are in the front.

Throws:

java.io.IOException

setPartitionFilter

public void setPartitionFilter(org.apache.pig.Expression partitionFilter)
                        throws java.io.IOException

Throws:

java.io.IOException

putNext

public void putNext(org.apache.pig.data.Tuple t)
             throws java.io.IOException

output: (((name, value), (name, value)), (value ... value), (value...value)) bulk output: ((value ... value), (value...value))

Throws:

java.io.IOException

setLocation

public void setLocation(java.lang.String location,
               org.apache.hadoop.mapreduce.Job job)
                 throws java.io.IOException

set read configuration settings

Specified by:

setLocation in class org.apache.pig.LoadFunc

Throws:

java.io.IOException

setStoreLocation

public void setStoreLocation(java.lang.String location,
                    org.apache.hadoop.mapreduce.Job job)
                      throws java.io.IOException

set store configuration settings

Throws:

java.io.IOException

nullToBB

public java.nio.ByteBuffer nullToBB()

Thrift API can't handle null, so use empty byte array

Specified by:

nullToBB in class AbstractCassandraStorage