org.apache.cassandra.hadoop.cql3

Class CqlBulkOutputFormat

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<K,V>

org.apache.cassandra.hadoop.AbstractBulkOutputFormat<java.lang.Object,java.util.List<java.nio.ByteBuffer>>

org.apache.cassandra.hadoop.cql3.CqlBulkOutputFormat

All Implemented Interfaces:

org.apache.hadoop.mapred.OutputFormat<java.lang.Object,java.util.List<java.nio.ByteBuffer>>

public class CqlBulkOutputFormat
extends AbstractBulkOutputFormat<java.lang.Object,java.util.List<java.nio.ByteBuffer>>

The CqlBulkOutputFormat acts as a Hadoop-specific OutputFormat that allows reduce tasks to store keys (and corresponding bound variable values) as CQL rows (and respective columns) in a given ColumnFamily.

As is the case with the org.apache.cassandra.hadoop.CqlOutputFormat, you need to set the prepared statement in your Hadoop job Configuration. The CqlConfigHelper class, through its ConfigHelper#setOutputPreparedStatement method, is provided to make this simple. you need to set the Keyspace. The ConfigHelper class, through its ConfigHelper.setOutputColumnFamily(org.apache.hadoop.conf.Configuration, java.lang.String) method, is provided to make this simple.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.cassandra.hadoop.AbstractBulkOutputFormat

AbstractBulkOutputFormat.NullOutputCommitter

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_SSL_STORAGE_PORT
static int	DEFAULT_STORAGE_PORT

Constructor Summary

Constructors

Constructor and Description
CqlBulkOutputFormat()

Method Summary

Modifier and Type	Method and Description
static java.lang.String	getColumnFamilyInsertStatement(org.apache.hadoop.conf.Configuration conf, java.lang.String columnFamily)
static java.lang.String	getColumnFamilySchema(org.apache.hadoop.conf.Configuration conf, java.lang.String columnFamily)
static boolean	getDeleteSourceOnSuccess(org.apache.hadoop.conf.Configuration conf)
static java.lang.String	getInternodeEncryption(org.apache.hadoop.conf.Configuration conf)
CqlBulkRecordWriter	getRecordWriter(org.apache.hadoop.fs.FileSystem filesystem, org.apache.hadoop.mapred.JobConf job, java.lang.String name, org.apache.hadoop.util.Progressable progress) Deprecated.
CqlBulkRecordWriter	getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context) Get the RecordWriter for the given task.
static java.lang.String	getServerCipherSuites(org.apache.hadoop.conf.Configuration conf)
static java.lang.String	getServerKeystore(org.apache.hadoop.conf.Configuration conf)
static java.lang.String	getServerKeystorePassword(org.apache.hadoop.conf.Configuration conf)
static java.lang.String	getServerTruststore(org.apache.hadoop.conf.Configuration conf)
static java.lang.String	getServerTruststorePassword(org.apache.hadoop.conf.Configuration conf)
static int	getSSLStoragePort(org.apache.hadoop.conf.Configuration conf)
static int	getStoragePort(org.apache.hadoop.conf.Configuration conf)
static void	setColumnFamilyInsertStatement(org.apache.hadoop.conf.Configuration conf, java.lang.String columnFamily, java.lang.String insertStatement)
static void	setColumnFamilySchema(org.apache.hadoop.conf.Configuration conf, java.lang.String columnFamily, java.lang.String schema)
static void	setDeleteSourceOnSuccess(org.apache.hadoop.conf.Configuration conf, boolean deleteSrc)
static void	setInternodeEncryption(org.apache.hadoop.conf.Configuration conf, java.lang.String encrypt)
static void	setServerCipherSuites(org.apache.hadoop.conf.Configuration conf, java.lang.String cipherSuites)
static void	setServerKeystore(org.apache.hadoop.conf.Configuration conf, java.lang.String keystore)
static void	setServerKeystorePassword(org.apache.hadoop.conf.Configuration conf, java.lang.String keystorePass)
static void	setServerTruststore(org.apache.hadoop.conf.Configuration conf, java.lang.String truststore)
static void	setServerTruststorePassword(org.apache.hadoop.conf.Configuration conf, java.lang.String truststorePass)
static void	setSSLStoragePort(org.apache.hadoop.conf.Configuration conf, int port)
static void	setStoragePort(org.apache.hadoop.conf.Configuration conf, int port)

Methods inherited from class org.apache.cassandra.hadoop.AbstractBulkOutputFormat

checkOutputSpecs, checkOutputSpecs, getIgnoreHosts, getOutputCommitter, setIgnoreHosts, setIgnoreHosts

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_STORAGE_PORT

public static final int DEFAULT_STORAGE_PORT

See Also:

Constant Field Values

DEFAULT_SSL_STORAGE_PORT

public static final int DEFAULT_SSL_STORAGE_PORT

See Also:

Constant Field Values

Constructor Detail

CqlBulkOutputFormat

public CqlBulkOutputFormat()

Method Detail

getRecordWriter

@Deprecated
public CqlBulkRecordWriter getRecordWriter(org.apache.hadoop.fs.FileSystem filesystem,
                                                       org.apache.hadoop.mapred.JobConf job,
                                                       java.lang.String name,
                                                       org.apache.hadoop.util.Progressable progress)
                                                throws java.io.IOException

Deprecated.

Fills the deprecated OutputFormat interface for streaming.

Throws:

java.io.IOException

getRecordWriter

public CqlBulkRecordWriter getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                    throws java.io.IOException,
                                           java.lang.InterruptedException

Get the RecordWriter for the given task.

Specified by:

getRecordWriter in class org.apache.hadoop.mapreduce.OutputFormat<java.lang.Object,java.util.List<java.nio.ByteBuffer>>

Parameters:

context - the information about the current task.

Returns:

a RecordWriter to write the output for the job.

Throws:

java.io.IOException

java.lang.InterruptedException

setColumnFamilySchema

public static void setColumnFamilySchema(org.apache.hadoop.conf.Configuration conf,
                                         java.lang.String columnFamily,
                                         java.lang.String schema)

setColumnFamilyInsertStatement

public static void setColumnFamilyInsertStatement(org.apache.hadoop.conf.Configuration conf,
                                                  java.lang.String columnFamily,
                                                  java.lang.String insertStatement)

setStoragePort

public static void setStoragePort(org.apache.hadoop.conf.Configuration conf,
                                  int port)

setSSLStoragePort

public static void setSSLStoragePort(org.apache.hadoop.conf.Configuration conf,
                                     int port)

setInternodeEncryption

public static void setInternodeEncryption(org.apache.hadoop.conf.Configuration conf,
                                          java.lang.String encrypt)

setServerKeystore

public static void setServerKeystore(org.apache.hadoop.conf.Configuration conf,
                                     java.lang.String keystore)

setServerKeystorePassword

public static void setServerKeystorePassword(org.apache.hadoop.conf.Configuration conf,
                                             java.lang.String keystorePass)

setServerTruststore

public static void setServerTruststore(org.apache.hadoop.conf.Configuration conf,
                                       java.lang.String truststore)

setServerTruststorePassword

public static void setServerTruststorePassword(org.apache.hadoop.conf.Configuration conf,
                                               java.lang.String truststorePass)

setServerCipherSuites

public static void setServerCipherSuites(org.apache.hadoop.conf.Configuration conf,
                                         java.lang.String cipherSuites)

getStoragePort

public static int getStoragePort(org.apache.hadoop.conf.Configuration conf)

getSSLStoragePort

public static int getSSLStoragePort(org.apache.hadoop.conf.Configuration conf)

getInternodeEncryption

public static java.lang.String getInternodeEncryption(org.apache.hadoop.conf.Configuration conf)

getServerKeystore

public static java.lang.String getServerKeystore(org.apache.hadoop.conf.Configuration conf)

getServerTruststore

public static java.lang.String getServerTruststore(org.apache.hadoop.conf.Configuration conf)

getServerKeystorePassword

public static java.lang.String getServerKeystorePassword(org.apache.hadoop.conf.Configuration conf)

getServerTruststorePassword

public static java.lang.String getServerTruststorePassword(org.apache.hadoop.conf.Configuration conf)

getServerCipherSuites

public static java.lang.String getServerCipherSuites(org.apache.hadoop.conf.Configuration conf)

getColumnFamilySchema

public static java.lang.String getColumnFamilySchema(org.apache.hadoop.conf.Configuration conf,
                                                     java.lang.String columnFamily)

getColumnFamilyInsertStatement

public static java.lang.String getColumnFamilyInsertStatement(org.apache.hadoop.conf.Configuration conf,
                                                              java.lang.String columnFamily)

setDeleteSourceOnSuccess

public static void setDeleteSourceOnSuccess(org.apache.hadoop.conf.Configuration conf,
                                            boolean deleteSrc)

getDeleteSourceOnSuccess

public static boolean getDeleteSourceOnSuccess(org.apache.hadoop.conf.Configuration conf)