connector

The root package of Cassandra connector for Apache Spark. Offers handy implicit conversions that add Cassandra-specific methods to SparkContext and RDD.

Call cassandraTable method on the SparkContext object to create a CassandraRDD exposing Cassandra tables as Spark RDDs.

Call com.datastax.spark.connector.RDDFunctions saveToCassandra function on any RDD to save distributed collection to a Cassandra table.

Example:

CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1 };
CREATE TABLE test.words (word text PRIMARY KEY, count int);
INSERT INTO test.words(word, count) VALUES ("and", 50);

import com.datastax.spark.connector._

val sparkMasterHost = "127.0.0.1"
val cassandraHost = "127.0.0.1"
val keyspace = "test"
val table = "words"

// Tell Spark the address of one Cassandra node:
val conf = new SparkConf(true).set("spark.cassandra.connection.host", cassandraHost)

// Connect to the Spark cluster:
val sc = new SparkContext("spark://" + sparkMasterHost + ":7077", "example", conf)

// Read the table and print its contents:
val rdd = sc.cassandraTable(keyspace, table)
rdd.toArray().foreach(println)

// Write two rows to the table:
val col = sc.parallelize(Seq(("of", 1200), ("the", "863")))
col.saveToCassandra(keyspace, table)

sc.stop()

Linear Supertypes

AnyRef, Any

Type Members

class CassandraRow extends Serializable

Represents a single row fetched from Cassandra.
class ColumnNotFoundException extends Exception

Thrown when the requested column does not exist in the result set.
sealed trait ColumnSelector extends AnyRef
class RDDFunctions[T] extends WritableToCassandra[T] with Serializable

Provides Cassandra-specific methods on RDD
case class SomeColumns(columns: String*) extends ColumnSelector with Product with Serializable
class SparkContextFunctions extends AnyRef

Provides Cassandra-specific methods on SparkContext

Value Members

object AllColumns extends ColumnSelector with Product with Serializable
object CassandraRow extends Serializable
object SomeColumns extends Serializable
package cql

Contains a cql.CassandraConnector object which is used to connect to a Cassandra cluster and to send CQL statements to it.
package mapper

Provides machinery for mapping Cassandra tables to user defined Scala classes or tuples.
package rdd

Contains com.datastax.spark.connector.rdd.CassandraRDD class that is the main entry point for analyzing Cassandra data from Spark.
package streaming
implicit def toRDDFunctions[T](rdd: RDD[T])(implicit arg0: ClassTag[T]): RDDFunctions[T]
implicit def toSparkContextFunctions(sc: SparkContext): SparkContextFunctions
package types

Offers type conversion magic, so you can receive Cassandra column values in a form you like the most.
package util

Useful stuff that didn't fit elsewhere.
package writer

Contains components for writing RDDs to Cassandra

package connector

Type Members

class CassandraRow extends Serializable

class ColumnNotFoundException extends Exception

sealed trait ColumnSelector extends AnyRef

class RDDFunctions[T] extends WritableToCassandra[T] with Serializable

case class SomeColumns(columns: String*) extends ColumnSelector with Product with Serializable

class SparkContextFunctions extends AnyRef

Value Members

object AllColumns extends ColumnSelector with Product with Serializable

object CassandraRow extends Serializable

object SomeColumns extends Serializable

package cql

package mapper

package rdd

package streaming

implicit def toRDDFunctions[T](rdd: RDD[T])(implicit arg0: ClassTag[T]): RDDFunctions[T]

implicit def toSparkContextFunctions(sc: SparkContext): SparkContextFunctions

package types

package util

package writer

Inherited from AnyRef

Inherited from Any

Ungrouped