cassandra

Type Members

1. class CassandraSQLContext extends SQLContext

   

   Allows to execute SQL queries against Cassandra and access results as SchemaRDD collections.

   Allows to execute SQL queries against Cassandra and access results as SchemaRDD collections. Predicate pushdown to Cassandra is supported.

   Example:

   import com.datastax.spark.connector._
   
   val sparkMasterHost = "127.0.0.1"
   val cassandraHost = "127.0.0.1"
   
   // Tell Spark the address of one Cassandra node:
   val conf = new SparkConf(true).set("spark.cassandra.connection.host", cassandraHost)
   
   // Connect to the Spark cluster:
   val sc = new SparkContext("spark://" + sparkMasterHost + ":7077", "example", conf)
   
   // Create CassandraSQLContext:
   val cc = new CassandraSQLContext(sc)
   
   // Execute SQL query:
   val rdd = cc.sql("SELECT * FROM keyspace.table ...")

2. final class CassandraSQLRow extends GettableData with catalyst.expressions.Row with Serializable

   

3. case class CassandraSourceOptions(pushdown: Boolean = true, cassandraConfs: Map[String, String] = Map.empty) extends Product with Serializable

   

   Store data source options

4. case class CassandraTableScan(attributes: Seq[Attribute], relation: CassandraRelation, pushdownPred: Seq[Expression])(context: CassandraSQLContext) extends SparkPlan with LeafNode with Logging with Product with Serializable

   

   Annotations

   @DeveloperApi()

5. class DefaultSource extends RelationProvider with SchemaRelationProvider with CreatableRelationProvider with Logging

   

   Cassandra data source extends RelationProvider, SchemaRelationProvider and CreatableRelationProvider.

   Cassandra data source extends RelationProvider, SchemaRelationProvider and CreatableRelationProvider. It's used internally by Spark SQL to create Relation for a table which specifies the Cassandra data source e.g.

   CREATE TEMPORARY TABLE tmpTable USING org.apache.spark.sql.cassandra OPTIONS ( table "table", keyspace "keyspace", cluster "test_cluster", pushdown "true", spark_cassandra_input_page_row_size "10", spark_cassandra_output_consistency_level "ONE", spark_cassandra_connection_timeout_ms "1000" )

6. case class InsertIntoCassandraTable(cassandraRelation: CassandraRelation, childPlan: SparkPlan, overwrite: Boolean)(cc: CassandraSQLContext) extends SparkPlan with UnaryNode with Product with Serializable

   

   Annotations

   @DeveloperApi()

7. trait PredicateOps[Predicate] extends AnyRef

   

   A unified API for predicates, used by PredicatePushDown.

   A unified API for predicates, used by PredicatePushDown.

   Keeps all the Spark-specific stuff out of PredicatePushDown It is also easy to plug-in custom predicate implementations for unit-testing.

8. class PredicatePushDown[Predicate] extends AnyRef

   

   Determines which filter predicates can be pushed down to Cassandra.

   Determines which filter predicates can be pushed down to Cassandra.

   1. Only push down no-partition key column predicates with =, >, <, >=, <= predicate 2. Only push down primary key column predicates with = or IN predicate. 3. If there are regular columns in the pushdown predicates, they should have at least one EQ expression on an indexed column and no IN predicates. 4. All partition column predicates must be included in the predicates to be pushed down, only the last part of the partition key can be an IN predicate. For each partition column, only one predicate is allowed. 5. For cluster column predicates, only last predicate can be non-EQ predicate including IN predicate, and preceding column predicates must be EQ predicates. If there is only one cluster column predicate, the predicates could be any non-IN predicate. 6. There is no pushdown predicates if there is any OR condition or NOT IN condition. 7. We're not allowed to push down multiple predicates for the same column if any of them is equality or IN predicate.

   The list of predicates to be pushed down is available in predicatesToPushDown property. The list of predicates that cannot be pushed down is available in predicatesToPreserve property.

9. case class TableRef(table: String, keyspace: String, cluster: Option[String] = None) extends Product with Serializable

   

   Store table name, keyspace name and option cluster name, keyspace is equivalent to database