Determines which filter predicates can be pushed down to Cassandra.
Determines which filter predicates can be pushed down to Cassandra.
The list of predicates to be pushed down is available in predicatesToPushDown
property.
The list of predicates that cannot be pushed down is available in predicatesToPreserve
property.
Allows to execute SQL queries against Cassandra and access results as
SchemaRDD
collections.
Allows to execute SQL queries against Cassandra and access results as
SchemaRDD
collections. Predicate pushdown to Cassandra is supported.
Example:
import com.datastax.spark.connector._ val sparkMasterHost = "127.0.0.1" val cassandraHost = "127.0.0.1" // Tell Spark the address of one Cassandra node: val conf = new SparkConf(true).set("spark.cassandra.connection.host", cassandraHost) // Connect to the Spark cluster: val sc = new SparkContext("spark://" + sparkMasterHost + ":7077", "example", conf) // Create CassandraSQLContext: val cc = new CassandraSQLContext(sc) // Execute SQL query: val rdd = cc.sql("SELECT * FROM keyspace.table ...")
Store data source options
Cassandra data source extends RelationProvider, SchemaRelationProvider and CreatableRelationProvider.
Cassandra data source extends RelationProvider, SchemaRelationProvider and CreatableRelationProvider. It's used internally by Spark SQL to create Relation for a table which specifies the Cassandra data source e.g.
CREATE TEMPORARY TABLE tmpTable USING org.apache.spark.sql.cassandra OPTIONS ( table "table", keyspace "keyspace", cluster "test_cluster", pushdown "true", spark_cassandra_input_page_row_size "10", spark_cassandra_output_consistency_level "ONE", spark_cassandra_connection_timeout_ms "1000" )
A unified API for predicates, used by BasicCassandraPredicatePushDown.
A unified API for predicates, used by BasicCassandraPredicatePushDown.
Keeps all the Spark-specific stuff out of BasicCassandraPredicatePushDown
It is also easy to plug-in custom predicate implementations for unit-testing.
Store table name, keyspace name and option cluster name, keyspace is equivalent to database
A data frame format used to access Cassandra through Connector
Convert Cassandra data type to Catalyst data type
Provides PredicateOps
adapters for Expression and Filter classes
Returns a map of options which configure the path to Cassandra table as well as whether pushdown is enabled or not