package cql
Contains a cql.CassandraConnector object which is used to connect
to a Cassandra cluster and to send CQL statements to it. CassandraConnector
provides a Scala-idiomatic way of working with Cluster
and Session
object
and takes care of connection pooling and proper resource disposal.
- Alphabetic
- By Inheritance
- cql
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Type Members
-
trait
AuthConf extends Serializable
Stores credentials used to authenticate to a Cassandra cluster and uses them to configure a Cassandra connection.
Stores credentials used to authenticate to a Cassandra cluster and uses them to configure a Cassandra connection. This driver provides implementations NoAuthConf for no authentication and PasswordAuthConf for password authentication. Other authentication configurators can be plugged in by setting
cassandra.authentication.conf.factory.class
option. See AuthConfFactory. -
trait
AuthConfFactory extends AnyRef
Obtains authentication configuration by reading SparkConf object.
-
trait
CassandraConnectionFactory extends Serializable
Creates both native and Thrift connections to Cassandra.
Creates both native and Thrift connections to Cassandra. The connector provides a DefaultConnectionFactory. Other factories can be plugged in by setting
spark.cassandra.connection.factory
option. -
class
CassandraConnector extends Serializable with Logging
Provides and manages connections to Cassandra.
Provides and manages connections to Cassandra.
A
CassandraConnector
instance is serializable and can be safely sent over network, because it automatically reestablishes the connection to the same cluster after deserialization. Internally it saves a list of all nodes in the cluster, so a connection can be established even if the host given in the initial config is down.Multiple
CassandraConnector
s in the same JVM connected to the same Cassandra cluster will share a single underlyingCluster
object.CassandraConnector
will close the underlyingCluster
object automatically whenever it is not used i.e. noSession
orCluster
is open for longer thanspark.cassandra.connection.keep_alive_ms
property value.A
CassandraConnector
object is configured from CassandraConnectorConf object which can be either given explicitly or automatically configured from SparkConf. The connection options are:spark.cassandra.connection.host
: contact points to connect to the Cassandra cluster, defaults to spark master hostspark.cassandra.connection.port
: Cassandra native port, defaults to 9042spark.cassandra.connection.factory
: name of a Scala module or class implementing CassandraConnectionFactory that allows to plugin custom code for connecting to Cassandraspark.cassandra.connection.keep_alive_ms
: how long to keep unused connection before closing it (default 250 ms)spark.cassandra.connection.timeout_ms
: how long to wait for connection to the Cassandra cluster (default 5 s)spark.cassandra.connection.reconnection_delay_ms.min
: initial delay determining how often to try to reconnect to a dead node (default 1 s)spark.cassandra.connection.reconnection_delay_ms.max
: final delay determining how often to try to reconnect to a dead node (default 60 s)spark.cassandra.auth.username
: login for password authenticationspark.cassandra.auth.password
: password for password authenticationspark.cassandra.auth.conf.factory
: name of a Scala module or class implementing AuthConfFactory that allows to plugin custom authentication configurationspark.cassandra.query.retry.count
: how many times to reattempt a failed query (default 10)spark.cassandra.read.timeout_ms
: maximum period of time to wait for a read to returnspark.cassandra.connection.ssl.enabled
: enable secure connection to Cassandra clusterspark.cassandra.connection.ssl.trustStore.path
: path for the trust store being usedspark.cassandra.connection.ssl.trustStore.password
: trust store passwordspark.cassandra.connection.ssl.trustStore.type
: trust store type (default JKS)spark.cassandra.connection.ssl.protocol
: SSL protocol (default TLS)spark.cassandra.connection.ssl.enabledAlgorithms
: SSL cipher suites (default TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA)
-
case class
CassandraConnectorConf(hosts: Set[InetAddress], port: Int = ..., authConf: AuthConf = NoAuthConf, localDC: Option[String] = None, keepAliveMillis: Int = ..., minReconnectionDelayMillis: Int = ..., maxReconnectionDelayMillis: Int = ..., maxConnectionsPerExecutor: Option[Int] = ..., compression: Compression = ..., queryRetryCount: Int = ..., connectTimeoutMillis: Int = ..., readTimeoutMillis: Int = ..., connectionFactory: CassandraConnectionFactory = DefaultConnectionFactory, cassandraSSLConf: CassandraSSLConf = ...) extends Product with Serializable
Stores configuration of a connection to Cassandra.
Stores configuration of a connection to Cassandra. Provides information about cluster nodes, ports and optional credentials for authentication.
- case class ClusteringColumn(index: Int) extends ColumnRole with Product with Serializable
-
case class
ColumnDef(columnName: String, columnRole: ColumnRole, columnType: ColumnType[_]) extends FieldDef with Product with Serializable
A Cassandra column metadata that can be serialized.
- sealed trait ColumnRole extends AnyRef
- class DefaultScanner extends Scanner
-
trait
FieldDef extends Serializable
Abstract column / field definition.
Abstract column / field definition. Common to tables and user-defined types
-
case class
IndexDef(className: String, target: String, indexName: String, options: Map[String, String]) extends Serializable with Product
Cassandra Index Metadata that can be serialized *
-
case class
KeyspaceDef(keyspaceName: String, tables: Set[TableDef]) extends Product with Serializable
A Cassandra keyspace metadata that can be serialized.
-
class
LocalNodeFirstLoadBalancingPolicy extends LoadBalancingPolicy with Logging
Selects local node first and then nodes in local DC in random order.
Selects local node first and then nodes in local DC in random order. Never selects nodes from other DCs. For writes, if a statement has a routing key set, this LBP is token aware - it prefers the nodes which are replicas of the computed token to the other nodes.
-
class
MultipleRetryPolicy extends RetryPolicy
Always retries with the same CL (null forces the original statement CL see SPARKC-494), constant number of times, regardless of circumstances
Always retries with the same CL (null forces the original statement CL see SPARKC-494), constant number of times, regardless of circumstances
Retries indefinitely if maxRetryCount is -1
-
case class
PasswordAuthConf(user: String, password: String) extends AuthConf with Product with Serializable
Performs plain-text password authentication.
Performs plain-text password authentication. Use with
PasswordAuthenticator
in Cassandra. -
class
RefCountMap[T] extends AnyRef
Atomically counts references to objects of any type
-
final
class
RefCountedCache[K, V] extends AnyRef
A lockless cache that caches values for multiple users and destroys them once all users release them.
A lockless cache that caches values for multiple users and destroys them once all users release them. One value can be associated with many keys. Useful for sharing a costly resource.
- case class ScanResult(rows: Iterator[Row], metadata: CassandraRowMetadata) extends Product with Serializable
-
trait
Scanner extends AnyRef
Object which will be used in Table Scanning Operations.
Object which will be used in Table Scanning Operations. One Scanner will be created per Spark Partition, it will be created at the beginning of the compute method and Closed at the end of the compute method.
- case class Schema(clusterName: String, keyspaces: Set[KeyspaceDef]) extends Product with Serializable
-
class
SessionProxy extends InvocationHandler
Wraps a
Session
and intercepts:Wraps a
Session
and intercepts:close
method to invokeafterClose
handlerprepare
methods to cachePreparedStatement
objects.
-
trait
StructDef extends Serializable
Cassandra structure that contains columnar information, e.g.
Cassandra structure that contains columnar information, e.g. a table or a user defined type. This trait allows
ColumnMapper
to work on tables and user defined types. Cassandra tables and user defined types are similar in a way data are extracted from them, therefore a common interface to describe their metadata is handy. -
case class
TableDef(keyspaceName: String, tableName: String, partitionKey: Seq[ColumnDef], clusteringColumns: Seq[ColumnDef], regularColumns: Seq[ColumnDef], indexes: Seq[IndexDef] = Seq.empty, isView: Boolean = false) extends StructDef with Product with Serializable
A Cassandra table metadata that can be serialized.
Value Members
-
object
AuthConf extends Serializable
Entry point for obtaining
AuthConf
object from SparkConf, used when establishing connections to Cassandra.Entry point for obtaining
AuthConf
object from SparkConf, used when establishing connections to Cassandra. The actualAuthConf
creation is delegated to the AuthConfFactory pointed byspark.cassandra.auth.conf.factory
property. - object AuthConfFactory
-
object
CassandraConnectionFactory extends Serializable
Entry point for obtaining
CassandraConnectionFactory
object from SparkConf, used when establishing connections to Cassandra. - object CassandraConnector extends Logging with Serializable
-
object
CassandraConnectorConf extends Logging with Serializable
A factory for CassandraConnectorConf objects.
A factory for CassandraConnectorConf objects. Allows for manually setting connection properties or reading them from SparkConf object. By embedding connection information in SparkConf, SparkContext can offer Cassandra specific methods which require establishing connections to a Cassandra cluster.
- object ColumnDef extends Serializable
-
object
DefaultAuthConfFactory extends AuthConfFactory
Default
AuthConfFactory
that supports no authentication or password authentication.Default
AuthConfFactory
that supports no authentication or password authentication. Password authentication is enabled when bothspark.cassandra.auth.username
andspark.cassandra.auth.password
options are present in SparkConf. -
object
DefaultConnectionFactory extends CassandraConnectionFactory
Performs no authentication.
Performs no authentication. Use with
AllowAllAuthenticator
in Cassandra. - object LocalNodeFirstLoadBalancingPolicy
-
object
NoAuthConf extends AuthConf with Product with Serializable
Performs no authentication.
Performs no authentication. Use with
AllowAllAuthenticator
in Cassandra. - object PartitionKeyColumn extends ColumnRole with Product with Serializable
-
object
PreparedStatementCache extends Logging
Caches prepared statements so they are not prepared multiple times by different threads.
- object RegularColumn extends ColumnRole with Product with Serializable
- object Schema extends Logging with Serializable
- object SessionProxy extends Logging
- object StaticColumn extends ColumnRole with Product with Serializable
- object TableDef extends Serializable
Cassandra connector for Apache Spark. See documentation of package com.datastax.spark.connector.