com.datastax.spark.connector

Class SparkContextJavaFunctions

java.lang.Object

com.datastax.spark.connector.SparkContextJavaFunctions

Direct Known Subclasses:

StreamingContextJavaFunctions

public class SparkContextJavaFunctions
extends java.lang.Object

Field Summary

Fields

Modifier and Type	Field and Description
org.apache.spark.SparkContext	sparkContext

Method Summary

Methods

Modifier and Type	Method and Description
CassandraJavaRDD<com.datastax.spark.connector.CassandraRow>	cassandraTable(java.lang.String keyspace, java.lang.String table) Returns a view of a Cassandra table as a CassandraJavaRDD.
<T extends java.io.Serializable> CassandraJavaRDD<T>	cassandraTable(java.lang.String keyspace, java.lang.String table, java.lang.Class<T> targetClass) Returns a view of a Cassandra table as a CassandraJavaRDD.
<T extends java.io.Serializable> CassandraJavaRDD<T>	cassandraTable(java.lang.String keyspace, java.lang.String table, java.lang.Class<T> targetClass, java.util.Map<java.lang.String,java.lang.String> columnNameOverride) Returns a view of a Cassandra table as a CassandraJavaRDD.
<T extends java.io.Serializable> CassandraJavaRDD<T>	cassandraTable(java.lang.String keyspace, java.lang.String table, com.datastax.spark.connector.mapper.ColumnMapper<T> columnMapper, java.lang.Class<T> targetClass) Returns a view of a Cassandra table as a CassandraJavaRDD.
<T extends java.io.Serializable> CassandraJavaRDD<T>	cassandraTable(java.lang.String keyspace, java.lang.String table, com.datastax.spark.connector.rdd.reader.RowReaderFactory<T> rowReaderFactory, java.lang.Class<T> targetClass) Returns a view of a Cassandra table as a CassandraJavaRDD.
<T extends java.io.Serializable> CassandraJavaRDD<T>	toJavaRDD(com.datastax.spark.connector.rdd.CassandraRDD<T> rdd, java.lang.Class<T> targetClass) Converts CassandraRDD into CassandraJavaRDD.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

sparkContext

public final org.apache.spark.SparkContext sparkContext

Method Detail

toJavaRDD

public <T extends java.io.Serializable> CassandraJavaRDD<T> toJavaRDD(com.datastax.spark.connector.rdd.CassandraRDD<T> rdd,
                                                             java.lang.Class<T> targetClass)

Converts CassandraRDD into CassandraJavaRDD.

cassandraTable

public <T extends java.io.Serializable> CassandraJavaRDD<T> cassandraTable(java.lang.String keyspace,
                                                                  java.lang.String table,
                                                                  com.datastax.spark.connector.rdd.reader.RowReaderFactory<T> rowReaderFactory,
                                                                  java.lang.Class<T> targetClass)

Returns a view of a Cassandra table as a CassandraJavaRDD. With this method, a RowReader created by the provided RowReaderFactory is used to produce object of targetClass for each fetched row.

See Also:

cassandraTable(String, String)

cassandraTable

public CassandraJavaRDD<com.datastax.spark.connector.CassandraRow> cassandraTable(java.lang.String keyspace,
                                                                         java.lang.String table)

Returns a view of a Cassandra table as a CassandraJavaRDD. With this method, each row is converted to a CassandraRow object.

Example:

 CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1 };
 CREATE TABLE test.words (word text PRIMARY KEY, count int);
 INSERT INTO test.words (word, count) VALUES ('foo', 20);
 INSERT INTO test.words (word, count) VALUES ('bar', 20);
 ...

 // Obtaining RDD of CassandraRow objects:
 CassandraJavaRDD rdd = CassandraJavaUtil.javaFunctions(jsc).cassandraTable("test", "words");
 rdd.first().getString("word");     // foo
 rdd.first().getInt("count");       // 20
 

cassandraTable

public <T extends java.io.Serializable> CassandraJavaRDD<T> cassandraTable(java.lang.String keyspace,
                                                                  java.lang.String table,
                                                                  com.datastax.spark.connector.mapper.ColumnMapper<T> columnMapper,
                                                                  java.lang.Class<T> targetClass)

Returns a view of a Cassandra table as a CassandraJavaRDD. With this method, each row is converted into the instance of targetClass with use of the provided custom ColumnMapper. By default, JavaBeanColumnMapper is used to map object properties into column names.

See Also:

cassandraTable(String, String, Class)

cassandraTable

public <T extends java.io.Serializable> CassandraJavaRDD<T> cassandraTable(java.lang.String keyspace,
                                                                  java.lang.String table,
                                                                  java.lang.Class<T> targetClass)

Returns a view of a Cassandra table as a CassandraJavaRDD. With this method, each row is converted into the instance of targetClass.

Example:

 CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1 };
 CREATE TABLE test.words (word text PRIMARY KEY, count int);
 INSERT INTO test.words (word, count) VALUES ('foo', 20);
 INSERT INTO test.words (word, count) VALUES ('bar', 20);

 // all the Java classes which you want to use with Spark should be serializable
 public class WordCount implements Serializable {
     private String word;
     private Integer count;
     private String other;

     // it is very important to implement no-args constructor in a Java class which is to be used
     // with the Connector
     public WordCount() {
     }

     // other constructors, getters, etc.

     public void setWord(String word) {
         this.word = word;
     }

     public void setCount(Integer count) {
         this.count = count;
     }

     public void setOther(String other) {
         this.other = other;
     }
 }

 // Obtaining RDD of targetClass objects:
 CassandraJavaRDD rdd = CassandraJavaUtil.javaFunctions(jsc)
      .cassandraTable("test", "words", WordCount.class);
 rdd.first().getWord();     // foo
 rdd.first().getCount();    // 20
 

cassandraTable

public <T extends java.io.Serializable> CassandraJavaRDD<T> cassandraTable(java.lang.String keyspace,
                                                                  java.lang.String table,
                                                                  java.lang.Class<T> targetClass,
                                                                  java.util.Map<java.lang.String,java.lang.String> columnNameOverride)

Returns a view of a Cassandra table as a CassandraJavaRDD. With this method, each row is converted into the instance of targetClass. It works just like cassandraTable(String, String, Class) but it additionally allows the specification of a custom property to column name mappings.

See Also:

cassandraTable(String, String, Class, java.util.Map)