impl

Type Members

1. class BaseColumnFormatRelation extends JDBCAppendableRelation with PartitionedDataSourceScan with RowInsertableRelation

   

   This class acts as a DataSource provider for column format tables provided Snappy.

   This class acts as a DataSource provider for column format tables provided Snappy. It uses GemFireXD as actual datastore to physically locate the tables. Column tables can be used for storing data in columnar compressed format. A example usage is given below.

   val data = Seq(Seq(1, 2, 3), Seq(7, 8, 9), Seq(9, 2, 3), Seq(4, 2, 3), Seq(5, 6, 7)) val rdd = sc.parallelize(data, data.length).map(s => new Data(s(0), s(1), s(2))) val dataDF = snc.createDataFrame(rdd) snc.createTable(tableName, "column", dataDF.schema, props) dataDF.write.insertInto(tableName)

   This provider scans underlying tables in parallel and is aware of the data partition. It does not introduces a shuffle if simple table query is fired. One can insert a single or multiple rows into this table as well as do a bulk insert by a Spark DataFrame. Bulk insert example is shown above.

2. class ColumnFormatRelation extends BaseColumnFormatRelation with ParentRelation

   

3. class ColumnarStorePartitionedRDD[T] extends RDD[Any]

   

4. final class DefaultSource extends ColumnarRelationProvider

   

5. class IndexColumnFormatRelation extends BaseColumnFormatRelation with DependentRelation

   

   Currently this is same as ColumnFormatRelation but has kept it as a separate class to allow adding of any index specific functionality in future.

6. final class JDBCSourceAsColumnarStore extends JDBCSourceAsStore

   

   Column Store implementation for GemFireXD.

7. class SparkShellCachedBatchRDD[T] extends RDD[CachedBatch]

   

8. class SparkShellRowRDD[T] extends RowFormatScanRDD

   

9. trait StoreCallback extends Serializable