sources

Type Members

1. trait CastDouble extends AnyRef

   

   Optimized cast for a column in a row to double.

2. trait CastLongTime extends AnyRef

   

   Cast a given column in a schema to epoch time in long milliseconds.

3. case class ConnectionProperties(url: String, driver: String, dialect: JdbcDialect, poolProps: Map[String, String], connProps: Properties, executorConnProps: Properties, hikariCP: Boolean) extends Product with Serializable

   

4. trait DeletableRelation extends AnyRef

   

   Annotations

   @DeveloperApi()

5. trait DependentRelation extends BaseRelation

   

   A relation having a parent-child relationship with a base relation.

   A relation having a parent-child relationship with a base relation.

   Annotations

   @DeveloperApi()

6. trait DestroyRelation extends AnyRef

   

   Annotations

   @DeveloperApi()

7. trait ExternalSchemaRelationProvider extends AnyRef

   

   ::DeveloperApi:: Implemented by objects that produce relations for a specific kind of data source with a given schema.

   ::DeveloperApi:: Implemented by objects that produce relations for a specific kind of data source with a given schema. When Spark SQL is given a DDL operation with a USING clause specified (to specify the implemented SchemaRelationProvider) and a user defined schema, this interface is used to pass in the parameters specified by a user.

   Users may specify the fully qualified class name of a given data source. When that class is not found Spark SQL will append the class name DefaultSource to the path, allowing for less verbose invocation. For example, 'org.apache.spark.sql.json' would resolve to the data source 'org.apache.spark.sql.json.DefaultSource'.

   A new instance of this class with be instantiated each time a DDL call is made.

   The difference between a SchemaRelationProvider and an ExternalSchemaRelationProvider is that latter accepts schema and other clauses in DDL string and passes over to the backend as is, while the schema specified for former is parsed by Spark SQL. A relation provider can inherit both SchemaRelationProvider and ExternalSchemaRelationProvider if it can support both Spark SQL schema and backend-specific schema.

   Annotations

   @DeveloperApi()

8. trait IndexableRelation extends AnyRef

   

   Annotations

   @DeveloperApi()

9. abstract class JdbcExtendedDialect extends JdbcDialect

   

   Some extensions to JdbcDialect used by Snappy implementation.

10. abstract class MutableRelationProvider extends ExternalSchemaRelationProvider with SchemaRelationProvider with RelationProvider with CreatableRelationProvider

    

11. trait ParentRelation extends BaseRelation

    

    A relation having a parent-child relationship with one or more DependentRelations as children.

    A relation having a parent-child relationship with one or more DependentRelations as children.

    Annotations

    @DeveloperApi()

12. trait PrunedUnsafeFilteredScan extends AnyRef

    

    ::DeveloperApi:: A BaseRelation that can eliminate unneeded columns and filter using selected predicates before producing an RDD containing all matching tuples as Unsafe Row objects.

    ::DeveloperApi:: A BaseRelation that can eliminate unneeded columns and filter using selected predicates before producing an RDD containing all matching tuples as Unsafe Row objects.

    The actual filter should be the conjunction of all filters, i.e. they should be "and" together.

    The pushed down filters are currently purely an optimization as they will all be evaluated again. This means it is safe to use them with methods that produce false positives such as filtering partitions based on a bloom filter.

    Annotations

    @DeveloperApi()

    Since

    1.3.0

13. trait RowInsertableRelation extends SingleRowInsertableRelation

    

    Annotations

    @DeveloperApi()

14. trait RowPutRelation extends SingleRowInsertableRelation

    

15. trait SamplingRelation extends BaseRelation with DependentRelation with SchemaInsertableRelation

    

    Annotations

    @DeveloperApi()

16. trait SchemaInsertableRelation extends InsertableRelation

    

    ::DeveloperApi

    ::DeveloperApi

    An extension to InsertableRelation that allows for data to be inserted (possibily having different schema) into the target relation after comparing against the result of insertSchema.

    Annotations

    @DeveloperApi()

17. trait SingleRowInsertableRelation extends AnyRef

    

    Annotations

    @DeveloperApi()

18. final class StatCounter extends StatVarianceCounter with Serializable

    

19. trait StatVarianceCounter extends Serializable

    

    A class for tracking the statistics of a set of numbers (count, mean and variance) in a numerically robust way.

    A class for tracking the statistics of a set of numbers (count, mean and variance) in a numerically robust way. Includes support for merging two StatVarianceCounters.

    Taken from Spark's StatCounter implementation removing max and min.

20. trait UpdatableRelation extends SingleRowInsertableRelation

    

    Annotations

    @DeveloperApi()