Class/Object

org.apache.spark.sql.hive

SnappyStoreHiveCatalog

Related Docs: object SnappyStoreHiveCatalog | package hive

Permalink

class SnappyStoreHiveCatalog extends SessionCatalog

Catalog using Hive for persistence and adding Snappy extensions like stream/topK tables and returning LogicalPlan to materialize these entities.

Linear Supertypes
SessionCatalog, internal.Logging, AnyRef, Any
Known Subclasses
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SnappyStoreHiveCatalog
  2. SessionCatalog
  3. Logging
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new SnappyStoreHiveCatalog(externalCatalog: SnappyExternalCatalog, snappySession: SnappySession, metadataHive: HiveClient, globalTempViewManager: GlobalTempViewManager, functionResourceLoader: FunctionResourceLoader, functionRegistry: FunctionRegistry, sqlConf: SQLConf, hadoopConf: Configuration)

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. def addDependentRelation(inTable: QualifiedTableName, dependentRelation: QualifiedTableName): Unit

    Permalink
  5. def alterDatabase(dbDefinition: CatalogDatabase): Unit

    Permalink
    Definition Classes
    SessionCatalog
  6. def alterPartitions(tableName: TableIdentifier, parts: Seq[CatalogTablePartition]): Unit

    Permalink
    Definition Classes
    SessionCatalog
  7. def alterTable(tableDefinition: CatalogTable): Unit

    Permalink
    Definition Classes
    SessionCatalog
  8. def alterTempViewDefinition(name: TableIdentifier, viewDefinition: LogicalPlan): Boolean

    Permalink
    Definition Classes
    SessionCatalog
  9. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  10. val cachedDataSourceTables: LoadingCache[QualifiedTableName, (LogicalRelation, CatalogTable, RelationInfo)]

    Permalink

    A cache of Spark SQL data source tables that have been accessed.

    A cache of Spark SQL data source tables that have been accessed.

    Attributes
    protected
  11. val cachedSampleTables: LoadingCache[QualifiedTableName, Seq[(LogicalPlan, String)]]

    Permalink
  12. def clearTempTables(): Unit

    Permalink
    Definition Classes
    SessionCatalog
  13. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  14. def close(): Unit

    Permalink
  15. def compatibleSchema(schema1: StructType, schema2: StructType): Boolean

    Permalink
  16. def createCachedSampleTables: LoadingCache[QualifiedTableName, Seq[(LogicalPlan, String)]]

    Permalink
    Attributes
    protected
  17. def createDatabase(dbDefinition: CatalogDatabase, ignoreIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  18. def createFunction(funcDefinition: CatalogFunction, ignoreIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  19. def createGlobalTempView(name: String, viewDefinition: LogicalPlan, overrideIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  20. def createPartitions(tableName: TableIdentifier, parts: Seq[CatalogTablePartition], ignoreIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  21. def createTable(tableDefinition: CatalogTable, ignoreIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  22. def createTempFunction(name: String, info: ExpressionInfo, funcDefinition: FunctionBuilder, ignoreIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  23. def createTempView(name: String, tableDefinition: LogicalPlan, overrideIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  24. var currentDb: String

    Permalink
    Attributes
    protected
    Definition Classes
    SessionCatalog
  25. var currentSchema: String

    Permalink
    Attributes
    protected
  26. def databaseExists(db: String): Boolean

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  27. def defaultTablePath(tableIdent: TableIdentifier): String

    Permalink
    Definition Classes
    SessionCatalog
  28. def destroyAndRegisterBuiltInFunctions(): Unit

    Permalink

    Test only method

  29. def dropDatabase(db: String, ignoreIfNotExists: Boolean, cascade: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  30. def dropFunction(name: FunctionIdentifier, ignoreIfNotExists: Boolean): Unit

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  31. def dropGlobalTempView(name: String): Boolean

    Permalink
    Definition Classes
    SessionCatalog
  32. def dropPartitions(tableName: TableIdentifier, specs: Seq[TablePartitionSpec], ignoreIfNotExists: Boolean, purge: Boolean, retainData: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  33. def dropTable(name: TableIdentifier, ignoreIfNotExists: Boolean, purge: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  34. def dropTempFunction(name: String, ignoreIfNotExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  35. def dropTempView(name: String): Boolean

    Permalink
    Definition Classes
    SessionCatalog
  36. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  37. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  38. def failFunctionLookup(name: String): Nothing

    Permalink
    Attributes
    protected
    Definition Classes
    SessionCatalog
  39. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  40. def formatDatabaseName(name: String): String

    Permalink

    Format database name, taking into account case sensitivity.

    Format database name, taking into account case sensitivity.

    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  41. def formatTableName(name: String): String

    Permalink

    Format table name, taking into account case sensitivity.

    Format table name, taking into account case sensitivity.

    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  42. def functionExists(name: FunctionIdentifier): Boolean

    Permalink
    Definition Classes
    SessionCatalog
  43. def getCachedCatalogTable(table: QualifiedTableName): Option[CatalogTable]

    Permalink
  44. def getCachedHiveTable(table: QualifiedTableName): LogicalRelation

    Permalink
  45. def getCachedHiveTableProperties(table: QualifiedTableName): Map[String, String]

    Permalink
  46. def getCachedSampledRelations(table: QualifiedTableName): Seq[(LogicalPlan, String)]

    Permalink
  47. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  48. def getCurrentDatabase: String

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  49. def getCurrentSchema: String

    Permalink
  50. def getDataSourceRelations[T](tableTypes: Seq[ExternalTableType], baseTable: Option[String] = None): Seq[T]

    Permalink
  51. def getDataSourceTables(tableTypes: Seq[ExternalTableType], baseTable: Option[String] = None): Seq[QualifiedTableName]

    Permalink
  52. def getDatabaseMetadata(db: String): CatalogDatabase

    Permalink
    Definition Classes
    SessionCatalog
  53. def getDefaultDBPath(db: String): String

    Permalink
    Definition Classes
    SessionCatalog
  54. def getFunctionMetadata(name: FunctionIdentifier): CatalogFunction

    Permalink
    Definition Classes
    SessionCatalog
  55. def getGlobalTempView(name: String): Option[LogicalPlan]

    Permalink
    Definition Classes
    SessionCatalog
  56. def getPartition(tableName: TableIdentifier, spec: TablePartitionSpec): CatalogTablePartition

    Permalink
    Definition Classes
    SessionCatalog
  57. def getTableMetadata(name: TableIdentifier): CatalogTable

    Permalink
    Definition Classes
    SessionCatalog
  58. def getTableMetadataOption(name: TableIdentifier): Option[CatalogTable]

    Permalink
    Definition Classes
    SessionCatalog
  59. def getTableOption(qtn: QualifiedTableName): Option[CatalogTable]

    Permalink
  60. def getTableType(relation: BaseRelation): ExternalTableType

    Permalink
  61. def getTables(db: Option[String]): Seq[(String, Boolean)]

    Permalink
  62. def getTempView(name: String): Option[LogicalPlan]

    Permalink
    Definition Classes
    SessionCatalog
  63. def getTempViewOrPermanentTableMetadata(name: TableIdentifier): CatalogTable

    Permalink
    Definition Classes
    SessionCatalog
  64. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  65. def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  66. def invalidateAll(): Unit

    Permalink
  67. def invalidateTable(tableIdent: QualifiedTableName): Unit

    Permalink
  68. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  69. def isTemporaryFunction(name: FunctionIdentifier): Boolean

    Permalink
    Definition Classes
    SessionCatalog
  70. def isTemporaryTable(tableIdent: QualifiedTableName): Boolean

    Permalink

    Return whether a table with the specified name is a temporary table.

  71. def isTemporaryTable(name: TableIdentifier): Boolean

    Permalink
    Definition Classes
    SessionCatalog
  72. def isTraceEnabled(): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  73. def listDatabases(pattern: String): Seq[String]

    Permalink
    Definition Classes
    SessionCatalog
  74. def listDatabases(): Seq[String]

    Permalink
    Definition Classes
    SessionCatalog
  75. def listFunctions(db: String, pattern: String): Seq[(FunctionIdentifier, String)]

    Permalink
    Definition Classes
    SessionCatalog
  76. def listFunctions(db: String): Seq[(FunctionIdentifier, String)]

    Permalink
    Definition Classes
    SessionCatalog
  77. def listPartitionNames(tableName: TableIdentifier, partialSpec: Option[TablePartitionSpec]): Seq[String]

    Permalink
    Definition Classes
    SessionCatalog
  78. def listPartitions(tableName: TableIdentifier, partialSpec: Option[TablePartitionSpec]): Seq[CatalogTablePartition]

    Permalink
    Definition Classes
    SessionCatalog
  79. def listPartitionsByFilter(tableName: TableIdentifier, predicates: Seq[Expression]): Seq[CatalogTablePartition]

    Permalink
    Definition Classes
    SessionCatalog
  80. def listTables(db: String, pattern: String): Seq[TableIdentifier]

    Permalink
    Definition Classes
    SessionCatalog
  81. def listTables(db: String): Seq[TableIdentifier]

    Permalink
    Definition Classes
    SessionCatalog
  82. def loadFunctionResources(resources: Seq[FunctionResource]): Unit

    Permalink
    Definition Classes
    SessionCatalog
  83. def loadPartition(name: TableIdentifier, loadPath: String, spec: TablePartitionSpec, isOverwrite: Boolean, holdDDLTime: Boolean, inheritTableSpecs: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  84. def loadTable(name: TableIdentifier, loadPath: String, isOverwrite: Boolean, holdDDLTime: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  85. def log: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  86. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  87. def logDebug(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  88. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  89. def logError(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  90. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  91. def logInfo(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  92. def logName: String

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  93. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  94. def logTrace(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  95. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  96. def logWarning(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  97. def lookupFunction(name: FunctionIdentifier, children: Seq[Expression]): Expression

    Permalink

    Return an Expression that represents the specified function, assuming it exists.

    Return an Expression that represents the specified function, assuming it exists.

    For a temporary function or a permanent function that has been loaded, this method will simply lookup the function through the FunctionRegistry and create an expression based on the builder.

    For a permanent function that has not been loaded, we will first fetch its metadata from the underlying external catalog. Then, we will load all resources associated with this function (i.e. jars and files). Finally, we create a function builder based on the function class and put the builder into the FunctionRegistry. The name of this function in the FunctionRegistry will be databaseName.functionName.

    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  98. def lookupFunctionInfo(name: FunctionIdentifier): ExpressionInfo

    Permalink

    Look up the ExpressionInfo associated with the specified function, assuming it exists.

    Look up the ExpressionInfo associated with the specified function, assuming it exists.

    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  99. def lookupRelation(tableIdent: TableIdentifier, alias: Option[String]): LogicalPlan

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  100. final def lookupRelation(tableIdent: QualifiedTableName): LogicalPlan

    Permalink
  101. final def lookupRelationOption(tableIdent: QualifiedTableName): Option[LogicalPlan]

    Permalink
  102. def makeFunctionBuilder(funcName: String, className: String): FunctionBuilder

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  103. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  104. def newQualifiedTableName(tableIdent: String): QualifiedTableName

    Permalink
  105. def newQualifiedTableName(tableIdent: TableIdentifier): QualifiedTableName

    Permalink
  106. def newQualifiedTempTableName(tableIdent: String): QualifiedTableName

    Permalink
  107. def normalizeSchema(schema: StructType): StructType

    Permalink
  108. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  109. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  110. def refreshTable(tableIdent: TableIdentifier): Unit

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  111. def registerDataSourceTable(tableIdent: QualifiedTableName, userSpecifiedSchema: Option[StructType], partitionColumns: Array[String], provider: String, options: Map[String, String], relation: Option[BaseRelation]): Unit

    Permalink

    Creates a data source table (a table created with USING clause) in Hive's meta-store.

  112. def registerRelationDestroy(): Unit

    Permalink
    Attributes
    protected
  113. def registerTable(tableName: QualifiedTableName, plan: LogicalPlan): Unit

    Permalink
  114. var relationDestroyVersion: Int

    Permalink
  115. def removeDependentRelation(table: QualifiedTableName, dependentRelation: QualifiedTableName): Unit

    Permalink
  116. def removeDependentRelationFromHive(table: QualifiedTableName, dependentRelation: QualifiedTableName): Unit

    Permalink
  117. def renamePartitions(tableName: TableIdentifier, specs: Seq[TablePartitionSpec], newSpecs: Seq[TablePartitionSpec]): Unit

    Permalink
    Definition Classes
    SessionCatalog
  118. def renameTable(oldName: TableIdentifier, newName: TableIdentifier): Unit

    Permalink
    Definition Classes
    SessionCatalog
  119. def reset(): Unit

    Permalink

    Drop all existing databases (except "default"), tables, partitions and functions, and set the current database to "default".

    Drop all existing databases (except "default"), tables, partitions and functions, and set the current database to "default". This method will only remove tables from hive catalog.Don't use this method if you want to delete physical tables This is mainly used for tests.

    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  120. def setCurrentDatabase(db: String): Unit

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  121. final def setSchema(schema: String): Unit

    Permalink
  122. val snappySession: SnappySession

    Permalink
  123. val sparkConf: SparkConf

    Permalink
  124. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  125. def tableExists(tableName: QualifiedTableName): Boolean

    Permalink
  126. def tableExists(tableIdentifier: String): Boolean

    Permalink
  127. def tableExists(tableIdentifier: TableIdentifier): Boolean

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  128. val tempTables: HashMap[String, LogicalPlan]

    Permalink
    Attributes
    protected
    Definition Classes
    SessionCatalog
  129. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  130. def unregisterAllTables(): Unit

    Permalink
  131. def unregisterDataSourceTable(tableIdent: QualifiedTableName, relation: Option[BaseRelation]): Unit

    Permalink

    Drops a data source table from Hive's meta-store.

  132. def unregisterGlobalView(tableIdent: QualifiedTableName): Boolean

    Permalink
  133. def unregisterTable(tableIdent: QualifiedTableName): Unit

    Permalink
  134. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  135. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  136. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  137. def withHiveExceptionHandling[T](function: ⇒ T): T

    Permalink

Inherited from SessionCatalog

Inherited from internal.Logging

Inherited from AnyRef

Inherited from Any

Ungrouped