Class/Object

org.apache.spark.sql.hive

SnappyStoreHiveCatalog

Related Docs: object SnappyStoreHiveCatalog | package hive

Permalink

class SnappyStoreHiveCatalog extends SessionCatalog

Catalog using Hive for persistence and adding Snappy extensions like stream/topK tables and returning LogicalPlan to materialize these entities.

Linear Supertypes
SessionCatalog, internal.Logging, AnyRef, Any
Known Subclasses
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SnappyStoreHiveCatalog
  2. SessionCatalog
  3. Logging
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new SnappyStoreHiveCatalog(externalCatalog: SnappyExternalCatalog, snappySession: SnappySession, metadataHive: HiveClient, globalTempViewManager: GlobalTempViewManager, functionResourceLoader: FunctionResourceLoader, functionRegistry: FunctionRegistry, sqlConf: SQLConf, hadoopConf: Configuration)

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. def addDependentRelation(inTable: QualifiedTableName, dependentRelation: QualifiedTableName): Unit

    Permalink
  5. def alterDatabase(dbDefinition: CatalogDatabase): Unit

    Permalink
    Definition Classes
    SessionCatalog
  6. def alterPartitions(tableName: TableIdentifier, parts: Seq[CatalogTablePartition]): Unit

    Permalink
    Definition Classes
    SessionCatalog
  7. def alterTable(tableDefinition: CatalogTable): Unit

    Permalink
    Definition Classes
    SessionCatalog
  8. def alterTempViewDefinition(name: TableIdentifier, viewDefinition: LogicalPlan): Boolean

    Permalink
    Definition Classes
    SessionCatalog
  9. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  10. val cachedDataSourceTables: LoadingCache[QualifiedTableName, (LogicalRelation, CatalogTable, RelationInfo)]

    Permalink

    A cache of Spark SQL data source tables that have been accessed.

    A cache of Spark SQL data source tables that have been accessed.

    Attributes
    protected
  11. val cachedSampleTables: LoadingCache[QualifiedTableName, Seq[(LogicalPlan, String)]]

    Permalink
  12. def clearTempTables(): Unit

    Permalink
    Definition Classes
    SessionCatalog
  13. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  14. def compatibleSchema(schema1: StructType, schema2: StructType): Boolean

    Permalink
  15. def createCachedSampleTables: LoadingCache[QualifiedTableName, Seq[(LogicalPlan, String)]]

    Permalink
    Attributes
    protected
  16. def createDatabase(dbDefinition: CatalogDatabase, ignoreIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  17. def createFunction(funcDefinition: CatalogFunction, ignoreIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  18. def createGlobalTempView(name: String, viewDefinition: LogicalPlan, overrideIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  19. def createPartitions(tableName: TableIdentifier, parts: Seq[CatalogTablePartition], ignoreIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  20. def createTable(tableDefinition: CatalogTable, ignoreIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  21. def createTempFunction(name: String, info: ExpressionInfo, funcDefinition: FunctionBuilder, ignoreIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  22. def createTempView(name: String, tableDefinition: LogicalPlan, overrideIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  23. var currentDb: String

    Permalink
    Attributes
    protected
    Definition Classes
    SessionCatalog
  24. var currentSchema: String

    Permalink
    Attributes
    protected
  25. def databaseExists(db: String): Boolean

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  26. def defaultTablePath(tableIdent: TableIdentifier): String

    Permalink
    Definition Classes
    SessionCatalog
  27. def destroyAndRegisterBuiltInFunctions(): Unit

    Permalink

    Test only method

  28. def dropDatabase(db: String, ignoreIfNotExists: Boolean, cascade: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  29. def dropFunction(name: FunctionIdentifier, ignoreIfNotExists: Boolean): Unit

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  30. def dropGlobalTempView(name: String): Boolean

    Permalink
    Definition Classes
    SessionCatalog
  31. def dropPartitions(tableName: TableIdentifier, specs: Seq[TablePartitionSpec], ignoreIfNotExists: Boolean, purge: Boolean, retainData: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  32. def dropTable(name: TableIdentifier, ignoreIfNotExists: Boolean, purge: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  33. def dropTempFunction(name: String, ignoreIfNotExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  34. def dropTempView(name: String): Boolean

    Permalink
    Definition Classes
    SessionCatalog
  35. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  36. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  37. def failFunctionLookup(name: String): Nothing

    Permalink
    Attributes
    protected
    Definition Classes
    SessionCatalog
  38. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  39. def formatDatabaseName(name: String): String

    Permalink

    Format database name, taking into account case sensitivity.

    Format database name, taking into account case sensitivity.

    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  40. def formatTableName(name: String): String

    Permalink

    Format table name, taking into account case sensitivity.

    Format table name, taking into account case sensitivity.

    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  41. def functionExists(name: FunctionIdentifier): Boolean

    Permalink
    Definition Classes
    SessionCatalog
  42. def getCachedCatalogTable(table: QualifiedTableName): Option[CatalogTable]

    Permalink
  43. def getCachedHiveTable(table: QualifiedTableName): LogicalRelation

    Permalink
  44. def getCachedHiveTableProperties(table: QualifiedTableName): Map[String, String]

    Permalink
  45. def getCachedSampledRelations(table: QualifiedTableName): Seq[(LogicalPlan, String)]

    Permalink
  46. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  47. def getCurrentDatabase: String

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  48. def getCurrentSchema: String

    Permalink
  49. def getDataSourceRelations[T](tableTypes: Seq[ExternalTableType], baseTable: Option[String] = None): Seq[T]

    Permalink
  50. def getDataSourceTables(tableTypes: Seq[ExternalTableType], baseTable: Option[String] = None): Seq[QualifiedTableName]

    Permalink
  51. def getDatabaseMetadata(db: String): CatalogDatabase

    Permalink
    Definition Classes
    SessionCatalog
  52. def getDefaultDBPath(db: String): String

    Permalink
    Definition Classes
    SessionCatalog
  53. def getFunctionMetadata(name: FunctionIdentifier): CatalogFunction

    Permalink
    Definition Classes
    SessionCatalog
  54. def getGlobalTempView(name: String): Option[LogicalPlan]

    Permalink
    Definition Classes
    SessionCatalog
  55. def getPartition(tableName: TableIdentifier, spec: TablePartitionSpec): CatalogTablePartition

    Permalink
    Definition Classes
    SessionCatalog
  56. def getTableMetadata(name: TableIdentifier): CatalogTable

    Permalink
    Definition Classes
    SessionCatalog
  57. def getTableMetadataOption(name: TableIdentifier): Option[CatalogTable]

    Permalink
    Definition Classes
    SessionCatalog
  58. def getTableOption(qtn: QualifiedTableName): Option[CatalogTable]

    Permalink
  59. def getTableType(relation: BaseRelation): ExternalTableType

    Permalink
  60. def getTables(db: Option[String]): Seq[(String, Boolean)]

    Permalink
  61. def getTempView(name: String): Option[LogicalPlan]

    Permalink
    Definition Classes
    SessionCatalog
  62. def getTempViewOrPermanentTableMetadata(name: TableIdentifier): CatalogTable

    Permalink
    Definition Classes
    SessionCatalog
  63. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  64. def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  65. def invalidateAll(): Unit

    Permalink
  66. def invalidateTable(tableIdent: QualifiedTableName): Unit

    Permalink
  67. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  68. def isTemporaryFunction(name: FunctionIdentifier): Boolean

    Permalink
    Definition Classes
    SessionCatalog
  69. def isTemporaryTable(tableIdent: QualifiedTableName): Boolean

    Permalink

    Return whether a table with the specified name is a temporary table.

  70. def isTemporaryTable(name: TableIdentifier): Boolean

    Permalink
    Definition Classes
    SessionCatalog
  71. def isTraceEnabled(): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  72. def listDatabases(pattern: String): Seq[String]

    Permalink
    Definition Classes
    SessionCatalog
  73. def listDatabases(): Seq[String]

    Permalink
    Definition Classes
    SessionCatalog
  74. def listFunctions(db: String, pattern: String): Seq[(FunctionIdentifier, String)]

    Permalink
    Definition Classes
    SessionCatalog
  75. def listFunctions(db: String): Seq[(FunctionIdentifier, String)]

    Permalink
    Definition Classes
    SessionCatalog
  76. def listPartitionNames(tableName: TableIdentifier, partialSpec: Option[TablePartitionSpec]): Seq[String]

    Permalink
    Definition Classes
    SessionCatalog
  77. def listPartitions(tableName: TableIdentifier, partialSpec: Option[TablePartitionSpec]): Seq[CatalogTablePartition]

    Permalink
    Definition Classes
    SessionCatalog
  78. def listPartitionsByFilter(tableName: TableIdentifier, predicates: Seq[Expression]): Seq[CatalogTablePartition]

    Permalink
    Definition Classes
    SessionCatalog
  79. def listTables(db: String, pattern: String): Seq[TableIdentifier]

    Permalink
    Definition Classes
    SessionCatalog
  80. def listTables(db: String): Seq[TableIdentifier]

    Permalink
    Definition Classes
    SessionCatalog
  81. def loadFunctionResources(resources: Seq[FunctionResource]): Unit

    Permalink
    Definition Classes
    SessionCatalog
  82. def loadPartition(name: TableIdentifier, loadPath: String, spec: TablePartitionSpec, isOverwrite: Boolean, holdDDLTime: Boolean, inheritTableSpecs: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  83. def loadTable(name: TableIdentifier, loadPath: String, isOverwrite: Boolean, holdDDLTime: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  84. def log: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  85. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  86. def logDebug(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  87. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  88. def logError(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  89. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  90. def logInfo(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  91. def logName: String

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  92. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  93. def logTrace(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  94. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  95. def logWarning(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  96. def lookupFunction(name: FunctionIdentifier, children: Seq[Expression]): Expression

    Permalink

    Return an Expression that represents the specified function, assuming it exists.

    Return an Expression that represents the specified function, assuming it exists.

    For a temporary function or a permanent function that has been loaded, this method will simply lookup the function through the FunctionRegistry and create an expression based on the builder.

    For a permanent function that has not been loaded, we will first fetch its metadata from the underlying external catalog. Then, we will load all resources associated with this function (i.e. jars and files). Finally, we create a function builder based on the function class and put the builder into the FunctionRegistry. The name of this function in the FunctionRegistry will be databaseName.functionName.

    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  97. def lookupFunctionInfo(name: FunctionIdentifier): ExpressionInfo

    Permalink

    Look up the ExpressionInfo associated with the specified function, assuming it exists.

    Look up the ExpressionInfo associated with the specified function, assuming it exists.

    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  98. def lookupRelation(tableIdent: TableIdentifier, alias: Option[String]): LogicalPlan

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  99. final def lookupRelation(tableIdent: QualifiedTableName): LogicalPlan

    Permalink
  100. final def lookupRelationOption(tableIdent: QualifiedTableName): Option[LogicalPlan]

    Permalink
  101. def makeFunctionBuilder(funcName: String, className: String): FunctionBuilder

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  102. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  103. def newQualifiedTableName(tableIdent: String): QualifiedTableName

    Permalink
  104. def newQualifiedTableName(tableIdent: TableIdentifier): QualifiedTableName

    Permalink
  105. def newQualifiedTempTableName(tableIdent: String): QualifiedTableName

    Permalink
  106. def normalizeSchema(schema: StructType): StructType

    Permalink
  107. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  108. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  109. def refreshTable(tableIdent: TableIdentifier): Unit

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  110. def registerDataSourceTable(tableIdent: QualifiedTableName, userSpecifiedSchema: Option[StructType], partitionColumns: Array[String], provider: String, options: Map[String, String], relation: BaseRelation): Unit

    Permalink

    Creates a data source table (a table created with USING clause) in Hive's meta-store.

  111. def registerRelationDestroy(): Unit

    Permalink
    Attributes
    protected
  112. def registerTable(tableName: QualifiedTableName, plan: LogicalPlan): Unit

    Permalink
  113. var relationDestroyVersion: Int

    Permalink
  114. def removeDependentRelation(table: QualifiedTableName, dependentRelation: QualifiedTableName): Unit

    Permalink
  115. def removeDependentRelationFromHive(table: QualifiedTableName, dependentRelation: QualifiedTableName): Unit

    Permalink
  116. def renamePartitions(tableName: TableIdentifier, specs: Seq[TablePartitionSpec], newSpecs: Seq[TablePartitionSpec]): Unit

    Permalink
    Definition Classes
    SessionCatalog
  117. def renameTable(oldName: TableIdentifier, newName: TableIdentifier): Unit

    Permalink
    Definition Classes
    SessionCatalog
  118. def reset(): Unit

    Permalink

    Drop all existing databases (except "default"), tables, partitions and functions, and set the current database to "default".

    Drop all existing databases (except "default"), tables, partitions and functions, and set the current database to "default". This method will only remove tables from hive catalog.Don't use this method if you want to delete physical tables This is mainly used for tests.

    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  119. def setCurrentDatabase(db: String): Unit

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  120. final def setSchema(schema: String): Unit

    Permalink
  121. val snappySession: SnappySession

    Permalink
  122. val sparkConf: SparkConf

    Permalink
  123. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  124. def tableExists(tableName: QualifiedTableName): Boolean

    Permalink
  125. def tableExists(tableIdentifier: String): Boolean

    Permalink
  126. def tableExists(tableIdentifier: TableIdentifier): Boolean

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  127. val tempTables: HashMap[String, LogicalPlan]

    Permalink
    Attributes
    protected
    Definition Classes
    SessionCatalog
  128. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  129. def unregisterAllTables(): Unit

    Permalink
  130. def unregisterDataSourceTable(tableIdent: QualifiedTableName, relation: Option[BaseRelation]): Unit

    Permalink

    Drops a data source table from Hive's meta-store.

  131. def unregisterTable(tableIdent: QualifiedTableName): Unit

    Permalink
  132. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  133. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  134. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  135. def withHiveExceptionHandling[T](function: ⇒ T): T

    Permalink

Inherited from SessionCatalog

Inherited from internal.Logging

Inherited from AnyRef

Inherited from Any

Ungrouped