Class/Object

org.apache.spark.sql.hive

SnappyStoreHiveCatalog

Related Docs: object SnappyStoreHiveCatalog | package hive

Permalink

class SnappyStoreHiveCatalog extends SessionCatalog with internal.Logging

Catalog using Hive for persistence and adding Snappy extensions like stream/topK tables and returning LogicalPlan to materialize these entities.

Linear Supertypes
SessionCatalog, internal.Logging, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SnappyStoreHiveCatalog
  2. SessionCatalog
  3. Logging
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new SnappyStoreHiveCatalog(externalCatalog: ExternalCatalog, snappySession: SnappySession, functionResourceLoader: FunctionResourceLoader, functionRegistry: FunctionRegistry, sqlConf: SQLConf, hadoopConf: Configuration)

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. def alterDatabase(dbDefinition: CatalogDatabase): Unit

    Permalink
    Definition Classes
    SessionCatalog
  5. def alterPartitions(tableName: TableIdentifier, parts: Seq[CatalogTablePartition]): Unit

    Permalink
    Definition Classes
    SessionCatalog
  6. def alterTable(tableDefinition: CatalogTable): Unit

    Permalink
    Definition Classes
    SessionCatalog
  7. def alterTableToAddIndexProp(inTable: QualifiedTableName, index: QualifiedTableName): Unit

    Permalink
  8. def alterTableToRemoveIndexProp(inTable: QualifiedTableName, index: QualifiedTableName): Unit

    Permalink
  9. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  10. def clearTempTables(): Unit

    Permalink
    Definition Classes
    SessionCatalog
  11. var client: HiveClient

    Permalink

    Hive client that is used to retrieve metadata from the Hive MetaStore.

    Hive client that is used to retrieve metadata from the Hive MetaStore. The version of the Hive client that is used here must match the meta-store that is configured in the hive-site.xml file.

    Attributes
    protected[org.apache.spark.sql]
  12. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  13. def compatibleSchema(schema1: StructType, schema2: StructType): Boolean

    Permalink
  14. def configure(): Map[String, String]

    Permalink

    Overridden by child classes that need to set configuration before client init (but after hive-site.xml).

    Overridden by child classes that need to set configuration before client init (but after hive-site.xml).

    Attributes
    protected
  15. def createDatabase(dbDefinition: CatalogDatabase, ignoreIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  16. def createFunction(funcDefinition: CatalogFunction, ignoreIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  17. def createPartitions(tableName: TableIdentifier, parts: Seq[CatalogTablePartition], ignoreIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  18. def createTable(tableDefinition: CatalogTable, ignoreIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  19. def createTempFunction(name: String, info: ExpressionInfo, funcDefinition: FunctionBuilder, ignoreIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  20. def createTempView(name: String, tableDefinition: LogicalPlan, overrideIfExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  21. var currentDb: String

    Permalink
    Attributes
    protected
    Definition Classes
    SessionCatalog
  22. var currentSchema: String

    Permalink
    Attributes
    protected
  23. def databaseExists(db: String): Boolean

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  24. def defaultTablePath(tableIdent: TableIdentifier): String

    Permalink
    Definition Classes
    SessionCatalog
  25. def dropDatabase(db: String, ignoreIfNotExists: Boolean, cascade: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  26. def dropFunction(name: FunctionIdentifier, ignoreIfNotExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  27. def dropPartitions(tableName: TableIdentifier, specs: Seq[TablePartitionSpec], ignoreIfNotExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  28. def dropTable(name: TableIdentifier, ignoreIfNotExists: Boolean): Unit

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  29. def dropTempFunction(name: String, ignoreIfNotExists: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  30. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  31. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  32. def failFunctionLookup(name: String): Nothing

    Permalink
    Attributes
    protected
    Definition Classes
    SessionCatalog
  33. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  34. def formatDatabaseName(name: String): String

    Permalink

    Format database name, taking into account case sensitivity.

    Format database name, taking into account case sensitivity.

    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  35. def formatTableName(name: String): String

    Permalink

    Format table name, taking into account case sensitivity.

    Format table name, taking into account case sensitivity.

    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  36. def functionExists(name: FunctionIdentifier): Boolean

    Permalink
    Definition Classes
    SessionCatalog
  37. def getCachedHiveTable(table: QualifiedTableName): LogicalRelation

    Permalink
  38. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  39. def getCurrentDatabase: String

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  40. def getCurrentSchema: String

    Permalink
  41. def getDataSourceRelations[T](tableTypes: Seq[ExternalTableType], baseTable: Option[String] = None): Seq[T]

    Permalink
  42. def getDataSourceTables(tableTypes: Seq[ExternalTableType], baseTable: Option[String] = None): Seq[QualifiedTableName]

    Permalink
  43. def getDatabaseMetadata(db: String): CatalogDatabase

    Permalink
    Definition Classes
    SessionCatalog
  44. def getDefaultDBPath(db: String): String

    Permalink
    Definition Classes
    SessionCatalog
  45. def getFunctionMetadata(name: FunctionIdentifier): CatalogFunction

    Permalink
    Definition Classes
    SessionCatalog
  46. def getPartition(tableName: TableIdentifier, spec: TablePartitionSpec): CatalogTablePartition

    Permalink
    Definition Classes
    SessionCatalog
  47. def getTableMetadata(name: TableIdentifier): CatalogTable

    Permalink

    Retrieve the metadata of an existing metastore table.

    Retrieve the metadata of an existing metastore table. If no database is specified, assume the table is in the current database. If the specified table is not found in the database then a NoSuchTableException is thrown.

    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  48. def getTableMetadataOption(name: TableIdentifier): Option[CatalogTable]

    Permalink
    Definition Classes
    SessionCatalog
  49. def getTableType(relation: BaseRelation): ExternalTableType

    Permalink
  50. def getTables(db: Option[String]): Seq[(String, Boolean)]

    Permalink
  51. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  52. def hiveMetastoreBarrierPrefixes(): Seq[String]

    Permalink

    A comma separated list of class prefixes that should explicitly be reloaded for each version of Hive that Spark SQL is communicating with.

    A comma separated list of class prefixes that should explicitly be reloaded for each version of Hive that Spark SQL is communicating with. For example, Hive UDFs that are declared in a prefix that typically would be shared (i.e. org.apache.spark.*)

    Attributes
    protected[org.apache.spark.sql]
  53. def hiveMetastoreJars(): String

    Permalink

    The location of the jars that should be used to instantiate the Hive meta-store client.

    The location of the jars that should be used to instantiate the Hive meta-store client. This property can be one of three options:

    a classpath in the standard format for both hive and hadoop.

    builtin - attempt to discover the jars that were used to load Spark SQL and use those. This option is only valid when using the execution version of Hive.

    maven - download the correct version of hive on demand from maven.

    Attributes
    protected[org.apache.spark.sql]
  54. def hiveMetastoreSharedPrefixes(): Seq[String]

    Permalink

    A comma separated list of class prefixes that should be loaded using the ClassLoader that is shared between Spark SQL and a specific version of Hive.

    A comma separated list of class prefixes that should be loaded using the ClassLoader that is shared between Spark SQL and a specific version of Hive. An example of classes that should be shared is JDBC drivers that are needed to talk to the meta-store. Other classes that need to be shared are those that interact with classes that are already shared. For example, custom appender used by log4j.

    Attributes
    protected[org.apache.spark.sql]
  55. val hiveMetastoreVersion: String

    Permalink

    The version of the hive client that will be used to communicate with the meta-store forL catalog.

    The version of the hive client that will be used to communicate with the meta-store forL catalog.

    Attributes
    protected[org.apache.spark.sql]
  56. def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  57. def invalidateTable(tableIdent: QualifiedTableName): Unit

    Permalink
  58. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  59. def isTemporaryTable(tableIdent: QualifiedTableName): Boolean

    Permalink

    Return whether a table with the specified name is a temporary table.

  60. def isTemporaryTable(name: TableIdentifier): Boolean

    Permalink
    Definition Classes
    SessionCatalog
  61. def isTraceEnabled(): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  62. def listDatabases(pattern: String): Seq[String]

    Permalink
    Definition Classes
    SessionCatalog
  63. def listDatabases(): Seq[String]

    Permalink
    Definition Classes
    SessionCatalog
  64. def listFunctions(db: String, pattern: String): Seq[(FunctionIdentifier, String)]

    Permalink
    Definition Classes
    SessionCatalog
  65. def listFunctions(db: String): Seq[(FunctionIdentifier, String)]

    Permalink
    Definition Classes
    SessionCatalog
  66. def listPartitions(tableName: TableIdentifier, partialSpec: Option[TablePartitionSpec]): Seq[CatalogTablePartition]

    Permalink
    Definition Classes
    SessionCatalog
  67. def listTables(db: String, pattern: String): Seq[TableIdentifier]

    Permalink

    List all matching tables in the specified database, including temporary tables.

    List all matching tables in the specified database, including temporary tables.

    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  68. def listTables(db: String): Seq[TableIdentifier]

    Permalink
    Definition Classes
    SessionCatalog
  69. def loadFunctionResources(resources: Seq[FunctionResource]): Unit

    Permalink
    Definition Classes
    SessionCatalog
  70. def loadPartition(name: TableIdentifier, loadPath: String, partition: TablePartitionSpec, isOverwrite: Boolean, holdDDLTime: Boolean, inheritTableSpecs: Boolean, isSkewedStoreAsSubdir: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  71. def loadTable(name: TableIdentifier, loadPath: String, isOverwrite: Boolean, holdDDLTime: Boolean): Unit

    Permalink
    Definition Classes
    SessionCatalog
  72. def log: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  73. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  74. def logDebug(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  75. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  76. def logError(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  77. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  78. def logInfo(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  79. def logName: String

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  80. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  81. def logTrace(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  82. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  83. def logWarning(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  84. def lookupFunction(name: FunctionIdentifier, children: Seq[Expression]): Expression

    Permalink
    Definition Classes
    SessionCatalog
  85. def lookupRelation(tableIdent: TableIdentifier, alias: Option[String]): LogicalPlan

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  86. final def lookupRelation(tableIdent: QualifiedTableName): LogicalPlan

    Permalink
  87. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  88. def newQualifiedTableName(tableIdent: String): QualifiedTableName

    Permalink
  89. def newQualifiedTableName(tableIdent: TableIdentifier): QualifiedTableName

    Permalink
  90. def newQualifiedTempTableName(tableIdent: String): QualifiedTableName

    Permalink
  91. def normalizeSchema(schema: StructType): StructType

    Permalink
  92. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  93. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  94. def refreshTable(tableIdent: TableIdentifier): Unit

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  95. def registerDataSourceTable(tableIdent: QualifiedTableName, userSpecifiedSchema: Option[StructType], partitionColumns: Array[String], provider: String, options: Map[String, String], relation: BaseRelation): Unit

    Permalink

    Creates a data source table (a table created with USING clause) in Hive's meta-store.

  96. def registerTable(tableName: QualifiedTableName, plan: LogicalPlan): Unit

    Permalink
  97. def removeIndexProp(inTable: QualifiedTableName, index: QualifiedTableName): Unit

    Permalink
  98. def renamePartitions(tableName: TableIdentifier, specs: Seq[TablePartitionSpec], newSpecs: Seq[TablePartitionSpec]): Unit

    Permalink
    Definition Classes
    SessionCatalog
  99. def renameTable(oldName: TableIdentifier, newName: TableIdentifier): Unit

    Permalink
    Definition Classes
    SessionCatalog
  100. def setCurrentDatabase(db: String): Unit

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  101. final def setSchema(schema: String): Unit

    Permalink
  102. val snappySession: SnappySession

    Permalink
  103. val sparkConf: SparkConf

    Permalink
  104. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  105. def tableExists(tableName: QualifiedTableName): Boolean

    Permalink
  106. def tableExists(tableIdentifier: String): Boolean

    Permalink
  107. def tableExists(tableIdentifier: TableIdentifier): Boolean

    Permalink
    Definition Classes
    SnappyStoreHiveCatalog → SessionCatalog
  108. val tempTables: HashMap[String, LogicalPlan]

    Permalink
    Attributes
    protected
    Definition Classes
    SessionCatalog
  109. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  110. def unregisterAllTables(): Unit

    Permalink
  111. def unregisterDataSourceTable(tableIdent: QualifiedTableName, relation: Option[BaseRelation]): Unit

    Permalink

    Drops a data source table from Hive's meta-store.

  112. def unregisterTable(tableIdent: QualifiedTableName): Unit

    Permalink
  113. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  114. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  115. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  116. def withHiveExceptionHandling[T](function: ⇒ T): T

    Permalink

Inherited from SessionCatalog

Inherited from internal.Logging

Inherited from AnyRef

Inherited from Any

Ungrouped