CatalogTable

A table defined in the catalog.

Note that Hive's metastore also tracks skewed columns. We should consider adding that in the future once we have a better understanding of how we want to handle skewed columns.

hasUnsupportedFeatures: is used to indicate whether all table metadata entries retrieved from the concrete underlying external catalog (e.g. Hive metastore) are supported by Spark SQL. For example, if the underlying Hive table has skewed columns, this information can't be mapped to CatalogTable since Spark SQL doesn't handle skewed columns for now. In this case hasUnsupportedFeatures is set to true. By default, it is false.

Linear Supertypes

Serializable, Serializable, Product, Equals, AnyRef, Any

Instance Constructors

new CatalogTable(identifier: TableIdentifier, tableType: CatalogTableType, storage: CatalogStorageFormat, schema: Seq[CatalogColumn], partitionColumnNames: Seq[String] = Seq.empty, sortColumnNames: Seq[String] = Seq.empty, bucketColumnNames: Seq[String] = Seq.empty, numBuckets: Int = 1, owner: String = "", createTime: Long = System.currentTimeMillis, lastAccessTime: Long = 1, properties: Map[String, String] = Map.empty, viewOriginalText: Option[String] = None, viewText: Option[String] = None, comment: Option[String] = None, hasUnsupportedFeatures: Boolean = false)

hasUnsupportedFeatures
is used to indicate whether all table metadata entries retrieved from the concrete underlying external catalog (e.g. Hive metastore) are supported by Spark SQL. For example, if the underlying Hive table has skewed columns, this information can't be mapped to CatalogTable since Spark SQL doesn't handle skewed columns for now. In this case hasUnsupportedFeatures is set to true. By default, it is false.

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
val bucketColumnNames: Seq[String]
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
val comment: Option[String]
val createTime: Long
def database: String

Return the database this table was specified to belong to, assuming it exists.
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
val hasUnsupportedFeatures: Boolean

is used to indicate whether all table metadata entries retrieved from the concrete underlying external catalog (e.g.
is used to indicate whether all table metadata entries retrieved from the concrete underlying external catalog (e.g. Hive metastore) are supported by Spark SQL. For example, if the underlying Hive table has skewed columns, this information can't be mapped to CatalogTable since Spark SQL doesn't handle skewed columns for now. In this case hasUnsupportedFeatures is set to true. By default, it is false.
val identifier: TableIdentifier
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
val lastAccessTime: Long
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
val numBuckets: Int
val owner: String
val partitionColumnNames: Seq[String]
def partitionColumns: Seq[CatalogColumn]

Columns this table is partitioned by.
val properties: Map[String, String]
def qualifiedName: String

Return the fully qualified name of this table, assuming the database was specified.
val schema: Seq[CatalogColumn]
val sortColumnNames: Seq[String]
val storage: CatalogStorageFormat
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
val tableType: CatalogTableType
val viewOriginalText: Option[String]
val viewText: Option[String]
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
def withNewStorage(locationUri: Option[String] = storage.locationUri, inputFormat: Option[String] = storage.inputFormat, outputFormat: Option[String] = storage.outputFormat, compressed: Boolean = false, serde: Option[String] = storage.serde, serdeProperties: Map[String, String] = storage.serdeProperties): CatalogTable

Syntactic sugar to update a field in storage.

Related Doc: package catalog

Instance Constructors

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

final def asInstanceOf[T0]: T0

val bucketColumnNames: Seq[String]

def clone(): AnyRef

val comment: Option[String]

val createTime: Long

def database: String

final def eq(arg0: AnyRef): Boolean

def finalize(): Unit

final def getClass(): Class[_]

val hasUnsupportedFeatures: Boolean

val identifier: TableIdentifier

final def isInstanceOf[T0]: Boolean

val lastAccessTime: Long

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

val numBuckets: Int

val owner: String

val partitionColumnNames: Seq[String]

def partitionColumns: Seq[CatalogColumn]

val properties: Map[String, String]

def qualifiedName: String

val schema: Seq[CatalogColumn]

val sortColumnNames: Seq[String]

val storage: CatalogStorageFormat

final def synchronized[T0](arg0: ⇒ T0): T0

val tableType: CatalogTableType

val viewOriginalText: Option[String]

val viewText: Option[String]

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped