org.apache.spark.sql.catalyst.catalog
InMemoryCatalog
Related Doc:
package catalog
Alter a database whose name matches the one specified in dbDefinition,
assuming the database exists.
Alter a database whose name matches the one specified in dbDefinition,
assuming the database exists.
Note: If the underlying implementation does not support altering a certain field, this becomes a no-op.
Alter one or many table partitions whose specs that match those specified in parts,
assuming the partitions exist.
Alter one or many table partitions whose specs that match those specified in parts,
assuming the partitions exist.
Note: If the underlying implementation does not support altering a certain field, this becomes a no-op.
Alter a table whose database and name match the ones specified in tableDefinition, assuming
the table exists.
Alter a table whose database and name match the ones specified in tableDefinition, assuming
the table exists. Note that, even though we can specify database in tableDefinition, it's
used to identify the table, not to alter the table's database, which is not allowed.
Note: If the underlying implementation does not support altering a certain field, this becomes a no-op.
Alter the schema of a table identified by the provided database and table name.
Alter the schema of a table identified by the provided database and table name. The new schema should still contain the existing bucket columns and partition columns used by the table. This method will also update any Spark SQL-related parameters stored as Hive table properties (such as the schema itself).
Database that table to alter schema for exists in
Name of table to alter schema for
Updated schema to be used for the table (must contain existing partition and bucket columns)
Returns the specified partition or None if it does not exist.
Returns the specified partition or None if it does not exist.
List the names of all partitions that belong to the specified table, assuming it exists.
List the names of all partitions that belong to the specified table, assuming it exists.
For a table with partition columns p1, p2, p3, each partition name is formatted as
p1=v1/p2=v2/p3=v3. Each partition column name and value is an escaped path name, and can be
decoded with the ExternalCatalogUtils.unescapePathName method.
The returned sequence is sorted as strings.
A partial partition spec may optionally be provided to filter the partitions returned, as
described in the listPartitions method.
database name
table name
partition spec
List the metadata of all partitions that belong to the specified table, assuming it exists.
List the metadata of all partitions that belong to the specified table, assuming it exists.
A partial partition spec may optionally be provided to filter the partitions returned. For instance, if there exist partitions (a='1', b='2'), (a='1', b='3') and (a='2', b='4'), then a partial spec of (a='1') will return the first two only.
database name
table name
partition spec
List the metadata of partitions that belong to the specified table, assuming it exists, that satisfy the given partition-pruning predicate expressions.
List the metadata of partitions that belong to the specified table, assuming it exists, that satisfy the given partition-pruning predicate expressions.
database name
table name
partition-pruning predicates
default timezone id to parse partition values of TimestampType
Loads data into a partition.
Loads data into a partition.
Whether the source data is local, as defined by the "LOAD DATA LOCAL" HiveQL command.
Loads data into a table.
Loads data into a table.
Whether the source data is local, as defined by the "LOAD DATA LOCAL" HiveQL command.
Override the specs of one or many existing table partitions, assuming they exist.
Override the specs of one or many existing table partitions, assuming they exist.
This assumes index i of specs corresponds to index i of newSpecs.
An in-memory (ephemeral) implementation of the system catalog.
This is a dummy implementation that does not require setting up external systems. It is intended for testing or exploration purposes only and should not be used in production.
All public methods should be synchronized for thread-safety.