Packages

object SharedState extends Logging

Linear Supertypes
Logging, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SharedState
  2. Logging
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Value Members

  1. def qualifyWarehousePath(hadoopConf: Configuration, warehousePath: String): String
  2. def resolveWarehousePath(sparkConf: SparkConf, hadoopConf: Configuration, initialConfigs: Map[String, String] = Map.empty): String

    Determine the warehouse path using the key spark.sql.warehouse.dir in the SparkConf or the initial options from the very first created SparkSession instance, and hive.metastore.warehouse.dir in hadoop Configuration.

    Determine the warehouse path using the key spark.sql.warehouse.dir in the SparkConf or the initial options from the very first created SparkSession instance, and hive.metastore.warehouse.dir in hadoop Configuration. The priority order is: s.s.w.d in initialConfigs > s.s.w.d in spark conf (user specified) > h.m.w.d in hadoop conf (user specified) > s.s.w.d in spark conf (default)

    returns

    the resolved warehouse path.

  3. def setWarehousePathConf(sparkConf: SparkConf, hadoopConf: Configuration, warehousePath: String): Unit