org.apache.spark.sql.hive

LocalHiveContext

class LocalHiveContext extends HiveContext

DEPRECATED: Use HiveContext instead.

Annotations
@deprecated
Deprecated

(Since version 1.1) Use HiveContext instead. It will still create a local metastore if one is not specified. However, note that the default directory is ./metastore_db, not ./metastore

Linear Supertypes
HiveContext, SQLContext, Serializable, Serializable, UDFRegistration, ExpressionConversions, CacheManager, SQLConf, Logging, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. LocalHiveContext
  2. HiveContext
  3. SQLContext
  4. Serializable
  5. Serializable
  6. UDFRegistration
  7. ExpressionConversions
  8. CacheManager
  9. SQLConf
  10. Logging
  11. AnyRef
  12. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new LocalHiveContext(sc: SparkContext)

Type Members

  1. implicit class DslAttribute extends AnyRef

    Definition Classes
    ExpressionConversions

  2. implicit class DslExpression extends ImplicitOperators

    Definition Classes
    ExpressionConversions

  3. implicit class DslString extends ImplicitOperators

    Definition Classes
    ExpressionConversions

  4. implicit class DslSymbol extends ImplicitAttribute

    Definition Classes
    ExpressionConversions

  5. abstract class ImplicitAttribute extends ImplicitOperators

    Definition Classes
    ExpressionConversions

  6. abstract class QueryExecution extends HiveContext.QueryExecution

    Extends QueryExecution with hive specific features.

    Extends QueryExecution with hive specific features.

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    HiveContext

  7. class SparkPlanner extends SparkStrategies

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext

Value Members

  1. final def !=(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  4. def abs(e: Expression): Abs

    Definition Classes
    ExpressionConversions

  5. def analyze(tableName: String): Unit

    Analyzes the given table in the current database to generate statistics, which will be used in query optimizations.

    Analyzes the given table in the current database to generate statistics, which will be used in query optimizations.

    Right now, it only supports Hive tables and it only updates the size of a Hive table in the Hive metastore.

    Definition Classes
    HiveContext

  6. lazy val analyzer: Analyzer { val extendedRules: List[org.apache.spark.sql.catalyst.rules.Rule[org.apache.spark.sql.catalyst.plans.logical.LogicalPlan]] }

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    HiveContext → SQLContext

  7. def applySchema(rowRDD: RDD[Row], schema: StructType): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @DeveloperApi()

  8. def approxCountDistinct(e: Expression, rsd: Double): ApproxCountDistinct

    Definition Classes
    ExpressionConversions

  9. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  10. def avg(e: Expression): Average

    Definition Classes
    ExpressionConversions

  11. implicit def baseRelationToSchemaRDD(baseRelation: BaseRelation): SchemaRDD

    Definition Classes
    SQLContext

  12. implicit def bigDecimalToLiteral(d: BigDecimal): Literal

    Definition Classes
    ExpressionConversions

  13. implicit def binaryToLiteral(a: Array[Byte]): Literal

    Definition Classes
    ExpressionConversions

  14. implicit def booleanToLiteral(b: Boolean): Literal

    Definition Classes
    ExpressionConversions

  15. implicit def byteToLiteral(b: Byte): Literal

    Definition Classes
    ExpressionConversions

  16. def cacheTable(tableName: String): Unit

    Definition Classes
    CacheManager

  17. lazy val catalog: HiveMetastoreCatalog with OverrideCatalog

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    HiveContext → SQLContext

  18. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  19. def configure(): Unit

    Sets up the system initially or after a RESET command

    Sets up the system initially or after a RESET command

    Attributes
    protected

  20. def count(e: Expression): Count

    Definition Classes
    ExpressionConversions

  21. def countDistinct(e: Expression*): CountDistinct

    Definition Classes
    ExpressionConversions

  22. def createParquetFile[A <: Product](path: String, allowExisting: Boolean, conf: Configuration)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[A]): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @Experimental()

  23. implicit def createSchemaRDD[A <: Product](rdd: RDD[A])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[A]): SchemaRDD

    Definition Classes
    SQLContext

  24. def createTable[A <: Product](tableName: String, allowExisting: Boolean = true)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[A]): Unit

    Creates a table using the schema of the given class.

    Creates a table using the schema of the given class.

    A

    A case class that is used to describe the schema of the table to be created.

    tableName

    The name of the table to create.

    allowExisting

    When false, an exception will be thrown if the table already exists.

    Definition Classes
    HiveContext

  25. implicit def dateToLiteral(d: Date): Literal

    Definition Classes
    ExpressionConversions

  26. val ddlParser: DDLParser

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext

  27. implicit def decimalToLiteral(d: Decimal): Literal

    Definition Classes
    ExpressionConversions

  28. implicit def doubleToLiteral(d: Double): Literal

    Definition Classes
    ExpressionConversions

  29. def dropTempTable(tableName: String): Unit

    Definition Classes
    SQLContext

  30. lazy val emptyResult: RDD[Row]

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext

  31. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  32. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  33. def executePlan(plan: LogicalPlan): QueryExecution

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    HiveContext → SQLContext

  34. def executeSql(sql: String): LocalHiveContext.QueryExecution

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext

  35. var extraStrategies: Seq[Strategy]

    Definition Classes
    SQLContext

  36. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  37. def first(e: Expression): First

    Definition Classes
    ExpressionConversions

  38. implicit def floatToLiteral(f: Float): Literal

    Definition Classes
    ExpressionConversions

  39. lazy val functionRegistry: HiveFunctionRegistry with OverrideFunctionRegistry

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    HiveContext → SQLContext

  40. def getAllConfs: Map[String, String]

    Definition Classes
    SQLConf

  41. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  42. def getConf(key: String, defaultValue: String): String

    Definition Classes
    SQLConf

  43. def getConf(key: String): String

    Definition Classes
    SQLConf

  44. def hashCode(): Int

    Definition Classes
    AnyRef → Any

  45. val hivePlanner: SparkPlanner with HiveStrategies

    Definition Classes
    HiveContext

  46. lazy val hiveconf: HiveConf

    SQLConf and HiveConf contracts:

    SQLConf and HiveConf contracts:

    1. reuse existing started SessionState if any 2. when the Hive session is first initialized, params in HiveConf will get picked up by the SQLConf. Additionally, any properties set by set() or a SET command inside sql() will be set in the SQLConf *as well as* in the HiveConf.

    Attributes
    protected[org.apache.spark.sql.hive]
    Definition Classes
    HiveContext

  47. implicit def intToLiteral(i: Int): Literal

    Definition Classes
    ExpressionConversions

  48. def isCached(tableName: String): Boolean

    Definition Classes
    CacheManager

  49. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  50. def isTraceEnabled(): Boolean

    Attributes
    protected
    Definition Classes
    Logging

  51. def jsonFile(path: String, samplingRatio: Double): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @Experimental()

  52. def jsonFile(path: String, schema: StructType): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @Experimental()

  53. def jsonFile(path: String): SchemaRDD

    Definition Classes
    SQLContext

  54. def jsonRDD(json: RDD[String], samplingRatio: Double): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @Experimental()

  55. def jsonRDD(json: RDD[String], schema: StructType): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @Experimental()

  56. def jsonRDD(json: RDD[String]): SchemaRDD

    Definition Classes
    SQLContext

  57. def last(e: Expression): Last

    Definition Classes
    ExpressionConversions

  58. def log: Logger

    Attributes
    protected
    Definition Classes
    Logging

  59. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  60. def logDebug(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  61. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  62. def logError(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  63. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  64. def logInfo(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  65. def logName: String

    Attributes
    protected
    Definition Classes
    Logging

  66. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  67. def logTrace(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  68. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  69. def logWarning(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  70. implicit def logicalPlanToSparkQuery(plan: LogicalPlan): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @DeveloperApi()

  71. implicit def longToLiteral(l: Long): Literal

    Definition Classes
    ExpressionConversions

  72. def lower(e: Expression): Lower

    Definition Classes
    ExpressionConversions

  73. def max(e: Expression): Max

    Definition Classes
    ExpressionConversions

  74. lazy val metastorePath: String

  75. def min(e: Expression): Min

    Definition Classes
    ExpressionConversions

  76. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  77. final def notify(): Unit

    Definition Classes
    AnyRef

  78. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  79. lazy val optimizer: Optimizer

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext

  80. lazy val outputBuffer: OutputStream { ... /* 4 definitions in type refinement */ }

    Attributes
    protected
    Definition Classes
    HiveContext

  81. def parquetFile(path: String): SchemaRDD

    Definition Classes
    SQLContext

  82. def parseSql(sql: String): LogicalPlan

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext

  83. val planner: SparkPlanner with HiveStrategies

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    HiveContext → SQLContext

  84. val prepareForExecution: RuleExecutor[SparkPlan] { val batches: List[this.Batch] }

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext

  85. def registerFunction[T](name: String, func: Function22[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  86. def registerFunction[T](name: String, func: Function21[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  87. def registerFunction[T](name: String, func: Function20[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  88. def registerFunction[T](name: String, func: Function19[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  89. def registerFunction[T](name: String, func: Function18[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  90. def registerFunction[T](name: String, func: Function17[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  91. def registerFunction[T](name: String, func: Function16[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  92. def registerFunction[T](name: String, func: Function15[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  93. def registerFunction[T](name: String, func: Function14[_, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  94. def registerFunction[T](name: String, func: Function13[_, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  95. def registerFunction[T](name: String, func: Function12[_, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  96. def registerFunction[T](name: String, func: Function11[_, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  97. def registerFunction[T](name: String, func: Function10[_, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  98. def registerFunction[T](name: String, func: Function9[_, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  99. def registerFunction[T](name: String, func: Function8[_, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  100. def registerFunction[T](name: String, func: Function7[_, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  101. def registerFunction[T](name: String, func: Function6[_, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  102. def registerFunction[T](name: String, func: Function5[_, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  103. def registerFunction[T](name: String, func: Function4[_, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  104. def registerFunction[T](name: String, func: Function3[_, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  105. def registerFunction[T](name: String, func: Function2[_, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  106. def registerFunction[T](name: String, func: Function1[_, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration

  107. def registerRDDAsTable(rdd: SchemaRDD, tableName: String): Unit

    Definition Classes
    SQLContext

  108. def runHive(cmd: String, maxRows: Int = 1000): Seq[String]

    Execute the command using Hive and return the results as a sequence.

    Execute the command using Hive and return the results as a sequence. Each element in the sequence is one row.

    Attributes
    protected
    Definition Classes
    HiveContext

  109. def runSqlHive(sql: String): Seq[String]

    Runs the specified SQL query using Hive.

    Runs the specified SQL query using Hive.

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    HiveContext

  110. lazy val sessionState: SessionState

    SQLConf and HiveConf contracts:

    SQLConf and HiveConf contracts:

    1. reuse existing started SessionState if any 2. when the Hive session is first initialized, params in HiveConf will get picked up by the SQLConf. Additionally, any properties set by set() or a SET command inside sql() will be set in the SQLConf *as well as* in the HiveConf.

    Attributes
    protected[org.apache.spark.sql.hive]
    Definition Classes
    HiveContext

  111. def setConf(key: String, value: String): Unit

    Definition Classes
    HiveContext → SQLConf

  112. def setConf(props: Properties): Unit

    Definition Classes
    SQLConf

  113. val settings: Map[String, String]

    Attributes
    protected[org.apache.spark]
    Definition Classes
    SQLConf

  114. implicit def shortToLiteral(s: Short): Literal

    Definition Classes
    ExpressionConversions

  115. val sparkContext: SparkContext

    Definition Classes
    SQLContext

  116. def sql(sqlText: String): SchemaRDD

    Definition Classes
    HiveContext → SQLContext

  117. val sqlParser: SparkSQLParser

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext

  118. def sqrt(e: Expression): Sqrt

    Definition Classes
    ExpressionConversions

  119. implicit def stringToLiteral(s: String): Literal

    Definition Classes
    ExpressionConversions

  120. def sum(e: Expression): Sum

    Definition Classes
    ExpressionConversions

  121. def sumDistinct(e: Expression): SumDistinct

    Definition Classes
    ExpressionConversions

  122. implicit def symbolToUnresolvedAttribute(s: Symbol): UnresolvedAttribute

    Definition Classes
    ExpressionConversions

  123. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  124. def table(tableName: String): SchemaRDD

    Definition Classes
    SQLContext

  125. implicit def timestampToLiteral(t: Timestamp): Literal

    Definition Classes
    ExpressionConversions

  126. def toString(): String

    Definition Classes
    AnyRef → Any

  127. def uncacheTable(tableName: String): Unit

    Definition Classes
    CacheManager

  128. def upper(e: Expression): Upper

    Definition Classes
    ExpressionConversions

  129. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  130. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  131. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  132. lazy val warehousePath: String

Deprecated Value Members

  1. def hiveql(hqlQuery: String): SchemaRDD

    Definition Classes
    HiveContext
    Annotations
    @deprecated
    Deprecated

    (Since version 1.1)

  2. def hql(hqlQuery: String): SchemaRDD

    Definition Classes
    HiveContext
    Annotations
    @deprecated
    Deprecated

    (Since version 1.1)

Inherited from HiveContext

Inherited from SQLContext

Inherited from Serializable

Inherited from Serializable

Inherited from UDFRegistration

Inherited from ExpressionConversions

Inherited from CacheManager

Inherited from SQLConf

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Ungrouped