org.apache.spark.sql.hive.test

TestHive

object TestHive extends TestHiveContext

Linear Supertypes
TestHiveContext, HiveContext, SQLContext, Serializable, Serializable, UDFRegistration, ExpressionConversions, CacheManager, SQLConf, Logging, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. TestHive
  2. TestHiveContext
  3. HiveContext
  4. SQLContext
  5. Serializable
  6. Serializable
  7. UDFRegistration
  8. ExpressionConversions
  9. CacheManager
  10. SQLConf
  11. Logging
  12. AnyRef
  13. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Type Members

  1. implicit class DslAttribute extends AnyRef

    Definition Classes
    ExpressionConversions
  2. implicit class DslExpression extends ImplicitOperators

    Definition Classes
    ExpressionConversions
  3. implicit class DslString extends ImplicitOperators

    Definition Classes
    ExpressionConversions
  4. implicit class DslSymbol extends ImplicitAttribute

    Definition Classes
    ExpressionConversions
  5. class HiveQLQueryExecution extends QueryExecution

    Attributes
    protected[org.apache.spark.sql.hive]
    Definition Classes
    TestHiveContext
  6. abstract class ImplicitAttribute extends ImplicitOperators

    Definition Classes
    ExpressionConversions
  7. abstract class QueryExecution extends TestHiveContext.QueryExecution

    Override QueryExecution with special debug workflow.

    Override QueryExecution with special debug workflow.

    Definition Classes
    TestHiveContext
  8. class SparkPlanner extends SparkStrategies

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext
  9. implicit class SqlCmd extends AnyRef

    Attributes
    protected[org.apache.spark.sql.hive]
    Definition Classes
    TestHiveContext
  10. case class TestTable(name: String, commands: () ⇒ Unit*) extends Product with Serializable

    Definition Classes
    TestHiveContext

Value Members

  1. final def !=(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  4. def abs(e: Expression): Abs

    Definition Classes
    ExpressionConversions
  5. def analyze(tableName: String): Unit

    Analyzes the given table in the current database to generate statistics, which will be used in query optimizations.

    Analyzes the given table in the current database to generate statistics, which will be used in query optimizations.

    Right now, it only supports Hive tables and it only updates the size of a Hive table in the Hive metastore.

    Definition Classes
    HiveContext
  6. lazy val analyzer: Analyzer { val extendedRules: List[org.apache.spark.sql.catalyst.rules.Rule[org.apache.spark.sql.catalyst.plans.logical.LogicalPlan]] }

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    HiveContext → SQLContext
  7. def applySchema(rowRDD: RDD[Row], schema: StructType): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @DeveloperApi()
  8. def approxCountDistinct(e: Expression, rsd: Double): ApproxCountDistinct

    Definition Classes
    ExpressionConversions
  9. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  10. def avg(e: Expression): Average

    Definition Classes
    ExpressionConversions
  11. implicit def baseRelationToSchemaRDD(baseRelation: BaseRelation): SchemaRDD

    Definition Classes
    SQLContext
  12. implicit def bigDecimalToLiteral(d: BigDecimal): Literal

    Definition Classes
    ExpressionConversions
  13. implicit def binaryToLiteral(a: Array[Byte]): Literal

    Definition Classes
    ExpressionConversions
  14. implicit def booleanToLiteral(b: Boolean): Literal

    Definition Classes
    ExpressionConversions
  15. implicit def byteToLiteral(b: Byte): Literal

    Definition Classes
    ExpressionConversions
  16. def cacheTable(tableName: String): Unit

    Definition Classes
    CacheManager
  17. var cacheTables: Boolean

    Definition Classes
    TestHiveContext
  18. lazy val catalog: HiveMetastoreCatalog with OverrideCatalog

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    HiveContext → SQLContext
  19. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  20. def configure(): Unit

    Sets up the system initially or after a RESET command

    Sets up the system initially or after a RESET command

    Attributes
    protected
    Definition Classes
    TestHiveContext
  21. def count(e: Expression): Count

    Definition Classes
    ExpressionConversions
  22. def countDistinct(e: Expression*): CountDistinct

    Definition Classes
    ExpressionConversions
  23. def createParquetFile[A <: Product](path: String, allowExisting: Boolean, conf: Configuration)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[A]): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @Experimental()
  24. implicit def createSchemaRDD[A <: Product](rdd: RDD[A])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[A]): SchemaRDD

    Definition Classes
    SQLContext
  25. def createTable[A <: Product](tableName: String, allowExisting: Boolean = true)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[A]): Unit

    Creates a table using the schema of the given class.

    Creates a table using the schema of the given class.

    A

    A case class that is used to describe the schema of the table to be created.

    tableName

    The name of the table to create.

    allowExisting

    When false, an exception will be thrown if the table already exists.

    Definition Classes
    HiveContext
  26. implicit def dateToLiteral(d: Date): Literal

    Definition Classes
    ExpressionConversions
  27. val ddlParser: DDLParser

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext
  28. implicit def decimalToLiteral(d: Decimal): Literal

    Definition Classes
    ExpressionConversions
  29. val describedTable: Regex

    Definition Classes
    TestHiveContext
  30. implicit def doubleToLiteral(d: Double): Literal

    Definition Classes
    ExpressionConversions
  31. def dropTempTable(tableName: String): Unit

    Definition Classes
    SQLContext
  32. lazy val emptyResult: RDD[Row]

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext
  33. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  34. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  35. def executePlan(plan: LogicalPlan): QueryExecution

    Definition Classes
    TestHiveContextHiveContext → SQLContext
  36. def executeSql(sql: String): TestHive.QueryExecution

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext
  37. var extraStrategies: Seq[Strategy]

    Definition Classes
    SQLContext
  38. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  39. def first(e: Expression): First

    Definition Classes
    ExpressionConversions
  40. implicit def floatToLiteral(f: Float): Literal

    Definition Classes
    ExpressionConversions
  41. lazy val functionRegistry: HiveFunctionRegistry with OverrideFunctionRegistry

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    HiveContext → SQLContext
  42. def getAllConfs: Map[String, String]

    Definition Classes
    SQLConf
  43. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  44. def getConf(key: String, defaultValue: String): String

    Definition Classes
    SQLConf
  45. def getConf(key: String): String

    Definition Classes
    SQLConf
  46. def getHiveFile(path: String): File

    Definition Classes
    TestHiveContext
  47. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  48. lazy val hiveDevHome: Option[File]

    The location of the hive source code.

    The location of the hive source code.

    Definition Classes
    TestHiveContext
  49. val hiveFilesTemp: File

    Definition Classes
    TestHiveContext
  50. lazy val hiveHome: Option[File]

    The location of the compiled hive distribution

    The location of the compiled hive distribution

    Definition Classes
    TestHiveContext
  51. val hivePlanner: SparkPlanner with HiveStrategies

    Definition Classes
    HiveContext
  52. val hiveQTestUtilTables: Seq[TestTable]

    Definition Classes
    TestHiveContext
  53. lazy val hiveconf: HiveConf

    SQLConf and HiveConf contracts:

    SQLConf and HiveConf contracts:

    1. reuse existing started SessionState if any 2. when the Hive session is first initialized, params in HiveConf will get picked up by the SQLConf. Additionally, any properties set by set() or a SET command inside sql() will be set in the SQLConf *as well as* in the HiveConf.

    Attributes
    protected[org.apache.spark.sql.hive]
    Definition Classes
    HiveContext
  54. val inRepoTests: File

    Definition Classes
    TestHiveContext
  55. implicit def intToLiteral(i: Int): Literal

    Definition Classes
    ExpressionConversions
  56. def isCached(tableName: String): Boolean

    Definition Classes
    CacheManager
  57. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  58. def isTraceEnabled(): Boolean

    Attributes
    protected
    Definition Classes
    Logging
  59. def jsonFile(path: String, samplingRatio: Double): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @Experimental()
  60. def jsonFile(path: String, schema: StructType): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @Experimental()
  61. def jsonFile(path: String): SchemaRDD

    Definition Classes
    SQLContext
  62. def jsonRDD(json: RDD[String], samplingRatio: Double): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @Experimental()
  63. def jsonRDD(json: RDD[String], schema: StructType): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @Experimental()
  64. def jsonRDD(json: RDD[String]): SchemaRDD

    Definition Classes
    SQLContext
  65. def last(e: Expression): Last

    Definition Classes
    ExpressionConversions
  66. def loadTestTable(name: String): Unit

    Definition Classes
    TestHiveContext
  67. def log: Logger

    Attributes
    protected
    Definition Classes
    Logging
  68. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  69. def logDebug(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  70. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  71. def logError(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  72. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  73. def logInfo(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  74. def logName: String

    Attributes
    protected
    Definition Classes
    Logging
  75. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  76. def logTrace(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  77. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  78. def logWarning(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  79. implicit def logicalPlanToSparkQuery(plan: LogicalPlan): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @DeveloperApi()
  80. implicit def longToLiteral(l: Long): Literal

    Definition Classes
    ExpressionConversions
  81. def lower(e: Expression): Lower

    Definition Classes
    ExpressionConversions
  82. def max(e: Expression): Max

    Definition Classes
    ExpressionConversions
  83. lazy val metastorePath: String

    Definition Classes
    TestHiveContext
  84. def min(e: Expression): Min

    Definition Classes
    ExpressionConversions
  85. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  86. final def notify(): Unit

    Definition Classes
    AnyRef
  87. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  88. lazy val optimizer: Optimizer

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext
  89. val originalUdfs: Set[String]

    Records the UDFs present when the server starts, so we can delete ones that are created by tests.

    Records the UDFs present when the server starts, so we can delete ones that are created by tests.

    Attributes
    protected
    Definition Classes
    TestHiveContext
  90. lazy val outputBuffer: OutputStream { ... /* 4 definitions in type refinement */ }

    Attributes
    protected
    Definition Classes
    HiveContext
  91. def parquetFile(path: String): SchemaRDD

    Definition Classes
    SQLContext
  92. def parseSql(sql: String): LogicalPlan

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext
  93. val planner: SparkPlanner with HiveStrategies

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    HiveContext → SQLContext
  94. val prepareForExecution: RuleExecutor[SparkPlan] { val batches: List[this.Batch] }

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext
  95. def registerFunction[T](name: String, func: Function22[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  96. def registerFunction[T](name: String, func: Function21[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  97. def registerFunction[T](name: String, func: Function20[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  98. def registerFunction[T](name: String, func: Function19[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  99. def registerFunction[T](name: String, func: Function18[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  100. def registerFunction[T](name: String, func: Function17[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  101. def registerFunction[T](name: String, func: Function16[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  102. def registerFunction[T](name: String, func: Function15[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  103. def registerFunction[T](name: String, func: Function14[_, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  104. def registerFunction[T](name: String, func: Function13[_, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  105. def registerFunction[T](name: String, func: Function12[_, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  106. def registerFunction[T](name: String, func: Function11[_, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  107. def registerFunction[T](name: String, func: Function10[_, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  108. def registerFunction[T](name: String, func: Function9[_, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  109. def registerFunction[T](name: String, func: Function8[_, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  110. def registerFunction[T](name: String, func: Function7[_, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  111. def registerFunction[T](name: String, func: Function6[_, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  112. def registerFunction[T](name: String, func: Function5[_, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  113. def registerFunction[T](name: String, func: Function4[_, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  114. def registerFunction[T](name: String, func: Function3[_, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  115. def registerFunction[T](name: String, func: Function2[_, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  116. def registerFunction[T](name: String, func: Function1[_, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  117. def registerRDDAsTable(rdd: SchemaRDD, tableName: String): Unit

    Definition Classes
    SQLContext
  118. def registerTestTable(testTable: TestTable): HashMap[String, TestTable]

    Definition Classes
    TestHiveContext
  119. def reset(): Unit

    Resets the test instance by deleting any tables that have been created.

    Resets the test instance by deleting any tables that have been created. TODO: also clear out UDFs, views, etc.

    Definition Classes
    TestHiveContext
  120. def runHive(cmd: String, maxRows: Int = 1000): Seq[String]

    Execute the command using Hive and return the results as a sequence.

    Execute the command using Hive and return the results as a sequence. Each element in the sequence is one row.

    Attributes
    protected
    Definition Classes
    HiveContext
  121. def runSqlHive(sql: String): Seq[String]

    Runs the specified SQL query using Hive.

    Runs the specified SQL query using Hive.

    Definition Classes
    TestHiveContextHiveContext
  122. lazy val sessionState: SessionState

    SQLConf and HiveConf contracts:

    SQLConf and HiveConf contracts:

    1. reuse existing started SessionState if any 2. when the Hive session is first initialized, params in HiveConf will get picked up by the SQLConf. Additionally, any properties set by set() or a SET command inside sql() will be set in the SQLConf *as well as* in the HiveConf.

    Attributes
    protected[org.apache.spark.sql.hive]
    Definition Classes
    HiveContext
  123. def setConf(key: String, value: String): Unit

    Definition Classes
    HiveContext → SQLConf
  124. def setConf(props: Properties): Unit

    Definition Classes
    SQLConf
  125. val settings: Map[String, String]

    Attributes
    protected[org.apache.spark]
    Definition Classes
    SQLConf
  126. implicit def shortToLiteral(s: Short): Literal

    Definition Classes
    ExpressionConversions
  127. val sparkContext: SparkContext

    Definition Classes
    SQLContext
  128. def sql(sqlText: String): SchemaRDD

    Definition Classes
    HiveContext → SQLContext
  129. val sqlParser: SparkSQLParser

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext
  130. def sqrt(e: Expression): Sqrt

    Definition Classes
    ExpressionConversions
  131. implicit def stringToLiteral(s: String): Literal

    Definition Classes
    ExpressionConversions
  132. def sum(e: Expression): Sum

    Definition Classes
    ExpressionConversions
  133. def sumDistinct(e: Expression): SumDistinct

    Definition Classes
    ExpressionConversions
  134. implicit def symbolToUnresolvedAttribute(s: Symbol): UnresolvedAttribute

    Definition Classes
    ExpressionConversions
  135. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  136. def table(tableName: String): SchemaRDD

    Definition Classes
    SQLContext
  137. lazy val testTables: HashMap[String, TestTable]

    A list of test tables and the DDL required to initialize them.

    A list of test tables and the DDL required to initialize them. A test table is loaded on demand when a query are run against it.

    Definition Classes
    TestHiveContext
  138. val testTempDir: File

    Definition Classes
    TestHiveContext
  139. implicit def timestampToLiteral(t: Timestamp): Literal

    Definition Classes
    ExpressionConversions
  140. def toString(): String

    Definition Classes
    AnyRef → Any
  141. def uncacheTable(tableName: String): Unit

    Definition Classes
    CacheManager
  142. def upper(e: Expression): Upper

    Definition Classes
    ExpressionConversions
  143. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  144. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  145. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  146. lazy val warehousePath: String

    Definition Classes
    TestHiveContext

Deprecated Value Members

  1. def hiveql(hqlQuery: String): SchemaRDD

    Definition Classes
    HiveContext
    Annotations
    @deprecated
    Deprecated

    (Since version 1.1)

  2. def hql(hqlQuery: String): SchemaRDD

    Definition Classes
    HiveContext
    Annotations
    @deprecated
    Deprecated

    (Since version 1.1)

Inherited from TestHiveContext

Inherited from HiveContext

Inherited from SQLContext

Inherited from Serializable

Inherited from Serializable

Inherited from UDFRegistration

Inherited from ExpressionConversions

Inherited from CacheManager

Inherited from SQLConf

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Ungrouped