org.apache.spark.sql.internal

SQLConf

class SQLConf extends Serializable with Logging

A class that enables the setting and getting of mutable config parameters/hints.

In the presence of a SQLContext, these can be set and queried by passing SET commands into Spark SQL's query functions (i.e. sql()). Otherwise, users of this class can modify the hints by programmatically calling the setters and getters of this class.

SQLConf is thread-safe (internally synchronized, so safe to be used in multiple threads).

Linear Supertypes
Logging, Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. SQLConf
  2. Logging
  3. Serializable
  4. Serializable
  5. AnyRef
  6. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new SQLConf()

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any

  3. final def ##(): Int

    Definition Classes
    AnyRef → Any

  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any

  6. def adaptiveExecutionEnabled: Boolean

  7. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  8. def autoBroadcastJoinThreshold: Long

  9. def broadcastTimeout: Int

  10. def bucketingEnabled: Boolean

  11. def cartesianProductExecBufferSpillThreshold: Int

  12. def caseSensitiveAnalysis: Boolean

  13. def caseSensitiveInferenceMode: SQLConf.HiveCaseSensitiveInferenceMode.Value

  14. def cboEnabled: Boolean

  15. def checkpointLocation: Option[String]

  16. def clear(): Unit

  17. def clone(): SQLConf

    Definition Classes
    SQLConf → AnyRef

  18. def columnBatchSize: Int

  19. def columnNameOfCorruptRecord: String

  20. def constraintPropagationEnabled: Boolean

  21. def contains(key: String): Boolean

    Return whether a given key is set in this SQLConf.

  22. def convertCTAS: Boolean

  23. def copy(entries: (ConfigEntry[_], Any)*): SQLConf

  24. def crossJoinEnabled: Boolean

  25. def dataFramePivotMaxValues: Int

  26. def dataFrameRetainGroupColumns: Boolean

  27. def dataFrameSelfJoinAutoResolveAmbiguity: Boolean

  28. def defaultDataSourceName: String

  29. def defaultSizeInBytes: Long

  30. def enableRadixSort: Boolean

  31. def enableTwoLevelAggMap: Boolean

  32. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  33. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  34. def escapedStringLiterals: Boolean

  35. def exchangeReuseEnabled: Boolean

  36. def fallBackToHdfsForStatsEnabled: Boolean

  37. def fileCommitProtocolClass: String

  38. def fileSinkLogCleanupDelay: Long

  39. def fileSinkLogCompactInterval: Int

  40. def fileSinkLogDeletion: Boolean

  41. def fileSourceLogCleanupDelay: Long

  42. def fileSourceLogCompactInterval: Int

  43. def fileSourceLogDeletion: Boolean

  44. def filesMaxPartitionBytes: Long

  45. def filesOpenCostInBytes: Long

  46. def filesourcePartitionFileCacheSize: Long

  47. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  48. def gatherFastStats: Boolean

  49. def getAllConfs: Map[String, String]

    Return all the configuration properties that have been set (i.

    Return all the configuration properties that have been set (i.e. not the default). This creates a new copy of the config properties in the form of a Map.

  50. def getAllDefinedConfs: Seq[(String, String, String)]

    Return all the configuration definitions that have been defined in SQLConf.

    Return all the configuration definitions that have been defined in SQLConf. Each definition contains key, defaultValue and doc.

  51. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  52. def getConf[T](entry: OptionalConfigEntry[T]): Option[T]

    Return the value of an optional Spark SQL configuration property for the given key.

    Return the value of an optional Spark SQL configuration property for the given key. If the key is not set yet, returns None.

  53. def getConf[T](entry: ConfigEntry[T]): T

    Return the value of Spark SQL configuration property for the given key.

    Return the value of Spark SQL configuration property for the given key. If the key is not set yet, return defaultValue in ConfigEntry.

  54. def getConf[T](entry: ConfigEntry[T], defaultValue: T): T

    Return the value of Spark SQL configuration property for the given key.

    Return the value of Spark SQL configuration property for the given key. If the key is not set yet, return defaultValue. This is useful when defaultValue in ConfigEntry is not the desired one.

  55. def getConfString(key: String, defaultValue: String): String

    Return the string value of Spark SQL configuration property for the given key.

    Return the string value of Spark SQL configuration property for the given key. If the key is not set yet, return defaultValue.

  56. def getConfString(key: String): String

    Return the value of Spark SQL configuration property for the given key.

    Return the value of Spark SQL configuration property for the given key.

    Annotations
    @throws( "if key is not set" )

  57. def groupByAliases: Boolean

  58. def groupByOrdinal: Boolean

  59. def hashCode(): Int

    Definition Classes
    AnyRef → Any

  60. def hiveThriftServerSingleSession: Boolean

  61. def ignoreCorruptFiles: Boolean

  62. def inMemoryPartitionPruning: Boolean

  63. def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    Attributes
    protected
    Definition Classes
    Logging

  64. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  65. def isParquetBinaryAsString: Boolean

  66. def isParquetINT64AsTimestampMillis: Boolean

  67. def isParquetINT96AsTimestamp: Boolean

  68. def isParquetSchemaMergingEnabled: Boolean

  69. def isParquetSchemaRespectSummaries: Boolean

  70. def isTraceEnabled(): Boolean

    Attributes
    protected
    Definition Classes
    Logging

  71. def isUnsupportedOperationCheckEnabled: Boolean

  72. def joinReorderCardWeight: Double

  73. def joinReorderDPStarFilter: Boolean

  74. def joinReorderDPThreshold: Int

  75. def joinReorderEnabled: Boolean

  76. def limitScaleUpFactor: Int

  77. def log: Logger

    Attributes
    protected
    Definition Classes
    Logging

  78. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  79. def logDebug(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  80. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  81. def logError(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  82. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  83. def logInfo(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  84. def logName: String

    Attributes
    protected
    Definition Classes
    Logging

  85. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  86. def logTrace(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  87. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging

  88. def logWarning(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging

  89. def manageFilesourcePartitions: Boolean

  90. def maxCaseBranchesForCodegen: Int

  91. def maxNestedViewDepth: Int

  92. def maxRecordsPerFile: Long

  93. def metastorePartitionPruning: Boolean

  94. def minBatchesToRetain: Int

  95. def minNumPostShufflePartitions: Int

  96. def ndvMaxError: Double

  97. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  98. final def notify(): Unit

    Definition Classes
    AnyRef

  99. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  100. def numShufflePartitions: Int

  101. def objectAggSortBasedFallbackThreshold: Int

  102. def optimizerInSetConversionThreshold: Int

  103. def optimizerMaxIterations: Int

    ************************ Spark SQL Params/Hints *******************

  104. def optimizerMetadataOnly: Boolean

  105. def orcFilterPushDown: Boolean

  106. def orderByOrdinal: Boolean

  107. def parallelPartitionDiscoveryParallelism: Int

  108. def parallelPartitionDiscoveryThreshold: Int

  109. def parquetCacheMetadata: Boolean

  110. def parquetCompressionCodec: String

  111. def parquetFilterPushDown: Boolean

  112. def parquetOutputCommitterClass: String

  113. def parquetVectorizedReaderEnabled: Boolean

  114. def partitionColumnTypeInferenceEnabled: Boolean

  115. def preferSortMergeJoin: Boolean

  116. def resolver: (String, String) ⇒ Boolean

    Returns the Resolver for the current configuration, which can be used to determine if two identifiers are equal.

  117. def runSQLonFile: Boolean

  118. def sessionLocalTimeZone: String

  119. def setConf[T](entry: ConfigEntry[T], value: T): Unit

    Set the given Spark SQL configuration property.

  120. def setConf(props: Properties): Unit

    Set Spark SQL configuration properties.

  121. def setConfString(key: String, value: String): Unit

    Set the given Spark SQL configuration property using a string value.

  122. val settings: Map[String, String]

    Only low degree of contention is expected for conf, thus NOT using ConcurrentHashMap.

    Only low degree of contention is expected for conf, thus NOT using ConcurrentHashMap.

    Attributes
    protected[org.apache.spark]

  123. def sortMergeJoinExecBufferSpillThreshold: Int

  124. def starSchemaDetection: Boolean

  125. def starSchemaFTRatio: Double

  126. def stateStoreMinDeltasForSnapshot: Int

  127. def streamingFileCommitProtocolClass: String

  128. def streamingMetricsEnabled: Boolean

  129. def streamingNoDataProgressEventInterval: Long

  130. def streamingPollingDelay: Long

  131. def streamingProgressRetention: Int

  132. def streamingSchemaInference: Boolean

  133. def subexpressionEliminationEnabled: Boolean

  134. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  135. def tableRelationCacheSize: Int

  136. def targetPostShuffleInputSize: Long

  137. def toString(): String

    Definition Classes
    AnyRef → Any

  138. def unsetConf(entry: ConfigEntry[_]): Unit

  139. def unsetConf(key: String): Unit

  140. def useCompression: Boolean

  141. def useObjectHashAggregation: Boolean

  142. def variableSubstituteDepth: Int

  143. def variableSubstituteEnabled: Boolean

  144. def verifyPartitionPath: Boolean

  145. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  146. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  147. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  148. def warehousePath: String

  149. def wholeStageEnabled: Boolean

  150. def wholeStageFallback: Boolean

  151. def wholeStageMaxNumFields: Int

  152. def windowExecBufferSpillThreshold: Int

  153. def writeLegacyParquetFormat: Boolean

Inherited from Logging

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped