package internal
All classes in this package are considered an internal API to Spark and are subject to change between minor releases.
- Alphabetic
- By Inheritance
- internal
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Type Members
- abstract class BaseSessionStateBuilder extends AnyRef
Builder class that coordinates construction of a new SessionState.
Builder class that coordinates construction of a new SessionState.
The builder explicitly defines all components needed by the session state, and creates a session state when
buildis called. Components should only be initialized once. This is not a problem for most components as they are only used in thebuildfunction. However some components (conf,catalog,functionRegistry,experimentalMethods&sqlParser) are as dependencies for other components and are shared as a result. These components are defined as lazy vals to make sure the component is created only once.A developer can modify the builder by providing custom versions of components, or by using the hooks provided for the analyzer, optimizer & planner. There are some dependencies between the components (they are documented per dependency), a developer should respect these when making modifications in order to prevent initialization problems.
A parent SessionState can be used to initialize the new SessionState. The new session state will clone the parent sessions state's
conf,functionRegistry,experimentalMethodsandcatalogfields. Note that the state is cloned whenbuildis called, and not before.- Annotations
- @Unstable()
- case class HiveSerDe(inputFormat: Option[String] = None, outputFormat: Option[String] = None, serde: Option[String] = None) extends Product with Serializable
- class NonClosableMutableURLClassLoader extends MutableURLClassLoader
This class loader cannot be closed (its
closemethod is a no-op). - class SessionResourceLoader extends FunctionResourceLoader
Session shared FunctionResourceLoader.
Session shared FunctionResourceLoader.
- Annotations
- @Unstable()
- class SessionStateBuilder extends BaseSessionStateBuilder
Concrete implementation of a BaseSessionStateBuilder.
Concrete implementation of a BaseSessionStateBuilder.
- Annotations
- @Unstable()
- trait SessionStateHelper extends AnyRef
Helper trait to access session state related configurations and utilities.
Helper trait to access session state related configurations and utilities. It also provides type annotations for IDEs to build indexes.
- class SparkUDFExpressionBuilder extends FunctionExpressionBuilder
- class VariableSubstitution extends SQLConfHelper
A helper class that enables substitution using syntax like
${var},${system:var}and${env:var}.A helper class that enables substitution using syntax like
${var},${system:var}and${env:var}.Variable substitution is controlled by
SQLConf.variableSubstituteEnabled.
Value Members
- object HiveSerDe extends Serializable
- object SessionStateHelper extends SessionStateHelper
- object SharedState extends Logging