package internal
All classes in this package are considered an internal API to Spark and are subject to change between minor releases.
- Alphabetic
- By Inheritance
- internal
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Type Members
- abstract class BaseSessionStateBuilder extends AnyRef
Builder class that coordinates construction of a new SessionState.
Builder class that coordinates construction of a new SessionState.
The builder explicitly defines all components needed by the session state, and creates a session state when
build
is called. Components should only be initialized once. This is not a problem for most components as they are only used in thebuild
function. However some components (conf
,catalog
,functionRegistry
,experimentalMethods
&sqlParser
) are as dependencies for other components and are shared as a result. These components are defined as lazy vals to make sure the component is created only once.A developer can modify the builder by providing custom versions of components, or by using the hooks provided for the analyzer, optimizer & planner. There are some dependencies between the components (they are documented per dependency), a developer should respect these when making modifications in order to prevent initialization problems.
A parent SessionState can be used to initialize the new SessionState. The new session state will clone the parent sessions state's
conf
,functionRegistry
,experimentalMethods
andcatalog
fields. Note that the state is cloned whenbuild
is called, and not before.- Annotations
- @Unstable()
- class CatalogImpl extends Catalog
Internal implementation of the user-facing
Catalog
. - case class HiveSerDe(inputFormat: Option[String] = None, outputFormat: Option[String] = None, serde: Option[String] = None) extends Product with Serializable
- class NonClosableMutableURLClassLoader extends MutableURLClassLoader
This class loader cannot be closed (its
close
method is a no-op). - class SessionResourceLoader extends FunctionResourceLoader
Session shared FunctionResourceLoader.
Session shared FunctionResourceLoader.
- Annotations
- @Unstable()
- class SessionStateBuilder extends BaseSessionStateBuilder
Concrete implementation of a BaseSessionStateBuilder.
Concrete implementation of a BaseSessionStateBuilder.
- Annotations
- @Unstable()
- class SparkUDFExpressionBuilder extends FunctionExpressionBuilder
- class VariableSubstitution extends SQLConfHelper
A helper class that enables substitution using syntax like
${var}
,${system:var}
and${env:var}
.A helper class that enables substitution using syntax like
${var}
,${system:var}
and${env:var}
.Variable substitution is controlled by
SQLConf.variableSubstituteEnabled
.
Value Members
- object HiveSerDe extends Serializable
- object SharedState extends Logging