ExpressionEncoder

Instance Constructors

new ExpressionEncoder(schema: StructType, flat: Boolean, serializer: Seq[Expression], deserializer: Expression, clsTag: ClassTag[T])

schema
The schema after converting T to a Spark SQL row.
serializer
A set of expressions, one for each top-level field that can be used to extract the values from a raw object into an InternalRow.
deserializer
An expression that will construct an object given an InternalRow.
clsTag
A classtag for T.

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
def assertUnresolved(): Unit

The process of resolution to a given schema throws away information about where a given field is being bound by ordinal instead of by name.
The process of resolution to a given schema throws away information about where a given field is being bound by ordinal instead of by name. This method checks to make sure this process has not been done already in places where we plan to do later composition of encoders.
val attrs: Seq[String]

Attributes
protected
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
val clsTag: ClassTag[T]

A classtag for T.
A classtag for T.

Definition Classes
ExpressionEncoder → Encoder
val deserializer: Expression

An expression that will construct an object given an InternalRow.
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
val flat: Boolean
def fromRow(row: InternalRow): T

Returns an object of type T, extracting the required values from the provided row.
Returns an object of type T, extracting the required values from the provided row. Note that you must resolveAndBind an encoder to a specific schema before you can call this function.
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
def namedExpressions: Seq[NamedExpression]

Returns a new set (with unique ids) of NamedExpression that represent the serialized form of this object.
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
def resolveAndBind(attrs: Seq[Attribute] = schema.toAttributes, analyzer: Analyzer = SimpleAnalyzer): ExpressionEncoder[T]

Returns a new copy of this encoder, where the deserializer is resolved and bound to the given schema.
Returns a new copy of this encoder, where the deserializer is resolved and bound to the given schema.
Note that, ideally encoder is used as a container of serde expressions, the resolution and binding stuff should happen inside query framework. However, in some cases we need to use encoder as a function to do serialization directly(e.g. Dataset.collect), then we can use this method to do resolution and binding outside of query framework.
val schema: StructType

The schema after converting T to a Spark SQL row.
The schema after converting T to a Spark SQL row.

Definition Classes
ExpressionEncoder → Encoder
val schemaString: String

Attributes
protected
val serializer: Seq[Expression]

A set of expressions, one for each top-level field that can be used to extract the values from a raw object into an InternalRow.
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toRow(t: T): InternalRow

Returns an encoded version of t as a Spark SQL row.
Returns an encoded version of t as a Spark SQL row. Note that multiple calls to toRow are allowed to return the same actual InternalRow object. Thus, the caller should copy the result before making another call if required.
def toString(): String

Definition Classes
ExpressionEncoder → AnyRef → Any
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

Related Docs: object ExpressionEncoder | package encoders

case class ExpressionEncoder[T](schema: StructType, flat: Boolean, serializer: Seq[Expression], deserializer: Expression, clsTag: ClassTag[T]) extends Encoder[T] with Product with Serializable

Instance Constructors

new ExpressionEncoder(schema: StructType, flat: Boolean, serializer: Seq[Expression], deserializer: Expression, clsTag: ClassTag[T])

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

final def asInstanceOf[T0]: T0

def assertUnresolved(): Unit

val attrs: Seq[String]

def clone(): AnyRef

val clsTag: ClassTag[T]

val deserializer: Expression

final def eq(arg0: AnyRef): Boolean

def finalize(): Unit

val flat: Boolean

def fromRow(row: InternalRow): T

final def getClass(): Class[_]

final def isInstanceOf[T0]: Boolean

def namedExpressions: Seq[NamedExpression]

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

def resolveAndBind(attrs: Seq[Attribute] = schema.toAttributes, analyzer: Analyzer = SimpleAnalyzer): ExpressionEncoder[T]

val schema: StructType

val schemaString: String

val serializer: Seq[Expression]

final def synchronized[T0](arg0: ⇒ T0): T0

def toRow(t: T): InternalRow

def toString(): String

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Inherited from Product

Inherited from Equals

Inherited from Encoder[T]

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped