case classAvroDeserializerExpression(child: Expression, schemaAvroJson: String, darwinConfig: Option[Config], avoidReevaluation: Boolean = true) extends UnaryExpression with ExpectsInputTypes with Product with Serializable
An Expression which deserializes a binary field encoded in Avro and returns the corresponding
representation in Spark.
An Expression which deserializes a binary field encoded in Avro and returns the corresponding
representation in Spark.
child
the Expression containing the binary Avro to be deserialized
schemaAvroJson
the JSON representation of the Avro schema
darwinConfig
the configuration for the AvroSchemaManager
avoidReevaluation
this filed forces the Expression to be non-deterministic. Setting this to true
measn that this expression is executed only once even though the Optimizer creates
several copies of it (eg. it happens usually with CollapseProject if the value
returned by the expression is used several times, as with a selection of fields
of the struct returned). If you set this flag to false, the expression may be evaluated
once for each occurence of ti you see in the physical plan.
case classAvroSerializerExpression extends UnaryExpression with ExpectsInputTypes with TimeZoneAwareExpression with Product with Serializable
case classCompressExpression(codecName: String, conf: HadoopConfiguration, _child: Expression) extends UnaryExpression with ExpectsInputTypes with Product with Serializable
case classDecodeUsingAvro[A](child: Expression, tag: ClassTag[A], schema: String, avroSchemaManager: () ⇒ AvroSchemaManager, fromGenericRecord: (GenericRecord) ⇒ A) extends UnaryExpression with NonSQLExpression with Product with Serializable
case classEncodeUsingAvro[A](child: Expression, schema: String, avroSchemaManager: () ⇒ AvroSchemaManager, toGenericRecord: (A) ⇒ GenericRecord) extends UnaryExpression with NonSQLExpression with Product with Serializable
An
Expression
which deserializes a binary field encoded in Avro and returns the corresponding representation in Spark.the
Expression
containing the binary Avro to be deserializedthe JSON representation of the Avro schema
the configuration for the AvroSchemaManager
this filed forces the Expression to be non-deterministic. Setting this to true measn that this expression is executed only once even though the Optimizer creates several copies of it (eg. it happens usually with CollapseProject if the value returned by the expression is used several times, as with a selection of fields of the struct returned). If you set this flag to false, the expression may be evaluated once for each occurence of ti you see in the physical plan.