utils

Type Members

1. case class AvroDeserializerExpression(child: Expression, schemaAvroJson: String, darwinConfig: Option[Config], avoidReevaluation: Boolean = true) extends UnaryExpression with ExpectsInputTypes with Product with Serializable

   

   An Expression which deserializes a binary field encoded in Avro and returns the corresponding representation in Spark.

   An Expression which deserializes a binary field encoded in Avro and returns the corresponding representation in Spark.

   child

   the Expression containing the binary Avro to be deserialized

   schemaAvroJson

   the JSON representation of the Avro schema

   darwinConfig

   the configuration for the AvroSchemaManager

   avoidReevaluation

   this filed forces the Expression to be non-deterministic. Setting this to true measn that this expression is executed only once even though the Optimizer creates several copies of it (eg. it happens usually with CollapseProject if the value returned by the expression is used several times, as with a selection of fields of the struct returned). If you set this flag to false, the expression may be evaluated once for each occurence of ti you see in the physical plan.

2. case class AvroSerializerExpression extends UnaryExpression with ExpectsInputTypes with TimeZoneAwareExpression with Product with Serializable

   

3. case class CompressExpression(codecName: String, conf: HadoopConfiguration, _child: Expression) extends UnaryExpression with ExpectsInputTypes with Product with Serializable

   

4. case class DecodeUsingAvro[A](child: Expression, tag: ClassTag[A], schema: String, avroSchemaManager: () ⇒ AvroSchemaManager, fromGenericRecord: (GenericRecord) ⇒ A) extends UnaryExpression with NonSQLExpression with Product with Serializable

   

5. case class EncodeUsingAvro[A](child: Expression, schema: String, avroSchemaManager: () ⇒ AvroSchemaManager, toGenericRecord: (A) ⇒ GenericRecord) extends UnaryExpression with NonSQLExpression with Product with Serializable

   

6. class HBaseConnection extends GuardedLogging

   

7. class HadoopConfiguration extends Serializable

   

   Serializable wrapper for a Hadoop Configuration