The data type for collections of multiple values.
The data type for collections of multiple values.
Internally these are represented as columns that contain a
.
scala.collection.Seq
The data type of values.
Indicates if values have null
values
A Decimal that might have fixed precision and scale, or unlimited values for these
The data type for Maps.
The data type for Maps. Keys in a map are not allowed to have null
values.
The data type of map keys.
The data type of map values.
Indicates if map values have null
values.
Precision parameters for a Decimal
A field inside a StructType.
A field inside a StructType.
The name of this field.
The data type of this field.
Indicates if values of this field can be null
values.
The metadata of this field. The metadata should be preserved during transformation if the content of the column is not modified, e.g, in selection.
::DeveloperApi:: The data type for User Defined Types (UDTs).
::DeveloperApi:: The data type for User Defined Types (UDTs).
This interface allows a user to make their own classes more interoperable with SparkSQL; e.g., by creating a UserDefinedType for a class X, it becomes possible to create a SchemaRDD which has class X in the schema.
For SparkSQL to recognize UDTs, the UDT must be annotated with org.apache.spark.sql.catalyst.annotation.SQLUserDefinedType.
The conversion via serialize
occurs when instantiating a SchemaRDD
from another RDD.
The conversion via deserialize
occurs when reading from a SchemaRDD
.
Extra factory methods and pattern matchers for Decimals
Matcher for any expressions that evaluate to FractionalTypes
Matcher for any expressions that evaluate to IntegralTypes
Contains a type system for attributes produced by relations, including complex types like structs, arrays and maps.