Python example to create a SparkSession with the Delta SQL parser (PySpark doesn't pick up the
SQL conf "spark.sql.extensions" in Apache Spark 2.4.x, hence we need to activate it manually in
2.4.x. However, because SparkSession has been created and everything has been materialized, we
need to clone a new session to trigger the initialization. See SPARK-25003):
An extension for Spark SQL to activate Delta SQL parser to support Delta SQL grammar.
Scala example to create a
SparkSession
with the Delta SQL parser:Java example to create a
SparkSession
with the Delta SQL parser:Python example to create a
SparkSession
with the Delta SQL parser (PySpark doesn't pick up the SQL conf "spark.sql.extensions" in Apache Spark 2.4.x, hence we need to activate it manually in 2.4.x. However, becauseSparkSession
has been created and everything has been materialized, we need to clone a new session to trigger the initialization. See SPARK-25003):0.4.0