Loads a dataframe from Dataset[String].
Loads a dataframe from Dataset[String].
See UnderlyingDataFrameReader.csv for more information.
Loads a dataframe from CSV files.
Loads a dataframe from CSV files.
See UnderlyingDataFrameReader.csv for more information.
Loads a dataframe from a CSV file.
Loads a dataframe from a CSV file.
See UnderlyingDataFrameReader.csv for more information.
Adds an option to say that spark should infer the schema.
Construct a DataFrame
representing the database table accessible
via JDBC.
Construct a DataFrame
representing the database table accessible
via JDBC.
See UnderlyingDataFrameReader.jdbc for more information.
Construct a DataFrame
representing the database table accessible
via JDBC.
Construct a DataFrame
representing the database table accessible
via JDBC.
See UnderlyingDataFrameReader.jdbc for more information.
Construct a DataFrame
representing the database table accessible
via JDBC.
Construct a DataFrame
representing the database table accessible
via JDBC.
See UnderlyingDataFrameReader.jdbc for more information.
Loads a dataframe from a Dataset[String].
Loads a dataframe from a Dataset[String].
See UnderlyingDataFrameReader.json for more information.
Loads a dataframe from JSON files.
Loads a dataframe from JSON files.
See UnderlyingDataFrameReader.json for more information.
Loads a dataframe from a JSON file.
Loads a dataframe from a JSON file.
See UnderlyingDataFrameReader.json for more information.
Loads a dataframe using one of the dataframe loader.
Adds an option to the DataFrameReader.
Adds an option to the DataFrameReader.
Adds an option to the DataFrameReader.
Adds an option to the DataFrameReader.
Adds an option to the DataFrameReader.
Adds multiple options to the DataFrameReader.
Loads a dataframe from ORC files.
Loads a dataframe from ORC files.
See UnderlyingDataFrameReader.orc for more information.
Loads a dataframe from a ORC file.
Loads a dataframe from a ORC file.
See UnderlyingDataFrameReader.orc for more information.
Loads a dataframe from PARQUET files.
Loads a dataframe from PARQUET files.
See UnderlyingDataFrameReader.parquet for more information.
Loads a dataframe from a PARQUET file.
Loads a dataframe from a PARQUET file.
See UnderlyingDataFrameReader.parquet for more information.
ZIO-Spark specifics function to generate the schema from a case class.
ZIO-Spark specifics function to generate the schema from a case class.
E.g.:
import zio.spark._ case class Person(name: String, age: Int) val ds: Dataset[Person] = SparkSession.read.schema[Person].csv("./path.csv").as[Person].getOrThrow
Replace the data schema of the DataFrameReader.
Replace the data schema of the DataFrameReader.
We advice you to always use a schema even if some data sources can infer the schema. It allows you to increase your job speed, it ensures that the schema is the expected one and it is useful as documentation.
E.g.:
schema("a INT, b STRING, c DOUBLE").csv("test.csv")
See UnderlyingDataFrameReader.schema for more information.
Replace the data schema of the DataFrameReader.
Replace the data schema of the DataFrameReader.
We advice you to always use a schema even if some data sources can infer the schema. It allows you to increase your job speed, it ensures that the schema is the expected one and it is useful as documentation.
See UnderlyingDataFrameReader.schema for more information.
Returns the specified table/view as a DataFrame
.
Returns the specified table/view as a DataFrame
.
See UnderlyingDataFrameReader.table for more information.
Loads a dataframe from text files.
Loads a dataframe from text files.
The underlying schema of the Dataset contains a single string column named "value". The text files must be encoded as UTF-8.
See UnderlyingDataFrameReader.textFile for more information.
Loads a dataframe from a text file.
Loads a dataframe from a text file.
The underlying schema of the Dataset contains a single string column named "value". The text files must be encoded as UTF-8.
See UnderlyingDataFrameReader.textFile for more information.
Loads a dataset[String] from text files.
Loads a dataset[String] from text files.
See UnderlyingDataFrameReader.textFile for more information.
Loads a dataset[String] from a text file.
Loads a dataset[String] from a text file.
See UnderlyingDataFrameReader.textFile for more information.
Adds an option to delimit the column from a csv file.
Adds an option to say that the file has a header.