Previously ColumnGenerator.
ColumnGenerator - prevously Column; it is now the base class for all ColumnGenerators.
ColumnList allows users to specify custom generators for a list of columns inside a StructType column.
:: Experimental :: Base class for testing Spark DataFrames.
This is the base trait for Spark Streaming testsuite.
Manages a local sc
SparkContext
variable,
correctly stopping it after each test.
Provides a local sc
SparkContext
variable, correctly stopping it after each test.
This listener collects basic execution time information to be used in micro type performance tests.
Shares an HDFS MiniCluster based SparkContext
between all tests in a suite and
closes it at the end.
Shares a local SparkContext
between all tests in a suite
and closes it at the end.
Methods for testing Spark actions.
This is the base trait for Spark Streaming testsuites.
This is a input stream just for the testsuites.
This is a output stream just for testing.
Shares an HDFS MiniCluster based SparkContext
between all tests in a suite and
closes it at the end.
Compatibility shim for the extractor that matches the UDTs exposed by Spark ML.