sparkutils
This class provides terasort-like capabilities but using dataframes and the parquet file format.
Terasort like benchmark that uses dataframes and parquet file formats for a more realistic test scenario.
Terasort like benchmark that uses dataframes and parquet file formats for a more realistic test scenario. "Generates a dataframe with the following schema: key:binary(10), value:string(78) record_count records of 100 Bytes each (before compression). They are key-value pairs"
This class provides terasort-like capabilities but using dataframes and the parquet file format. Compression is enabled by default but can be disabled. Created by Alex Bordei on 5/26/18.