This is the entry point for a Spark Job Server to execute Spark jobs.
This is the entry point for a Spark Job Server to execute Spark jobs. This function should create or reuse RDDs and return the result at the end, which the Job Server will cache or display.
a SparkContext or similar for the job. May be reused across jobs.
the job result
This method is called by the job server to allow jobs to validate their input and reject invalid job requests.
This method is called by the job server to allow jobs to validate their input and reject invalid job requests. If SparkJobInvalid is returned, then the job server returns 400 to the user. NOTE: this method should return very quickly. If it responds slowly then the job server may time out trying to start this job.
either SparkJobValid or SparkJobInvalid
A test job that accepts a SQLContext, as opposed to the regular SparkContext. Just initializes some dummy data into a table.