Package net.sansa_stack.spark.cli.util

Class SimpleSparkCmdTemplate<T>

java.lang.Object
net.sansa_stack.spark.cli.util.SimpleSparkCmdTemplate<T>
All Implemented Interfaces:
Callable<T>
Direct Known Subclasses:
SimpleSparkCmdRdfTemplate
public abstract class SimpleSparkCmdTemplate<T> extends Object implements Callable<T>

  • Field Details

    • appName

      protected String appName

    • inputFiles

      protected List<String> inputFiles

    • sparkSessionBuilder

      protected org.apache.spark.sql.SparkSession.Builder sparkSessionBuilder

    • sparkSession

      protected org.apache.spark.sql.SparkSession sparkSession

    • hadoopConfiguration

      protected org.apache.hadoop.conf.Configuration hadoopConfiguration

    • sparkContext

      protected org.apache.spark.api.java.JavaSparkContext sparkContext

  • Constructor Details

    • SimpleSparkCmdTemplate

      public SimpleSparkCmdTemplate(String appName, List<String> inputFiles)

  • Method Details

    • initSparkSessionBuilder

      protected void initSparkSessionBuilder()

    • finalizeSparkSessionBuilder

      protected void finalizeSparkSessionBuilder()

    • processInputs

      protected void processInputs()

    • call

      public T call() throws Exception
      Specified by:
      call in interface Callable<T>
      Throws:
      Exception

    • process

      protected abstract void process() throws Exception
      Throws:
      Exception