Class/Object

io.github.sparkdataprocessing

DataProcessingApp

Related Docs: object DataProcessingApp | package sparkdataprocessing

Permalink

case class DataProcessingApp(appName: String, configurations: Set[DataProcessingSettings], mainStep: BaseStep) extends Product with Serializable

A DataPrepApp brings together the steps and the configurations.

appName

Name of the Application.

configurations

Set of configurations

Linear Supertypes
Serializable, Serializable, Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. DataProcessingApp
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new DataProcessingApp(appName: String, configuration: DataProcessingSettings, steps: BaseStep*)

    Permalink

    Alternative Constructor accepting one configuration and a variable list of steps

  2. new DataProcessingApp(appName: String, configuration: DataProcessingSettings, mainStep: BaseStep)

    Permalink

    Alternative Constructor accepting one configuration

  3. new DataProcessingApp(appName: String, configurations: Set[DataProcessingSettings], steps: BaseStep*)

    Permalink

    Alternative Constructor accepting a variable list of steps

  4. new DataProcessingApp(appName: String, configurations: Set[DataProcessingSettings], mainStep: BaseStep)

    Permalink

    appName

    Name of the Application.

    configurations

    Set of configurations

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. val appName: String

    Permalink

    Name of the Application.

  5. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  6. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  7. val configurations: Set[DataProcessingSettings]

    Permalink

    Set of configurations

  8. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  9. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  10. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  11. def getConfiguration(name: String): DataProcessingSettings

    Permalink

    Get a configuration

    Get a configuration

    name

    of the configuration

  12. def getDependencies(state: State): Dependencies

    Permalink

    Analyze the dependencies of this application

    Analyze the dependencies of this application

    state

    Assumes that for each step the state is cached

  13. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  14. val mainStep: BaseStep

    Permalink

  15. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  16. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  17. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  18. def parseArgs(args: Array[String]): DataProcessingSettings

    Permalink

  19. def run(args: Array[String]): State

    Permalink

    Run the Application

    Run the Application

    args

    Arguments containing the name of the Configuration to execute

  20. def run(): State

    Permalink

    Run the Application.

    Run the Application. Can only be used when exactly one configuration is available.

  21. def run(config: String): State

    Permalink

    Run the Application

    Run the Application

    config

    Name of the Configuration

  22. def run(conf: DataProcessingSettings): State

    Permalink

    Run the Application

    Run the Application

    conf

    Configuration

  23. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  24. def toString(): String

    Permalink
    Definition Classes
    DataProcessingApp → AnyRef → Any

  25. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  26. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  27. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped