Class

io.smartdatalake.app

SmartDataLakeBuilderConfig

Related Doc: package app

Permalink

case class SmartDataLakeBuilderConfig(feedSel: String = null, applicationName: Option[String] = None, configuration: Option[String] = None, master: Option[String] = None, deployMode: Option[String] = None, username: Option[String] = None, kerberosDomain: Option[String] = None, keytabPath: Option[File] = None, partitionValues: Option[Seq[PartitionValues]] = None, multiPartitionValues: Option[Seq[PartitionValues]] = None, parallelism: Int = 1, statePath: Option[String] = None, overrideJars: Option[Seq[String]] = None, test: Option[TestMode.Value] = None) extends Product with Serializable

This case class represents a default configuration for the App. It is populated by parsing command-line arguments. It also specifies default values.

feedSel

Regex pattern to select the feed to execute.

applicationName

Application name.

configuration

A configuration file or a directory containing configuration files.

master

The Spark master URL passed to SparkContext when in local mode.

deployMode

The Spark deploy mode passed to SparkContext when in local mode.

username

Kerberos user name (username@kerberosDomain) for local mode.

kerberosDomain

Kerberos domain (username@kerberosDomain) for local mode.

keytabPath

Path to Kerberos keytab file for local mode.

test

Run in test mode:

  • "config": validate configuration
  • "dry-run": execute "prepare" and "init" phase to check environment
Linear Supertypes
Serializable, Serializable, Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SmartDataLakeBuilderConfig
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new SmartDataLakeBuilderConfig(feedSel: String = null, applicationName: Option[String] = None, configuration: Option[String] = None, master: Option[String] = None, deployMode: Option[String] = None, username: Option[String] = None, kerberosDomain: Option[String] = None, keytabPath: Option[File] = None, partitionValues: Option[Seq[PartitionValues]] = None, multiPartitionValues: Option[Seq[PartitionValues]] = None, parallelism: Int = 1, statePath: Option[String] = None, overrideJars: Option[Seq[String]] = None, test: Option[TestMode.Value] = None)

    Permalink

    feedSel

    Regex pattern to select the feed to execute.

    applicationName

    Application name.

    configuration

    A configuration file or a directory containing configuration files.

    master

    The Spark master URL passed to SparkContext when in local mode.

    deployMode

    The Spark deploy mode passed to SparkContext when in local mode.

    username

    Kerberos user name (username@kerberosDomain) for local mode.

    kerberosDomain

    Kerberos domain (username@kerberosDomain) for local mode.

    keytabPath

    Path to Kerberos keytab file for local mode.

    test

    Run in test mode:

    • "config": validate configuration
    • "dry-run": execute "prepare" and "init" phase to check environment

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. val appName: String

    Permalink

  5. val applicationName: Option[String]

    Permalink

    Application name.

  6. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  7. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  8. val configuration: Option[String]

    Permalink

    A configuration file or a directory containing configuration files.

  9. val deployMode: Option[String]

    Permalink

    The Spark deploy mode passed to SparkContext when in local mode.

  10. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  11. val feedSel: String

    Permalink

    Regex pattern to select the feed to execute.

  12. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  13. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  14. def getPartitionValues: Option[Seq[PartitionValues]]

    Permalink

  15. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  16. val kerberosDomain: Option[String]

    Permalink

    Kerberos domain (username@kerberosDomain) for local mode.

  17. val keytabPath: Option[File]

    Permalink

    Path to Kerberos keytab file for local mode.

  18. val master: Option[String]

    Permalink

    The Spark master URL passed to SparkContext when in local mode.

  19. val multiPartitionValues: Option[Seq[PartitionValues]]

    Permalink

  20. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  21. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  22. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  23. val overrideJars: Option[Seq[String]]

    Permalink

  24. val parallelism: Int

    Permalink

  25. val partitionValues: Option[Seq[PartitionValues]]

    Permalink

  26. val statePath: Option[String]

    Permalink

  27. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  28. val test: Option[TestMode.Value]

    Permalink

    Run in test mode:

    Run in test mode:

    • "config": validate configuration
    • "dry-run": execute "prepare" and "init" phase to check environment

  29. val username: Option[String]

    Permalink

    Kerberos user name (username@kerberosDomain) for local mode.

  30. def validate(): Unit

    Permalink

  31. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  32. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  33. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped