Class/Object

io.smartdatalake.app

GlobalConfig

Related Docs: object GlobalConfig | package app

Permalink

case class GlobalConfig(kryoClasses: Option[Seq[String]] = None, sparkOptions: Option[Map[String, String]] = None, enableHive: Boolean = true, memoryLogTimer: Option[MemoryLogTimerConfig] = None, shutdownHookLogger: Boolean = false, stateListeners: Seq[StateListenerConfig] = Seq()) extends SmartDataLakeLogger with Product with Serializable

Global configuration options

kryoClasses

classes to register for spark kryo serialization

sparkOptions

spark options

enableHive

enable hive for spark session

memoryLogTimer

enable periodic memory usage logging, see detailled configuration MemoryLogTimerConfig

shutdownHookLogger

enable shutdown hook logger to trace shutdown cause

Linear Supertypes
Serializable, Serializable, Product, Equals, SmartDataLakeLogger, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. GlobalConfig
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. SmartDataLakeLogger
  7. AnyRef
  8. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new GlobalConfig(kryoClasses: Option[Seq[String]] = None, sparkOptions: Option[Map[String, String]] = None, enableHive: Boolean = true, memoryLogTimer: Option[MemoryLogTimerConfig] = None, shutdownHookLogger: Boolean = false, stateListeners: Seq[StateListenerConfig] = Seq())

    Permalink

    kryoClasses

    classes to register for spark kryo serialization

    sparkOptions

    spark options

    enableHive

    enable hive for spark session

    memoryLogTimer

    enable periodic memory usage logging, see detailled configuration MemoryLogTimerConfig

    shutdownHookLogger

    enable shutdown hook logger to trace shutdown cause

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  6. def createSparkSession(appName: String, master: Option[String], deployMode: Option[String] = None): SparkSession

    Permalink

    Create a spark session using settings from this global config

  7. val enableHive: Boolean

    Permalink

    enable hive for spark session

  8. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  9. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  10. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  11. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  12. val kryoClasses: Option[Seq[String]]

    Permalink

    classes to register for spark kryo serialization

  13. lazy val logger: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    SmartDataLakeLogger

  14. val memoryLogTimer: Option[MemoryLogTimerConfig]

    Permalink

    enable periodic memory usage logging, see detailled configuration MemoryLogTimerConfig

  15. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  16. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  17. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  18. val shutdownHookLogger: Boolean

    Permalink

    enable shutdown hook logger to trace shutdown cause

  19. val sparkOptions: Option[Map[String, String]]

    Permalink

    spark options

  20. val stateListeners: Seq[StateListenerConfig]

    Permalink

  21. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  22. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  23. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  24. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from SmartDataLakeLogger

Inherited from AnyRef

Inherited from Any

Ungrouped