Class

com.vbounyasit.bigdata.testing.formats.json

JsonResourcesUpdater

Related Doc: package json

Permalink

abstract class JsonResourcesUpdater extends ResourcesUpdater

Linear Supertypes
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. JsonResourcesUpdater
  2. ResourcesUpdater
  3. LoggerProvider
  4. SparkSessionProvider
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new JsonResourcesUpdater()

    Permalink

Abstract Value Members

  1. abstract val sparkApplication: SparkApplication[_, _]

    Permalink

    The spark application to generate files for

    The spark application to generate files for

    Definition Classes
    ResourcesUpdater

Concrete Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  6. def createDefaultDataFrameFromConfigs(source: JobSource)(implicit spark: SparkSession): DataFrame

    Permalink

    Creates a default DataFrame based on the job source data

    Creates a default DataFrame based on the job source data

    source

    the source to create a dataFrame from

    spark

    the implicit spark session

    returns

    a DataFrame containing default rows and the right schema

    Definition Classes
    ResourcesUpdater
  7. final val dataFrameLoader: DataFrameLoader

    Permalink

    Our DataFrame loader and writer instances

    Our DataFrame loader and writer instances

    Definition Classes
    JsonResourcesUpdaterResourcesUpdater
  8. final val dataFrameWriter: DataFrameWriter

    Permalink
    Definition Classes
    JsonResourcesUpdaterResourcesUpdater
  9. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  10. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  11. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  12. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  13. final def getSparkSession(sparkParamsConf: SparkParamsConf): SparkSession

    Permalink
    Definition Classes
    SparkSessionProvider
  14. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  15. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  16. val logger: Logger

    Permalink
    Definition Classes
    LoggerProvider
  17. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  18. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  19. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  20. def runResourceUpdates(env: String = environment): Unit

    Permalink

    Main unit run method

    Main unit run method

    Definition Classes
    ResourcesUpdater
  21. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  22. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  23. def updateSources(sources: Seq[(JobTableMetadata, JobSource)])(implicit spark: SparkSession): Seq[History]

    Permalink

    Writing or updating source files in Resources folder

    Writing or updating source files in Resources folder

    sources

    the sources to update/create

    spark

    the implicit spark session

    returns

    A sequence of history objects to be recorded

    Definition Classes
    ResourcesUpdater
  24. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  25. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  26. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from ResourcesUpdater

Inherited from LoggerProvider

Inherited from SparkSessionProvider

Inherited from AnyRef

Inherited from Any

Ungrouped