Package

it.agilelab.bigdata.wasp.consumers.spark.plugins.parallel

model

Permalink

package model

Visibility
  1. Public
  2. All

Type Members

  1. case class ContinuousUpdate(tableName: String, keys: List[String], orderingExpression: String, fieldsToDrop: List[String]) extends WriterDetails with Product with Serializable

    Permalink

    Details needed by continuous update writer

    Details needed by continuous update writer

    tableName

    delta table name

    keys

    delta table unique keys column list

    orderingExpression

    monotonically increasing select expression to choose upsert candidate

    fieldsToDrop

    columns used in orderingExpression but not needed in delta table

  2. case class ParallelWrite(saveMode: String, partitionBy: Option[List[String]]) extends WriterDetails with Product with Serializable

    Permalink

    Details needeed by parallel writer

    Details needeed by parallel writer

    saveMode

    spark save mode

    partitionBy

    partition columns

  3. case class ParallelWriteModel(writerDetails: WriterDetails, entityDetails: Map[String, String]) extends Product with Serializable

    Permalink

    Model describing a parallel write destination

    Model describing a parallel write destination

    writerDetails

    details needed by writer

    entityDetails

    entity coordinates needed to retrieve entity endpoints

  4. sealed trait WriterDetails extends AnyRef

    Permalink

Value Members

  1. object ParallelWriteModelParser

    Permalink
  2. object WriterDetails

    Permalink

Ungrouped