GoldenGateAdapterFlatModelStrategy

Strategy that enable to map a flat mutation model to be mapped to an insert/update/delete object that can be sent to the CDC plugin that writes on DeltaLake. So having has input the raw flat mutations coming from a goldengate topic it will produce in output a dataframe composed of rows that has the shape accepted in input by the cdc plugin.

NB:

this strategy is used to map the mutation incoming from what in the oracle language is known as: Row Formatter, if you need to map a message that is incoming from an Operation Formatter you need to wait the new feature for that. More details are available under: operation vs row formatter
to enable the correct working of the strategy you need to ensure at runtime the configuration with path: goldengate.key.fields. This configuration is required and contains the list of primary keys fields for the mutation table. Suppose for example to have a table with the following structure: SHOP_TABLE ===> "PRODUCT_AMOUNT": Integer "TRANSACTION_ID": Integer "ORDER_DATE": Timestamp "PRODUCT_PRICE": Char "ORDER_ID": Integer "CUST_CODE": Long "PRODUCT_CODE": String and the primary key of this table is composed by the fields:

CUST_CODE
ORDER_DATE
PRODUCT_CODE
ORDER_ID

in this case you need to insert the configuration the following line:

goldengate.key.fields=["CUST_CODE", "ORDER_DATE", "PRODUCT_CODE", "ORDER_ID"]"

Linear Supertypes

Logging, Strategy, Serializable, Serializable, AnyRef, Any

Instance Constructors

new GoldenGateAdapterFlatModelStrategy()

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
val ALTER_DATE_DAY: String

Definition Classes
Strategy
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
var configuration: Config

Definition Classes
Strategy
def enrichFinalDf: (DataFrame) ⇒ DataFrame

Eventual enrichment function of the final DF.
Eventual enrichment function of the final DF.
returns
final output DF
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
def hashCode(): Int

Definition Classes
AnyRef → Any
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
val logger: WaspLogger

Attributes
protected
Definition Classes
Logging
var mlModelsBroadcast: MlModelsBroadcastDB

Definition Classes
Strategy
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
def prepareInitialDf: (DataFrame) ⇒ DataFrame

Eventual preparation function of the initial DF an example can be the removal of fields that are not required to be mapped in the table or a mapping with a default value.
Eventual preparation function of the initial DF an example can be the removal of fields that are not required to be mapped in the table or a mapping with a default value.
returns
final output DF
var sparkContext: Option[SparkContext]

Definition Classes
Strategy
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
Strategy → AnyRef → Any
def transform(dataFrames: Map[ReaderKey, DataFrame]): DataFrame

Strategy that read data from Kafka, transform the format to make it compliant with the DataLake format that the writer expect.
Strategy that read data from Kafka, transform the format to make it compliant with the DataLake format that the writer expect.
dataFrames
the dataframe that need to be transformed
returns
a dataframe transformed that can be sent to the CDCWriter

Definition Classes
GoldenGateAdapterFlatModelStrategy → Strategy
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

Related Doc: package cdc

class GoldenGateAdapterFlatModelStrategy extends Strategy with Logging

Instance Constructors

new GoldenGateAdapterFlatModelStrategy()

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

val ALTER_DATE_DAY: String

final def asInstanceOf[T0]: T0

def clone(): AnyRef

var configuration: Config

def enrichFinalDf: (DataFrame) ⇒ DataFrame

final def eq(arg0: AnyRef): Boolean

def equals(arg0: Any): Boolean

def finalize(): Unit

final def getClass(): Class[_]

def hashCode(): Int

final def isInstanceOf[T0]: Boolean

val logger: WaspLogger

var mlModelsBroadcast: MlModelsBroadcastDB

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

def prepareInitialDf: (DataFrame) ⇒ DataFrame

var sparkContext: Option[SparkContext]

final def synchronized[T0](arg0: ⇒ T0): T0

def toString(): String

def transform(dataFrames: Map[ReaderKey, DataFrame]): DataFrame

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Inherited from Logging

Inherited from Strategy

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped