definitions

sealed trait ExecutionMode extends AnyRef

Execution mode's defines how data is selected when running a data pipeline.
case class PartitionDiffMode(partitionColNb: Option[Int] = None, mainInputId: Option[String] = None, mainOutputId: Option[String] = None, nbOfPartitionValuesPerRun: Option[Int] = None) extends ExecutionMode with Product with Serializable

Partition difference execution mode lists partitions on input & output DataObject and starts loading all missing partitions.
Partition difference execution mode lists partitions on input & output DataObject and starts loading all missing partitions. Partition columns to be used for comparision need to be a common 'init' of input and output partition columns.
partitionColNb
optional number of partition columns to use as a common 'init'.
mainInputId
optional selection of inputId to be used for partition comparision. Only needed if there are multiple input DataObject's.
mainOutputId
optional selection of outputId to be used for partition comparision. Only needed if there are multiple output DataObject's.
nbOfPartitionValuesPerRun
optional restriction of the number of partition values per run.

object DateColumnType extends Enumeration

Datatype for date columns in Hive
object Environment

Set this environment dependent configurations at the beginning of the io.smartdatalake.app.SmartDataLakeBuilder implementation for your environment.
object HiveConventions

Hive conventions
object HiveTableLocationSuffix extends Enumeration

Suffix used for alternating parquet HDFS paths (usually in TickTockHiveTableDataObject for integration layer)
object OutputType extends Enumeration

Options for HDFS output
object TechnicalTableColumn extends Enumeration

Column names specific to historization of Hive tables