Package

com.ebiznext.comet

workflow

Permalink

package workflow

Visibility
  1. Public
  2. All

Type Members

  1. class IngestionWorkflow extends StrictLogging

    Permalink

    The whole worklfow works as follow :

    The whole worklfow works as follow :

    • loadLanding : Zipped files are uncompressed or raw files extracted from the local filesystem. -loadPending : files recognized with filename patterns are stored in the ingesting area and submitted for ingestion files with unrecognized filename patterns are stored in the unresolved area
    • ingest : files are finally ingested and saved as parquet/orc/... files and hive tables

Ungrouped