Class

com.ebiznext.comet.schema.handlers

AirflowLauncher

Related Doc: package handlers

Permalink

class AirflowLauncher extends LaunchHandler with StrictLogging

Airflow Launcher will submit a request for ingestion to Airflow using the REST API. The requested DAG must exist in Airflow first.

Linear Supertypes
StrictLogging, LaunchHandler, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. AirflowLauncher
  2. StrictLogging
  3. LaunchHandler
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new AirflowLauncher()

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  5. def bqload(workflow: IngestionWorkflow, config: BigQueryLoadConfig)(implicit settings: Settings): Boolean

    Permalink

    Load to BigQuery

    Load to BigQuery

    Definition Classes
    AirflowLauncher → LaunchHandler

  6. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  7. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  8. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  9. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  10. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  11. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  12. def index(workflow: IngestionWorkflow, config: IndexConfig)(implicit settings: Settings): Boolean

    Permalink

    Index into elasticsearch

    Index into elasticsearch

    Definition Classes
    AirflowLauncher → LaunchHandler

  13. def ingest(workflow: IngestionWorkflow, domain: Domain, schema: Schema, paths: List[Path])(implicit settings: Settings): Boolean

    Permalink

    Request the execution of the "comet-ingest" DAG in Airflow

    Request the execution of the "comet-ingest" DAG in Airflow

    domain

    : Domain to which belong this dataset

    schema

    : Schema of the dataset

    paths

    : absolute paths where the source datasets (JSON / CSV / ...) are located

    returns

    success if request accepted

    Definition Classes
    AirflowLauncher → LaunchHandler

  14. def ingest(workflow: IngestionWorkflow, domain: Domain, schema: Schema, path: Path)(implicit settings: Settings): Boolean

    Permalink

    Submit to the cron manager a single file for ingestion

    Submit to the cron manager a single file for ingestion

    domain

    : Domain to which belong this dataset

    schema

    : Schema of the dataset

    path

    : absolute path where the source dataset (JSON / CSV / ...) is located

    returns

    success / failure

    Definition Classes
    LaunchHandler

  15. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  16. def jdbcload(workflow: IngestionWorkflow, config: JdbcLoadConfig)(implicit settings: Settings): Boolean

    Permalink

    Load to JDBC sink

    Load to JDBC sink

    Definition Classes
    AirflowLauncher → LaunchHandler

  17. val logger: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    StrictLogging

  18. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  19. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  20. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  21. def post(url: String, command: String): Boolean

    Permalink
    Attributes
    protected

  22. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  23. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any

  24. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  25. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  26. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from StrictLogging

Inherited from LaunchHandler

Inherited from AnyRef

Inherited from Any

Ungrouped