com.krux.hyperion.objects.aws

AdpEmrActivity

Related Doc: package aws

case class AdpEmrActivity(id: String, name: Option[String], input: Option[AdpRef[AdpDataNode]], output: Option[AdpRef[AdpDataNode]], preStepCommand: Option[Seq[String]], postStepCommand: Option[Seq[String]], runsOn: AdpRef[AdpEmrCluster], step: Seq[String], dependsOn: Option[Seq[AdpRef[AdpActivity]]]) extends AdpDataPipelineAbstractObject with AdpActivity with Product with Serializable

Runs an Amazon EMR cluster.

AWS Data Pipeline uses a different format for steps than Amazon EMR, for example AWS Data Pipeline uses comma-separated arguments after the JAR name in the EmrActivity step field.

input

The input data source.

output

The location for the output

preStepCommand

Shell scripts to be run before any steps are run. To specify multiple scripts, up to 255, add multiple preStepCommand fields.

postStepCommand

Shell scripts to be run after all steps are finished. To specify multiple scripts, up to 255, add multiple postStepCommand fields.

runsOn

The Amazon EMR cluster to run this cluster.

step

One or more steps for the cluster to run. To specify multiple steps, up to 255, add multiple step fields. Use comma-separated arguments after the JAR name; for example, "s3://example-bucket/MyWork.jar,arg1,arg2,arg3".

Source
AdpActivities.scala
Linear Supertypes
Serializable, Serializable, Product, Equals, AdpActivity, AdpDataPipelineObject, AdpDataPipelineAbstractObject, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. AdpEmrActivity
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. AdpActivity
  7. AdpDataPipelineObject
  8. AdpDataPipelineAbstractObject
  9. AnyRef
  10. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new AdpEmrActivity(id: String, name: Option[String], input: Option[AdpRef[AdpDataNode]], output: Option[AdpRef[AdpDataNode]], preStepCommand: Option[Seq[String]], postStepCommand: Option[Seq[String]], runsOn: AdpRef[AdpEmrCluster], step: Seq[String], dependsOn: Option[Seq[AdpRef[AdpActivity]]])

    input

    The input data source.

    output

    The location for the output

    preStepCommand

    Shell scripts to be run before any steps are run. To specify multiple scripts, up to 255, add multiple preStepCommand fields.

    postStepCommand

    Shell scripts to be run after all steps are finished. To specify multiple scripts, up to 255, add multiple postStepCommand fields.

    runsOn

    The Amazon EMR cluster to run this cluster.

    step

    One or more steps for the cluster to run. To specify multiple steps, up to 255, add multiple step fields. Use comma-separated arguments after the JAR name; for example, "s3://example-bucket/MyWork.jar,arg1,arg2,arg3".

Value Members

  1. final def !=(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  4. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  5. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  6. val dependsOn: Option[Seq[AdpRef[AdpActivity]]]

    One or more references to other Activities that must reach the FINISHED state before this activity will start.

    One or more references to other Activities that must reach the FINISHED state before this activity will start.

    Definition Classes
    AdpEmrActivity → AdpActivity

  7. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  8. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  9. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  10. val id: String

    The ID of the object, IDs must be unique within a pipeline definition

    The ID of the object, IDs must be unique within a pipeline definition

    Definition Classes
    AdpEmrActivity → AdpDataPipelineObject → AdpDataPipelineAbstractObject

  11. val input: Option[AdpRef[AdpDataNode]]

    The input data source.

  12. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  13. val name: Option[String]

    The optional, user-defined label of the object.

    The optional, user-defined label of the object. If you do not provide a name for an object in a pipeline definition, AWS Data Pipeline automatically duplicates the value of id.

    Definition Classes
    AdpEmrActivity → AdpDataPipelineObject → AdpDataPipelineAbstractObject

  14. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  15. final def notify(): Unit

    Definition Classes
    AnyRef

  16. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  17. val output: Option[AdpRef[AdpDataNode]]

    The location for the output

  18. val postStepCommand: Option[Seq[String]]

    Shell scripts to be run after all steps are finished.

    Shell scripts to be run after all steps are finished. To specify multiple scripts, up to 255, add multiple postStepCommand fields.

  19. val preStepCommand: Option[Seq[String]]

    Shell scripts to be run before any steps are run.

    Shell scripts to be run before any steps are run. To specify multiple scripts, up to 255, add multiple preStepCommand fields.

  20. val runsOn: AdpRef[AdpEmrCluster]

    The Amazon EMR cluster to run this cluster.

    The Amazon EMR cluster to run this cluster.

    Definition Classes
    AdpEmrActivity → AdpActivity

  21. val step: Seq[String]

    One or more steps for the cluster to run.

    One or more steps for the cluster to run. To specify multiple steps, up to 255, add multiple step fields. Use comma-separated arguments after the JAR name; for example, "s3://example-bucket/MyWork.jar,arg1,arg2,arg3".

  22. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  23. val type: String

    The type of object.

    The type of object. Use one of the predefined AWS Data Pipeline object types.

    Definition Classes
    AdpEmrActivity → AdpDataPipelineObject

  24. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  25. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  26. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AdpActivity

Inherited from AdpDataPipelineObject

Inherited from AdpDataPipelineAbstractObject

Inherited from AnyRef

Inherited from Any

Ungrouped