com.krux.hyperion.activity

PigActivity

case class PigActivity[A <: EmrCluster] extends EmrTaskActivity[A] with Product with Serializable

PigActivity provides native support for Pig scripts in AWS Data Pipeline without the requirement to use ShellCommandActivity or EmrActivity. In addition, PigActivity supports data staging. When the stage field is set to true, AWS Data Pipeline stages the input data as a schema in Pig without additional code from the user.

Source
PigActivity.scala
Linear Supertypes
Serializable, Serializable, Product, Equals, EmrTaskActivity[A], EmrActivity[A], PipelineActivity[A], NamedPipelineObject, PipelineObject, Ordered[PipelineObject], Comparable[PipelineObject], AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. PigActivity
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. EmrTaskActivity
  7. EmrActivity
  8. PipelineActivity
  9. NamedPipelineObject
  10. PipelineObject
  11. Ordered
  12. Comparable
  13. AnyRef
  14. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Type Members

  1. type Self = PigActivity[A]

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. def <(that: PipelineObject): Boolean

    Definition Classes
    Ordered
  5. def <=(that: PipelineObject): Boolean

    Definition Classes
    Ordered
  6. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  7. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  8. def >(that: PipelineObject): Boolean

    Definition Classes
    Ordered
  9. def >=(that: PipelineObject): Boolean

    Definition Classes
    Ordered
  10. val activityFields: ActivityFields[A]

    Definition Classes
    PigActivityPipelineActivity
  11. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  12. def attemptTimeout: Option[HDuration]

    Definition Classes
    PipelineActivity
  13. val baseFields: BaseFields

    Definition Classes
    PigActivityNamedPipelineObject
  14. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  15. def compare(that: PipelineObject): Int

    Definition Classes
    PipelineObject → Ordered
  16. def compareTo(that: PipelineObject): Int

    Definition Classes
    Ordered → Comparable
  17. def dependsOn: Seq[PipelineActivity[_]]

    Definition Classes
    PipelineActivity
  18. val emrTaskActivityFields: EmrTaskActivityFields

    Definition Classes
    PigActivityEmrTaskActivity
  19. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  20. def failureAndRerunMode: Option[FailureAndRerunMode]

    Definition Classes
    PipelineActivity
  21. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  22. val generatedScriptsPath: Option[HS3Uri]

  23. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  24. def groupedBy(group: String): Self

    Postfix the name field

    Postfix the name field

    Definition Classes
    NamedPipelineObject
  25. def id: PipelineObjectId

    Definition Classes
    NamedPipelineObjectPipelineObject
  26. def idGroupedBy(group: String): Self

    Have a grouping postfix in the id field

    Have a grouping postfix in the id field

    Definition Classes
    NamedPipelineObject
    Note

    Id naming is more restrictive, it is recommended to not changing the id unleass you have a good reason

  27. def idNamed(namePrefix: String): Self

    Id field will be prefixed with name

    Id field will be prefixed with name

    Definition Classes
    NamedPipelineObject
    Note

    Id naming is more restrictive, it is recommended to not changing the id unless you have a good reason

  28. val input: Option[DataNode]

  29. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  30. def lateAfterTimeout: Option[HDuration]

    Definition Classes
    PipelineActivity
  31. def maximumRetries: Option[HInt]

    Definition Classes
    PipelineActivity
  32. def name: Option[String]

    Name of the pipeline object, if not set, it will defaults to

    Name of the pipeline object, if not set, it will defaults to

    Option(id)
    Definition Classes
    NamedPipelineObject
  33. def named(namePrefix: String): Self

    Give the object a name prefix

    Give the object a name prefix

    Definition Classes
    NamedPipelineObject
  34. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  35. final def notify(): Unit

    Definition Classes
    AnyRef
  36. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  37. def objects: Iterable[PipelineObject]

  38. def onFail(alarms: SnsAlarm*): Self

    Definition Classes
    PipelineActivity
  39. def onFailAlarms: Seq[SnsAlarm]

    Definition Classes
    PipelineActivity
  40. def onLateAction(alarms: SnsAlarm*): Self

    Definition Classes
    PipelineActivity
  41. def onLateActionAlarms: Seq[SnsAlarm]

    Definition Classes
    PipelineActivity
  42. def onSuccess(alarms: SnsAlarm*): Self

    Definition Classes
    PipelineActivity
  43. def onSuccessAlarms: Seq[SnsAlarm]

    Definition Classes
    PipelineActivity
  44. val output: Option[DataNode]

  45. def postActivityTaskConfig: Option[ShellScriptConfig]

    Definition Classes
    EmrTaskActivity
  46. def preActivityTaskConfig: Option[ShellScriptConfig]

    Definition Classes
    EmrTaskActivity
  47. def preconditions: Seq[Precondition]

    Definition Classes
    PipelineActivity
  48. def ref: AdpRef[AdpActivity]

    Definition Classes
    PipelineActivityPipelineObject
  49. def retryDelay: Option[HDuration]

    Definition Classes
    PipelineActivity
  50. def runsOn: Resource[A]

    Definition Classes
    PipelineActivity
  51. val script: Script

  52. val scriptVariables: Seq[HString]

  53. implicit def seq2Option[A](anySeq: Seq[A]): Option[Seq[A]]

    Definition Classes
    PipelineObject
  54. def seqToOption[A, B](anySeq: Seq[A])(transform: (A) ⇒ B): Option[Seq[B]]

    Definition Classes
    PipelineObject
  55. lazy val serialize: AdpPigActivity

    Definition Classes
    PigActivityPipelineActivityPipelineObject
  56. val stage: Option[HBoolean]

  57. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  58. implicit def uniquePipelineId2String(id: PipelineObjectId): String

    Definition Classes
    PipelineObject
  59. def updateActivityFields(fields: ActivityFields[A]): PigActivity[A]

    Definition Classes
    PigActivityPipelineActivity
  60. def updateBaseFields(fields: BaseFields): PigActivity[A]

    Definition Classes
    PigActivityNamedPipelineObject
  61. def updateEmrTaskActivityFields(fields: EmrTaskActivityFields): PigActivity[A]

    Definition Classes
    PigActivityEmrTaskActivity
  62. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  63. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  64. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  65. def whenMet(conditions: Precondition*): Self

    Definition Classes
    PipelineActivity
  66. def withAttemptTimeout(duration: HDuration): Self

    Definition Classes
    PipelineActivity
  67. def withFailureAndRerunMode(mode: FailureAndRerunMode): Self

    Definition Classes
    PipelineActivity
  68. def withGeneratedScriptsPath(generatedScriptsPath: HS3Uri): PigActivity[A]

  69. def withInput(in: DataNode): PigActivity[A]

  70. def withLateAfterTimeout(duration: HDuration): Self

    Definition Classes
    PipelineActivity
  71. def withMaximumRetries(retries: HInt): Self

    Definition Classes
    PipelineActivity
  72. def withOutput(out: DataNode): PigActivity[A]

  73. def withPostActivityTaskConfig(config: ShellScriptConfig): Self

    Definition Classes
    EmrTaskActivity
  74. def withPreActivityTaskConfig(config: ShellScriptConfig): Self

    Definition Classes
    EmrTaskActivity
  75. def withRetryDelay(duration: HDuration): Self

    Definition Classes
    PipelineActivity
  76. def withScriptVariable(scriptVariable: HString*): PigActivity[A]

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from EmrTaskActivity[A]

Inherited from EmrActivity[A]

Inherited from PipelineActivity[A]

Inherited from NamedPipelineObject

Inherited from PipelineObject

Inherited from Ordered[PipelineObject]

Inherited from Comparable[PipelineObject]

Inherited from AnyRef

Inherited from Any

Ungrouped