com.krux.hyperion.activity

PigActivity

case class PigActivity[A <: EmrCluster] extends EmrTaskActivity[A] with Product with Serializable

PigActivity provides native support for Pig scripts in AWS Data Pipeline without the requirement to use ShellCommandActivity or EmrActivity. In addition, PigActivity supports data staging. When the stage field is set to true, AWS Data Pipeline stages the input data as a schema in Pig without additional code from the user.

Source
PigActivity.scala
Linear Supertypes
Serializable, Serializable, Product, Equals, EmrTaskActivity[A], EmrActivity[A], PipelineActivity[A], NamedPipelineObject, PipelineObject, Ordered[PipelineObject], Comparable[PipelineObject], AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. PigActivity
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. EmrTaskActivity
  7. EmrActivity
  8. PipelineActivity
  9. NamedPipelineObject
  10. PipelineObject
  11. Ordered
  12. Comparable
  13. AnyRef
  14. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Type Members

  1. type Self = PigActivity[A]

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. def <(that: PipelineObject): Boolean

    Definition Classes
    Ordered
  5. def <=(that: PipelineObject): Boolean

    Definition Classes
    Ordered
  6. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  7. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  8. def >(that: PipelineObject): Boolean

    Definition Classes
    Ordered
  9. def >=(that: PipelineObject): Boolean

    Definition Classes
    Ordered
  10. val activityFields: ActivityFields[A]

    Definition Classes
    PigActivityPipelineActivity
  11. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  12. def attemptTimeout: Option[HDuration]

    Definition Classes
    PipelineActivity
  13. val baseFields: BaseFields

    Definition Classes
    PigActivityNamedPipelineObject
  14. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  15. def compare(that: PipelineObject): Int

    Definition Classes
    PipelineObject → Ordered
  16. def compareTo(that: PipelineObject): Int

    Definition Classes
    Ordered → Comparable
  17. def dependsOn: Seq[PipelineActivity[_]]

    Definition Classes
    PipelineActivity
  18. val emrTaskActivityFields: EmrTaskActivityFields

    Definition Classes
    PigActivityEmrTaskActivity
  19. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  20. def failureAndRerunMode: Option[FailureAndRerunMode]

    Definition Classes
    PipelineActivity
  21. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  22. val generatedScriptsPath: Option[HS3Uri]

  23. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  24. def groupedBy(group: String): Self

    Postfix the name field

    Postfix the name field

    Definition Classes
    NamedPipelineObject
  25. def id: PipelineObjectId

    Definition Classes
    NamedPipelineObjectPipelineObject
  26. def idGroupedBy(group: String): Self

    Have a grouping postfix in the id field

    Have a grouping postfix in the id field

    Definition Classes
    NamedPipelineObject
    Note

    Id naming is more restrictive, it is recommended to not changing the id unleass you have a good reason

  27. def idNamed(namePrefix: String): Self

    Id field will be prefixed with name

    Id field will be prefixed with name

    Definition Classes
    NamedPipelineObject
    Note

    Id naming is more restrictive, it is recommended to not changing the id unless you have a good reason

  28. val input: Option[DataNode]

  29. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  30. def lateAfterTimeout: Option[HDuration]

    Definition Classes
    PipelineActivity
  31. def maxActiveInstances: Option[HInt]

    Definition Classes
    PipelineActivity
  32. def maximumRetries: Option[HInt]

    Definition Classes
    PipelineActivity
  33. def name: Option[String]

    Name of the pipeline object, if not set, it will defaults to

    Name of the pipeline object, if not set, it will defaults to

    Option(id)
    Definition Classes
    NamedPipelineObject
  34. def named(namePrefix: String): Self

    Give the object a name prefix

    Give the object a name prefix

    Definition Classes
    NamedPipelineObject
  35. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  36. final def notify(): Unit

    Definition Classes
    AnyRef
  37. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  38. def objects: Iterable[PipelineObject]

  39. def onFail(alarms: SnsAlarm*): Self

    Definition Classes
    PipelineActivity
  40. def onFailAlarms: Seq[SnsAlarm]

    Definition Classes
    PipelineActivity
  41. def onLateAction(alarms: SnsAlarm*): Self

    Definition Classes
    PipelineActivity
  42. def onLateActionAlarms: Seq[SnsAlarm]

    Definition Classes
    PipelineActivity
  43. def onSuccess(alarms: SnsAlarm*): Self

    Definition Classes
    PipelineActivity
  44. def onSuccessAlarms: Seq[SnsAlarm]

    Definition Classes
    PipelineActivity
  45. val output: Option[DataNode]

  46. def postActivityTaskConfig: Option[ShellScriptConfig]

    Definition Classes
    EmrTaskActivity
  47. def preActivityTaskConfig: Option[ShellScriptConfig]

    Definition Classes
    EmrTaskActivity
  48. def preconditions: Seq[Precondition]

    Definition Classes
    PipelineActivity
  49. def ref: AdpRef[AdpActivity]

    Definition Classes
    PipelineActivityPipelineObject
  50. def retryDelay: Option[HDuration]

    Definition Classes
    PipelineActivity
  51. def runsOn: Resource[A]

    Definition Classes
    PipelineActivity
  52. val script: Script

  53. val scriptVariables: Seq[HString]

  54. implicit def seq2Option[A](anySeq: Seq[A]): Option[Seq[A]]

    Definition Classes
    PipelineObject
  55. def seqToOption[A, B](anySeq: Seq[A])(transform: (A) ⇒ B): Option[Seq[B]]

    Definition Classes
    PipelineObject
  56. lazy val serialize: AdpPigActivity

    Definition Classes
    PigActivityPipelineActivityPipelineObject
  57. val stage: Option[HBoolean]

  58. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  59. implicit def uniquePipelineId2String(id: PipelineObjectId): String

    Definition Classes
    PipelineObject
  60. def updateActivityFields(fields: ActivityFields[A]): PigActivity[A]

    Definition Classes
    PigActivityPipelineActivity
  61. def updateBaseFields(fields: BaseFields): PigActivity[A]

    Definition Classes
    PigActivityNamedPipelineObject
  62. def updateEmrTaskActivityFields(fields: EmrTaskActivityFields): PigActivity[A]

    Definition Classes
    PigActivityEmrTaskActivity
  63. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  64. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  65. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  66. def whenMet(conditions: Precondition*): Self

    Definition Classes
    PipelineActivity
  67. def withAttemptTimeout(duration: HDuration): Self

    Definition Classes
    PipelineActivity
  68. def withFailureAndRerunMode(mode: FailureAndRerunMode): Self

    Definition Classes
    PipelineActivity
  69. def withGeneratedScriptsPath(generatedScriptsPath: HS3Uri): PigActivity[A]

  70. def withInput(in: DataNode): PigActivity[A]

  71. def withLateAfterTimeout(duration: HDuration): Self

    Definition Classes
    PipelineActivity
  72. def withMaxActiveInstances(activeInstances: HInt): Self

    Definition Classes
    PipelineActivity
  73. def withMaximumRetries(retries: HInt): Self

    Definition Classes
    PipelineActivity
  74. def withOutput(out: DataNode): PigActivity[A]

  75. def withPostActivityTaskConfig(config: ShellScriptConfig): Self

    Definition Classes
    EmrTaskActivity
  76. def withPreActivityTaskConfig(config: ShellScriptConfig): Self

    Definition Classes
    EmrTaskActivity
  77. def withRetryDelay(duration: HDuration): Self

    Definition Classes
    PipelineActivity
  78. def withScriptVariable(scriptVariable: HString*): PigActivity[A]

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from EmrTaskActivity[A]

Inherited from EmrActivity[A]

Inherited from PipelineActivity[A]

Inherited from NamedPipelineObject

Inherited from PipelineObject

Inherited from Ordered[PipelineObject]

Inherited from Comparable[PipelineObject]

Inherited from AnyRef

Inherited from Any

Ungrouped