Class

com.twitter.scalding

InvalidSourceTap

Related Doc: package scalding

Permalink

class InvalidSourceTap extends SourceTap[JobConf, RecordReader[_, _]]

InvalidSourceTap used in createTap method when we want to defer the failures to validateTaps method.

This is used because for Job classes, createTap method on sources is called when the class is initialized. In most cases though, we want any exceptions to be thrown by validateTaps method, which is called subsequently during flow planning.

hdfsPaths represents user-supplied list that was detected as not containing any valid paths.

Linear Supertypes
SourceTap[JobConf, RecordReader[_, _]], Tap[JobConf, RecordReader[_, _], Void], Traceable, FlowElement, Serializable, ScopedElement, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. InvalidSourceTap
  2. SourceTap
  3. Tap
  4. Traceable
  5. FlowElement
  6. Serializable
  7. ScopedElement
  8. AnyRef
  9. Any
  1. Hide All
  2. Show all
Visibility
  1. Public
  2. All

Instance Constructors

  1. new InvalidSourceTap(hdfsPaths: Iterable[String])

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  6. def commitResource(arg0: JobConf): Boolean

    Permalink
    Definition Classes
    SourceTap → Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  7. def createResource(arg0: JobConf): Boolean

    Permalink
    Definition Classes
    SourceTap → Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  8. def createResource(arg0: FlowProcess[_ <: JobConf]): Boolean

    Permalink
    Definition Classes
    Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  9. def deleteResource(arg0: JobConf): Boolean

    Permalink
    Definition Classes
    SourceTap → Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  10. def deleteResource(arg0: FlowProcess[_ <: JobConf]): Boolean

    Permalink
    Definition Classes
    Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  11. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  12. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    Tap → AnyRef → Any

  13. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  14. def flowConfInit(arg0: Flow[JobConf]): Unit

    Permalink
    Definition Classes
    Tap

  15. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  16. def getConfigDef(): ConfigDef

    Permalink
    Definition Classes
    Tap → ScopedElement

  17. def getFullIdentifier(arg0: JobConf): String

    Permalink
    Definition Classes
    Tap

  18. def getFullIdentifier(arg0: FlowProcess[_ <: JobConf]): String

    Permalink
    Definition Classes
    Tap

  19. def getIdentifier(): String

    Permalink
    Definition Classes
    InvalidSourceTap → Tap

  20. def getModifiedTime(conf: JobConf): Long

    Permalink
    Definition Classes
    InvalidSourceTap → Tap

  21. def getModifiedTime(arg0: FlowProcess[_ <: JobConf]): Long

    Permalink
    Definition Classes
    Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  22. def getNodeConfigDef(): ConfigDef

    Permalink
    Definition Classes
    Tap → ScopedElement

  23. def getScheme(): NullScheme[JobConf, RecordReader[_, _], Void, Nothing, Nothing]

    Permalink
    Definition Classes
    InvalidSourceTap → Tap

  24. def getSinkFields(): Fields

    Permalink
    Definition Classes
    SourceTap → Tap

  25. def getSinkMode(): SinkMode

    Permalink
    Definition Classes
    Tap

  26. def getSourceFields(): Fields

    Permalink
    Definition Classes
    Tap

  27. def getStepConfigDef(): ConfigDef

    Permalink
    Definition Classes
    Tap → ScopedElement

  28. def getTrace(): String

    Permalink
    Definition Classes
    Tap → Traceable

  29. def hasConfigDef(): Boolean

    Permalink
    Definition Classes
    Tap → ScopedElement

  30. def hasNodeConfigDef(): Boolean

    Permalink
    Definition Classes
    Tap → ScopedElement

  31. def hasStepConfigDef(): Boolean

    Permalink
    Definition Classes
    Tap → ScopedElement

  32. def hashCode(): Int

    Permalink
    Definition Classes
    InvalidSourceTap → Tap → AnyRef → Any

  33. val hdfsPaths: Iterable[String]

    Permalink

  34. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  35. def isKeep(): Boolean

    Permalink
    Definition Classes
    Tap

  36. def isReplace(): Boolean

    Permalink
    Definition Classes
    Tap

  37. final def isSink(): Boolean

    Permalink
    Definition Classes
    SourceTap → Tap

  38. def isSource(): Boolean

    Permalink
    Definition Classes
    Tap

  39. def isTemporary(): Boolean

    Permalink
    Definition Classes
    Tap

  40. def isUpdate(): Boolean

    Permalink
    Definition Classes
    Tap

  41. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  42. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  43. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  44. def openForRead(flow: FlowProcess[_ <: JobConf], input: RecordReader[_, _]): TupleEntryIterator

    Permalink
    Definition Classes
    InvalidSourceTap → Tap

  45. def openForRead(arg0: FlowProcess[_ <: JobConf]): TupleEntryIterator

    Permalink
    Definition Classes
    Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  46. def openForWrite(arg0: FlowProcess[_ <: JobConf], arg1: Void): TupleEntryCollector

    Permalink
    Definition Classes
    SourceTap → Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  47. def openForWrite(arg0: FlowProcess[_ <: JobConf]): TupleEntryCollector

    Permalink
    Definition Classes
    Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  48. def outgoingScopeFor(arg0: Set[Scope]): Scope

    Permalink
    Definition Classes
    Tap → ScopedElement

  49. def prepareResourceForRead(arg0: JobConf): Boolean

    Permalink
    Definition Classes
    Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  50. def prepareResourceForWrite(arg0: JobConf): Boolean

    Permalink
    Definition Classes
    SourceTap → Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  51. def presentSinkFields(arg0: FlowProcess[_ <: JobConf], arg1: Fields): Unit

    Permalink
    Definition Classes
    Tap

  52. def presentSourceFields(arg0: FlowProcess[_ <: JobConf], arg1: Fields): Unit

    Permalink
    Definition Classes
    Tap

  53. def resolveIncomingOperationArgumentFields(arg0: Scope): Fields

    Permalink
    Definition Classes
    Tap → ScopedElement

  54. def resolveIncomingOperationPassThroughFields(arg0: Scope): Fields

    Permalink
    Definition Classes
    Tap → ScopedElement

  55. def resourceExists(conf: JobConf): Boolean

    Permalink
    Definition Classes
    InvalidSourceTap → Tap

  56. def resourceExists(arg0: FlowProcess[_ <: JobConf]): Boolean

    Permalink
    Definition Classes
    Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  57. def retrieveSinkFields(arg0: FlowProcess[_ <: JobConf]): Fields

    Permalink
    Definition Classes
    Tap

  58. def retrieveSourceFields(arg0: FlowProcess[_ <: JobConf]): Fields

    Permalink
    Definition Classes
    Tap

  59. def rollbackResource(arg0: JobConf): Boolean

    Permalink
    Definition Classes
    SourceTap → Tap
    Annotations
    @throws( classOf[java.io.IOException] )

  60. def setScheme(arg0: Scheme[JobConf, RecordReader[_, _], Void, _, _]): Unit

    Permalink
    Attributes
    protected[cascading.tap]
    Definition Classes
    Tap

  61. def sinkConfInit(arg0: FlowProcess[_ <: JobConf], arg1: JobConf): Unit

    Permalink
    Definition Classes
    SourceTap → Tap

  62. def sourceConfInit(flow: FlowProcess[_ <: JobConf], conf: JobConf): Unit

    Permalink
    Definition Classes
    InvalidSourceTap → Tap

  63. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  64. def toString(): String

    Permalink
    Definition Classes
    Tap → AnyRef → Any

  65. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  66. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  67. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from SourceTap[JobConf, RecordReader[_, _]]

Inherited from Tap[JobConf, RecordReader[_, _], Void]

Inherited from Traceable

Inherited from FlowElement

Inherited from Serializable

Inherited from ScopedElement

Inherited from AnyRef

Inherited from Any

Ungrouped