com.twitter.scalding.parquet.tuple

ParquetTupleSource

Related Docs: object ParquetTupleSource | package tuple

trait ParquetTupleSource extends FileSource with HasFilterPredicate

User should define their own source like: class MySource(path: String, dateRange: DateRange, requestedFields: Fields) extends DailySuffixParquetTuple(path, dateRange, requestedFields) with Mappable2[Int, Int] with TypedSink2[Int,Int]

Linear Supertypes
HasFilterPredicate, FileSource, LocalSourceOverride, SchemedSource, Source, Serializable, AnyRef, Any
Known Subclasses
DailySuffixParquetTuple, FixedPathParquetTuple, HourlySuffixParquetTuple
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. ParquetTupleSource
  2. HasFilterPredicate
  3. FileSource
  4. LocalSourceOverride
  5. SchemedSource
  6. Source
  7. Serializable
  8. AnyRef
  9. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def fields: Fields

  2. abstract def hdfsPaths: Iterable[String]

    Definition Classes
    FileSource

  3. abstract def localPaths: Iterable[String]

    Definition Classes
    LocalSourceOverride

Concrete Value Members

  1. final def !=(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  4. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  5. def checkFlowDefNotNull(implicit flowDef: FlowDef, mode: Mode): Unit

    Attributes
    protected
    Definition Classes
    Source

  6. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  7. def createHdfsReadTap(hdfsMode: Hdfs): Tap[JobConf, _, _]

    Attributes
    protected
    Definition Classes
    FileSource

  8. def createLocalTap(sinkMode: SinkMode): Tap[JobConf, _, _]

    Definition Classes
    LocalSourceOverride

  9. def createTap(readOrWrite: AccessMode)(implicit mode: Mode): Tap[_, _, _]

    Definition Classes
    FileSource → Source

  10. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  11. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  12. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  13. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  14. def goodHdfsPaths(hdfsMode: Hdfs): Iterable[String]

    Attributes
    protected
    Definition Classes
    FileSource

  15. def hashCode(): Int

    Definition Classes
    AnyRef → Any

  16. def hdfsReadPathsAreGood(conf: Configuration): Boolean

    Attributes
    protected
    Definition Classes
    FileSource

  17. def hdfsScheme: Scheme[JobConf, RecordReader[_, _], OutputCollector[_, _], _, _]

    Definition Classes
    ParquetTupleSource → SchemedSource

  18. def hdfsWritePath: String

    Definition Classes
    FileSource

  19. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  20. def localScheme: Scheme[Properties, InputStream, OutputStream, _, _]

    Definition Classes
    SchemedSource

  21. def localWritePath: String

    Definition Classes
    LocalSourceOverride

  22. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  23. final def notify(): Unit

    Definition Classes
    AnyRef

  24. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  25. def pathIsGood(p: String, conf: Configuration): Boolean

    Attributes
    protected
    Definition Classes
    FileSource

  26. def read(implicit flowDef: FlowDef, mode: Mode): Pipe

    Definition Classes
    Source

  27. val sinkMode: SinkMode

    Definition Classes
    SchemedSource

  28. def sourceId: String

    Definition Classes
    Source

  29. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  30. def toString(): String

    Definition Classes
    AnyRef → Any

  31. def transformForRead(pipe: Pipe): Pipe

    Attributes
    protected
    Definition Classes
    Source

  32. def transformForWrite(pipe: Pipe): Pipe

    Attributes
    protected
    Definition Classes
    Source

  33. def transformInTest: Boolean

    Definition Classes
    Source

  34. def validateTaps(mode: Mode): Unit

    Definition Classes
    FileSource → Source

  35. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  36. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  37. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  38. def withFilter: Option[FilterPredicate]

    Definition Classes
    HasFilterPredicate

  39. def writeFrom(pipe: Pipe)(implicit flowDef: FlowDef, mode: Mode): Pipe

    Definition Classes
    Source

Deprecated Value Members

  1. def readAtSubmitter[T](implicit mode: Mode, conv: TupleConverter[T]): Stream[T]

    Definition Classes
    Source
    Annotations
    @deprecated
    Deprecated

    (Since version 0.9.0) replace with Mappable.toIterator

Inherited from HasFilterPredicate

Inherited from FileSource

Inherited from LocalSourceOverride

Inherited from SchemedSource

Inherited from Source

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped