Trait

com.twitter.scalding.parquet.scrooge

ParquetScrooge

Related Doc: package scrooge

Permalink

trait ParquetScrooge[T <: ThriftStruct] extends FileSource with ParquetThriftBaseFileSource[T]

Linear Supertypes
ParquetThriftBaseFileSource[T], typed.TypedSink[T], SingleMappable[T], Mappable[T], typed.TypedSource[T], ParquetThriftBase[T], HasColumnProjection, HasFilterPredicate, LocalTapSource, FileSource, HfsTapProvider, LocalSourceOverride, SchemedSource, Source, Serializable, AnyRef, Any
Known Subclasses
DailySuffixParquetScrooge, FixedPathParquetScrooge, HourlySuffixParquetScrooge
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. ParquetScrooge
  2. ParquetThriftBaseFileSource
  3. TypedSink
  4. SingleMappable
  5. Mappable
  6. TypedSource
  7. ParquetThriftBase
  8. HasColumnProjection
  9. HasFilterPredicate
  10. LocalTapSource
  11. FileSource
  12. HfsTapProvider
  13. LocalSourceOverride
  14. SchemedSource
  15. Source
  16. Serializable
  17. AnyRef
  18. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. implicit abstract def ct: ClassTag[T]

    Permalink
    Definition Classes
    ParquetThriftBase

  2. abstract def hdfsPaths: Iterable[String]

    Permalink
    Definition Classes
    FileSource

  3. abstract def localPaths: Iterable[String]

    Permalink
    Definition Classes
    LocalSourceOverride

Concrete Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. def andThen[U](fn: (T) ⇒ U): Mappable[U]

    Permalink
    Definition Classes
    Mappable → TypedSource

  5. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  6. def checkFlowDefNotNull()(implicit flowDef: FlowDef, mode: Mode): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Source

  7. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  8. final def columnProjectionString: Option[ColumnProjectionString]

    Permalink
    Attributes
    protected[com.twitter.scalding.parquet]
    Definition Classes
    HasColumnProjection

  9. def config: ParquetValueScheme.Config[T]

    Permalink
    Definition Classes
    ParquetThriftBase

  10. def contraMap[U](fn: (U) ⇒ T): typed.TypedSink[U]

    Permalink
    Definition Classes
    TypedSink

  11. def converter[U >: T]: TupleConverter[U]

    Permalink
    Definition Classes
    SingleMappable → TypedSource

  12. def createHdfsReadTap(hdfsMode: Hdfs): Tap[JobConf, _, _]

    Permalink
    Attributes
    protected
    Definition Classes
    FileSource

  13. def createHfsTap(scheme: Scheme[JobConf, RecordReader[_, _], OutputCollector[_, _], _, _], path: String, sinkMode: SinkMode): Hfs

    Permalink
    Definition Classes
    HfsTapProvider

  14. def createLocalTap(sinkMode: SinkMode): Tap[JobConf, _, _]

    Permalink
    Definition Classes
    LocalTapSource → LocalSourceOverride

  15. def createTap(readOrWrite: AccessMode)(implicit mode: Mode): Tap[_, _, _]

    Permalink
    Definition Classes
    FileSource → Source

  16. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  17. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  18. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  19. final def flatMapTo[U](out: Fields)(mf: (T) ⇒ TraversableOnce[U])(implicit flowDef: FlowDef, mode: Mode, setter: TupleSetter[U]): Pipe

    Permalink
    Definition Classes
    Mappable

  20. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  21. def goodHdfsPaths(hdfsMode: Hdfs): Iterable[String]

    Permalink
    Attributes
    protected
    Definition Classes
    FileSource

  22. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  23. def hdfsReadPathsAreGood(conf: Configuration): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    FileSource

  24. def hdfsScheme: Scheme[JobConf, RecordReader[_, _], OutputCollector[_, _], _, _]

    Permalink
    Definition Classes
    ParquetScrooge → SchemedSource

  25. def hdfsWritePath: String

    Permalink
    Definition Classes
    FileSource

  26. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  27. def localScheme: Scheme[Properties, InputStream, OutputStream, _, _]

    Permalink
    Definition Classes
    SchemedSource

  28. def localWritePath: String

    Permalink
    Definition Classes
    LocalSourceOverride

  29. final def mapTo[U](out: Fields)(mf: (T) ⇒ U)(implicit flowDef: FlowDef, mode: Mode, setter: TupleSetter[U]): Pipe

    Permalink
    Definition Classes
    Mappable

  30. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  31. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  32. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  33. def pathIsGood(globPattern: String, conf: Configuration): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    FileSource

  34. def read(implicit flowDef: FlowDef, mode: Mode): Pipe

    Permalink
    Definition Classes
    Source

  35. def setter[U <: T]: TupleSetter[U]

    Permalink
    Definition Classes
    ParquetThriftBaseFileSource → TypedSink

  36. def sinkFields: Fields

    Permalink
    Definition Classes
    TypedSink

  37. val sinkMode: SinkMode

    Permalink
    Definition Classes
    SchemedSource

  38. def sourceFields: Fields

    Permalink
    Definition Classes
    TypedSource

  39. def sourceId: String

    Permalink
    Definition Classes
    Source

  40. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  41. def toIterator(implicit config: Config, mode: Mode): Iterator[T]

    Permalink
    Definition Classes
    Mappable

  42. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any

  43. def transformForRead(pipe: Pipe): Pipe

    Permalink
    Attributes
    protected
    Definition Classes
    Source

  44. def transformForWrite(pipe: Pipe): Pipe

    Permalink
    Attributes
    protected
    Definition Classes
    Source

  45. def transformInTest: Boolean

    Permalink
    Definition Classes
    Source

  46. def validateTaps(mode: Mode): Unit

    Permalink
    Definition Classes
    FileSource → Source

  47. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  48. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  49. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  50. def withColumnProjections: Set[String]

    Permalink
    Definition Classes
    HasColumnProjection

  51. def withFilter: Option[FilterPredicate]

    Permalink
    Definition Classes
    HasFilterPredicate

  52. def writeFrom(pipe: Pipe)(implicit flowDef: FlowDef, mode: Mode): Pipe

    Permalink
    Definition Classes
    Source

Deprecated Value Members

  1. def readAtSubmitter[T](implicit mode: Mode, conv: TupleConverter[T]): Stream[T]

    Permalink
    Definition Classes
    Source
    Annotations
    @deprecated
    Deprecated

    (Since version 0.9.0) replace with Mappable.toIterator

  2. def withColumns: Set[String]

    Permalink
    Definition Classes
    HasColumnProjection
    Annotations
    @deprecated
    Deprecated

    (Since version 0.15.1) Use withColumnProjections, which uses a different glob syntax

Inherited from ParquetThriftBaseFileSource[T]

Inherited from typed.TypedSink[T]

Inherited from SingleMappable[T]

Inherited from Mappable[T]

Inherited from typed.TypedSource[T]

Inherited from ParquetThriftBase[T]

Inherited from HasColumnProjection

Inherited from HasFilterPredicate

Inherited from LocalTapSource

Inherited from FileSource

Inherited from HfsTapProvider

Inherited from LocalSourceOverride

Inherited from SchemedSource

Inherited from Source

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped