com.intenthq.pucket

Pucket

trait Pucket[T] extends AnyRef

Trait for describing a bucket of data on a filesystem

T

The data type the pucket will contain

Linear Supertypes
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. Pucket
  2. AnyRef
  3. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def descriptor: PucketDescriptor[T]

    Descriptor object with information about the pucket.

    Descriptor object with information about the pucket. See com.intenthq.pucket.PucketDescriptor

  2. abstract def fs: FileSystem

    Hadoop filesystem instance

  3. abstract def newInstance(newPath: Path): Pucket[T]

    Used when obtaining a new pucket as a subdir of an existing one

    Used when obtaining a new pucket as a subdir of an existing one

    Attributes
    protected
  4. abstract def path: Path

    Fully qualified hadoop path to the pucket

  5. abstract def reader(filter: Option[Filter]): \/[Throwable, Reader[T]]

    Reader for the pucket.

    Reader for the pucket. See com.intenthq.pucket.reader.Reader

    filter
    returns

    a new Reader for reading data from the pucket

  6. abstract def writer: \/[Throwable, Writer[T, Throwable]]

    Writer implementation for the pucket.

    Writer implementation for the pucket. See com.intenthq.pucket.writer.Writer

Concrete Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. def absorb(pucket: Pucket[T]): \/[Throwable, Unit]

    Absorb another pucket of the same type into this one

    Absorb another pucket of the same type into this one

    Will only absorb if the pucket it is absorbing is on a different path and the descriptor is the same

    pucket

    the pucket to be absorbed

  7. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  8. def blockSize: Int

    Default Parquet block size

  9. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  10. val conf: Configuration

    Hadoop configuration

  11. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  12. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  13. def filename: Path

    Generate a fully qualified filename under the pucket

    Generate a fully qualified filename under the pucket

    returns

    a hadoop path to a new randomly generated filename

  14. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  15. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  16. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  17. val id: Int

    An identifier for the pucket for use in writer cache

    An identifier for the pucket for use in writer cache

    returns

    a string identifier

  18. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  19. def listFiles: \/[Throwable, Seq[Path]]

    List all the parquet files in the pucket

    List all the parquet files in the pucket

    returns

    a list of fully qualified paths to parquet files

  20. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  21. final def notify(): Unit

    Definition Classes
    AnyRef
  22. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  23. def partition(data: T): Path

    Use the partitioner specified in the descriptor to get a new pucket on a calculated path

    Use the partitioner specified in the descriptor to get a new pucket on a calculated path

    data

    the object to inspect for partitioning

    returns

    a new path which is a subpath under the pucket, otherwise the current path

  24. def reader: \/[Throwable, Reader[T]]

    Reader for the pucket.

    Reader for the pucket. See com.intenthq.pucket.reader.Reader

  25. def subPucket(subPath: Path): \/[Throwable, Pucket[T]]

    Create a directory as a subpath of the pucket and create a new pucket instance at the new path

    Create a directory as a subpath of the pucket and create a new pucket instance at the new path

    subPath
    returns

    a new pucket at the sub path specified or an error if the path creation fails

  26. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  27. def toString(): String

    Definition Classes
    AnyRef → Any
  28. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  29. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  30. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from AnyRef

Inherited from Any

Ungrouped