io

Type Members

1. trait ByteAccess extends AnyRef

   

   ByteAccess is a wrapper trait around sources of bytes which are accessible at arbitrary offsets.

   ByteAccess is a wrapper trait around sources of bytes which are accessible at arbitrary offsets.

   (We can use this to wrap byte buffers, S3, FileInputStreams, or even range-accessible web services.)

2. class ByteArrayByteAccess extends ByteAccess with Serializable

   

3. class ByteArrayLocator extends FileLocator

   

4. trait FileLocator extends Serializable

   

   FileLocator is a trait which is meant to combine aspects of - Java's File - Hadoop's Path - S3 locations (including bucket and key) - classpath-relative URLs (classpath://, used in testing)

   FileLocator is a trait which is meant to combine aspects of - Java's File - Hadoop's Path - S3 locations (including bucket and key) - classpath-relative URLs (classpath://, used in testing)

   It provides methods for relative addressing (parent and child locators, which are equivalent to the File(parent, child) constructor and the getParentFile method on the Java File class), as well as accessing the bytes named by a locator by retrieving a ByteAccess value.

   We're using implementations of FileLocator to provide a uniform access interface to Parquet files, whether they're in HDFS, a local filesystem, S3, or embedded in the classpath as part of tests.

5. class HTTPFileLocator extends FileLocator

   

6. class HTTPRangedByteAccess extends ByteAccess

   

   HTTPRangedByteAccess supports Ranged GET queries against HTTP resources.

7. class LocalFileByteAccess extends ByteAccess

   

   This is somewhat poorly named, it probably should be LocalFileByteAccess

8. class LocalFileLocator extends FileLocator