FileLocator is a trait which is meant to combine aspects of
- Java's File
- Hadoop's Path
- S3 locations (including bucket and key)
- classpath-relative URLs (classpath://, used in testing)
It provides methods for relative addressing (parent and child locators,
which are equivalent to the File(parent, child) constructor and the getParentFile method
on the Java File class), as well as accessing the bytes named by a locator
by retrieving a ByteAccess value.
We're using implementations of FileLocator to provide a uniform access interface
to Parquet files, whether they're in HDFS, a local filesystem, S3, or embedded in the
classpath as part of tests.
FileLocator is a trait which is meant to combine aspects of - Java's File - Hadoop's Path - S3 locations (including bucket and key) - classpath-relative URLs (classpath://, used in testing)
It provides methods for relative addressing (parent and child locators, which are equivalent to the File(parent, child) constructor and the getParentFile method on the Java File class), as well as accessing the bytes named by a locator by retrieving a ByteAccess value.
We're using implementations of FileLocator to provide a uniform access interface to Parquet files, whether they're in HDFS, a local filesystem, S3, or embedded in the classpath as part of tests.