org.apache.spark.sql.execution.streaming.HDFSMetadataLog
FileSystemManager
Related Doc:
package HDFSMetadataLog
org.apache.spark.sql.execution.streaming.HDFSMetadataLog
Create path, or throw exception if it already exists
Create path, or throw exception if it already exists
Recursively delete a path if it exists.
Recursively delete a path if it exists. Should not throw exception if file doesn't exist.
Whether path exists
Whether path exists
Whether the file systme is a local FS.
Whether the file systme is a local FS.
List the files in a path that matches a filter.
List the files in a path that matches a filter.
Make directory at the give path and all its parent directories as needed.
Make directory at the give path and all its parent directories as needed.
Open a file for reading, or throw exception if it does not exist.
Open a file for reading, or throw exception if it does not exist.
Rename a path.
Rename a path. Note that this implementation is not atomic.
FileAlreadyExistsException if destination path already exists.
FileNotFoundException if source path does not exist.
IOException if renaming fails for some unknown reason.
Implementation of FileManager using older FileSystem API. Note that this implementation cannot provide atomic renaming of paths, hence can lead to consistency issues. This should be used only as a backup option, when FileContextManager cannot be used.