org.apache.hadoop.fs

Class LocalFileSystem

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.fs.FileSystem

org.apache.hadoop.fs.FilterFileSystem

org.apache.hadoop.fs.ChecksumFileSystem

org.apache.hadoop.fs.LocalFileSystem

All Implemented Interfaces:

Closeable, AutoCloseable, Configurable

@InterfaceAudience.Public
@InterfaceStability.Stable
public class LocalFileSystem
extends ChecksumFileSystem

Implement the FileSystem API for the checksumed local filesystem.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.fs.FileSystem

FileSystem.Statistics

Field Summary

Fields inherited from class org.apache.hadoop.fs.FilterFileSystem

fs, swapScheme

Fields inherited from class org.apache.hadoop.fs.FileSystem

DEFAULT_FS, FS_DEFAULT_NAME_KEY, LOG, SHUTDOWN_HOOK_PRIORITY, statistics

Constructor Summary

Constructors

Constructor and Description
LocalFileSystem()
LocalFileSystem(FileSystem rawLocalFileSystem)

Method Summary

Methods

Modifier and Type	Method and Description
void	copyFromLocalFile(boolean delSrc, Path src, Path dst) The src file is on the local disk.
void	copyToLocalFile(boolean delSrc, Path src, Path dst) The src file is under FS, and the dst is on the local disk.
FileSystem	getRaw()
void	initialize(URI name, Configuration conf) Called after a new FileSystem instance is constructed.
File	pathToFile(Path path) Convert a path to a File.
boolean	reportChecksumFailure(Path p, FSDataInputStream in, long inPos, FSDataInputStream sums, long sumsPos) Moves files to a bad file directory on the same device, so that their storage will not be reused.

Methods inherited from class org.apache.hadoop.fs.ChecksumFileSystem

append, completeLocalOutput, copyToLocalFile, create, createNonRecursive, delete, getApproxChkSumLength, getBytesPerSum, getChecksumFile, getChecksumFileLength, getChecksumLength, getRawFileSystem, isChecksumFile, listLocatedStatus, listStatus, mkdirs, open, rename, setConf, setReplication, setVerifyChecksum, setWriteChecksum, startLocalOutput

Methods inherited from class org.apache.hadoop.fs.FilterFileSystem

checkPath, close, copyFromLocalFile, copyFromLocalFile, getCanonicalUri, getChildFileSystems, getConf, getContentSummary, getDefaultBlockSize, getDefaultBlockSize, getDefaultReplication, getDefaultReplication, getFileBlockLocations, getFileChecksum, getFileStatus, getHomeDirectory, getInitialWorkingDirectory, getServerDefaults, getServerDefaults, getStatus, getUri, getUsed, getWorkingDirectory, listCorruptFileBlocks, makeQualified, mkdirs, primitiveCreate, primitiveMkdir, resolvePath, setOwner, setPermission, setTimes, setWorkingDirectory

Methods inherited from class org.apache.hadoop.fs.FileSystem

append, append, cancelDeleteOnExit, clearStatistics, closeAll, closeAllForUGI, copyFromLocalFile, copyToLocalFile, copyToLocalFile, create, create, create, create, create, create, create, create, create, create, create, createNewFile, createNonRecursive, delete, deleteOnExit, exists, get, get, get, getAllStatistics, getBlockSize, getDefaultPort, getDefaultUri, getFileBlockLocations, getLength, getLocal, getName, getNamed, getReplication, getStatistics, getStatistics, getStatus, globStatus, globStatus, isDirectory, isFile, listFiles, listLocatedStatus, listStatus, listStatus, listStatus, mkdirs, moveFromLocalFile, moveFromLocalFile, moveToLocalFile, newInstance, newInstance, newInstance, newInstanceLocal, open, primitiveMkdir, printStatistics, processDeleteOnExit, rename, setDefaultUri, setDefaultUri

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LocalFileSystem

public LocalFileSystem()

LocalFileSystem

public LocalFileSystem(FileSystem rawLocalFileSystem)

Method Detail

initialize

public void initialize(URI name,
              Configuration conf)
                throws IOException

Description copied from class: FilterFileSystem

Called after a new FileSystem instance is constructed.

Overrides:

initialize in class FilterFileSystem

Parameters:

name - a uri whose authority section names the host, port, etc. for this FileSystem

conf - the configuration

Throws:

IOException

getRaw

public FileSystem getRaw()

pathToFile

public File pathToFile(Path path)

Convert a path to a File.

copyFromLocalFile

public void copyFromLocalFile(boolean delSrc,
                     Path src,
                     Path dst)
                       throws IOException

Description copied from class: FilterFileSystem

The src file is on the local disk. Add it to FS at the given dst name. delSrc indicates if the source should be removed

Overrides:

copyFromLocalFile in class ChecksumFileSystem

Parameters:

delSrc - whether to delete the src

src - path

dst - path

Throws:

IOException

copyToLocalFile

public void copyToLocalFile(boolean delSrc,
                   Path src,
                   Path dst)
                     throws IOException

Description copied from class: ChecksumFileSystem

The src file is under FS, and the dst is on the local disk. Copy it from FS control to the local dst name.

Overrides:

copyToLocalFile in class ChecksumFileSystem

Parameters:

delSrc - whether to delete the src

src - path

dst - path

Throws:

IOException

reportChecksumFailure

public boolean reportChecksumFailure(Path p,
                            FSDataInputStream in,
                            long inPos,
                            FSDataInputStream sums,
                            long sumsPos)

Moves files to a bad file directory on the same device, so that their storage will not be reused.

Overrides:

reportChecksumFailure in class ChecksumFileSystem

Parameters:

p - the file name containing the error

in - the stream open on the file

inPos - the position of the beginning of the bad data in the file

sums - the stream open on the checksum file

sumsPos - the position of the beginning of the bad data in the checksum file

Returns:

if retry is neccessary