GoogleHadoopFileSystemBase (gcs-connector-hadoop2 hadoop2-1.9.9 API)

java.lang.Object
- org.apache.hadoop.conf.Configured
- - org.apache.hadoop.fs.FileSystem
  - - com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

All Implemented Interfaces:

FileSystemDescriptor, java.io.Closeable, java.lang.AutoCloseable, org.apache.hadoop.conf.Configurable

Direct Known Subclasses:

GoogleHadoopFileSystem
```
public abstract class GoogleHadoopFileSystemBase
extends org.apache.hadoop.fs.FileSystem
implements FileSystemDescriptor
```
This class provides a Hadoop compatible File System on top of Google Cloud Storage (GCS).
It is implemented as a thin abstraction layer on top of GCS. The layer hides any specific characteristics of the underlying store and exposes FileSystem interface understood by the Hadoop engine.
Users interact with the files in the storage using fully qualified URIs. The file system exposed by this class is identified using the 'gs' scheme. For example, gs://dir1/dir2/file1.txt.
This implementation translates paths between hadoop Path and GCS URI with the convention that the Hadoop root directly corresponds to the GCS "root", e.g. gs:/. This is convenient for many reasons, such as data portability and close equivalence to gsutil paths, but imposes certain inherited constraints, such as files not being allowed in root (only 'directories' can be placed in root), and directory names inside root have a more limited set of allowed characters.
One of the main goals of this implementation is to maintain compatibility with behavior of HDFS implementation when accessed through FileSystem interface. HDFS implementation is not very consistent about the cases when it throws versus the cases when methods return false. We run GHFS tests and HDFS tests against the same test data and use that as a guide to decide whether to throw or to return false.

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static class`	`GoogleHadoopFileSystemBase.Counter` Defines names of counters we track for each operation.
`static class`	`GoogleHadoopFileSystemBase.GcsFileChecksumType` Available GCS checksum types for use with `GoogleHadoopFileSystemConfiguration.GCS_FILE_CHECKSUM_TYPE`.
`protected static class`	`GoogleHadoopFileSystemBase.ListStatusFileNotFoundBehavior` Behavior of listStatus when a path is not found.
`static class`	`GoogleHadoopFileSystemBase.OutputStreamType` Available types for use with `GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_TYPE`.
`static class`	`GoogleHadoopFileSystemBase.ParentTimestampUpdateIncludePredicate` A predicate that processes individual directory paths and evaluates the conditions set in fs.gs.parent.timestamp.update.enable, fs.gs.parent.timestamp.update.substrings.include and fs.gs.parent.timestamp.update.substrings.exclude to determine if a path should be ignored when running directory timestamp updates.

Nested classes/interfaces inherited from class org.apache.hadoop.fs.FileSystem
org.apache.hadoop.fs.FileSystem.DirectoryEntries, org.apache.hadoop.fs.FileSystem.Statistics

Field Summary

Fields
Modifier and Type	Field and Description
`static java.lang.String`	`AUTHENTICATION_PREFIX` Prefix to use for common authentication keys.
`static int`	`BLOCK_SIZE_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.BLOCK_SIZE`
`static java.lang.String`	`BLOCK_SIZE_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.BLOCK_SIZE`
`static int`	`BUFFERSIZE_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_BUFFER_SIZE`
`static java.lang.String`	`BUFFERSIZE_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_BUFFER_SIZE`
`protected com.google.common.collect.ImmutableMap<GoogleHadoopFileSystemBase.Counter,java.util.concurrent.atomic.AtomicLong>`	`counters` Map of counter values
`static org.apache.hadoop.fs.PathFilter`	`DEFAULT_FILTER` Default PathFilter that accepts all paths.
`protected long`	`defaultBlockSize` Default block size.
`static java.lang.String`	`ENABLE_GCE_SERVICE_ACCOUNT_AUTH_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.AUTH_SERVICE_ACCOUNT_ENABLE`
`static boolean`	`GCE_BUCKET_DELETE_ENABLE_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCE_BUCKET_DELETE_ENABLE`
`static java.lang.String`	`GCE_BUCKET_DELETE_ENABLE_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCE_BUCKET_DELETE_ENABLE`
`static java.lang.String`	`GCS_APPLICATION_NAME_SUFFIX_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_APPLICATION_NAME_SUFFIX`
`static java.lang.String`	`GCS_APPLICATION_NAME_SUFFIX_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_APPLICATION_NAME_SUFFIX`
`static java.lang.String`	`GCS_BATCH_THREADS` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_BATCH_THREADS`
`static int`	`GCS_BATCH_THREADS_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_BATCH_THREADS`
`static java.lang.String`	`GCS_CLIENT_ID_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.AUTH_CLIENT_ID`
`static java.lang.String`	`GCS_CLIENT_SECRET_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.AUTH_CLIENT_SECRET`
`static java.lang.String`	`GCS_COPY_BATCH_THREADS` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_COPY_BATCH_THREADS`
`static int`	`GCS_COPY_BATCH_THREADS_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_COPY_BATCH_THREADS`
`static java.lang.String`	`GCS_COPY_MAX_REQUESTS_PER_BATCH` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_COPY_MAX_REQUESTS_PER_BATCH`
`static long`	`GCS_COPY_MAX_REQUESTS_PER_BATCH_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_COPY_MAX_REQUESTS_PER_BATCH`
`static boolean`	`GCS_CREATE_SYSTEM_BUCKET_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_CREATE_SYSTEM_BUCKET`
`static java.lang.String`	`GCS_CREATE_SYSTEM_BUCKET_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_CREATE_SYSTEM_BUCKET`
`static boolean`	`GCS_ENABLE_COPY_WITH_REWRITE_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_COPY_WITH_REWRITE_ENABLE`
`static java.lang.String`	`GCS_ENABLE_COPY_WITH_REWRITE_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_COPY_WITH_REWRITE_ENABLE`
`static boolean`	`GCS_ENABLE_FLAT_GLOB_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_FLAT_GLOB_ENABLE`
`static java.lang.String`	`GCS_ENABLE_FLAT_GLOB_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_FLAT_GLOB_ENABLE`
`static boolean`	`GCS_ENABLE_INFER_IMPLICIT_DIRECTORIES_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_INFER_IMPLICIT_DIRECTORIES_ENABLE`
`static java.lang.String`	`GCS_ENABLE_INFER_IMPLICIT_DIRECTORIES_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_INFER_IMPLICIT_DIRECTORIES_ENABLE`
`static boolean`	`GCS_ENABLE_MARKER_FILE_CREATION_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_MARKER_FILE_CREATION_ENABLE`
`static java.lang.String`	`GCS_ENABLE_MARKER_FILE_CREATION_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_MARKER_FILE_CREATION_ENABLE`
`static boolean`	`GCS_ENABLE_PERFORMANCE_CACHE_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_ENABLE`
`static java.lang.String`	`GCS_ENABLE_PERFORMANCE_CACHE_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_ENABLE`
`static boolean`	`GCS_ENABLE_REPAIR_IMPLICIT_DIRECTORIES_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_REPAIR_IMPLICIT_DIRECTORIES_ENABLE`
`static java.lang.String`	`GCS_ENABLE_REPAIR_IMPLICIT_DIRECTORIES_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_REPAIR_IMPLICIT_DIRECTORIES_ENABLE`
`static java.lang.String`	`GCS_FILE_SIZE_LIMIT_250GB` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_FILE_SIZE_LIMIT_250GB`
`static boolean`	`GCS_FILE_SIZE_LIMIT_250GB_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_FILE_SIZE_LIMIT_250GB`
`static GoogleCloudStorageReadOptions.GenerationReadConsistency`	`GCS_GENERATION_READ_CONSISTENCY_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_GENERATION_READ_CONSISTENCY`
`static java.lang.String`	`GCS_GENERATION_READ_CONSISTENCY_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_GENERATION_READ_CONSISTENCY`
`static int`	`GCS_HTTP_CONNECT_TIMEOUT_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_HTTP_CONNECT_TIMEOUT`
`static java.lang.String`	`GCS_HTTP_CONNECT_TIMEOUT_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_HTTP_CONNECT_TIMEOUT`
`static int`	`GCS_HTTP_MAX_RETRY_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_HTTP_MAX_RETRY`
`static java.lang.String`	`GCS_HTTP_MAX_RETRY_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_HTTP_MAX_RETRY`
`static int`	`GCS_HTTP_READ_TIMEOUT_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_HTTP_READ_TIMEOUT`
`static java.lang.String`	`GCS_HTTP_READ_TIMEOUT_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_HTTP_READ_TIMEOUT`
`static java.lang.String`	`GCS_HTTP_TRANSPORT_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_HTTP_TRANSPORT`
`static java.lang.String`	`GCS_HTTP_TRANSPORT_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_HTTP_TRANSPORT`
`static GoogleCloudStorageReadOptions.Fadvise`	`GCS_INPUTSTREAM_FADVISE_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_FADVISE`
`static java.lang.String`	`GCS_INPUTSTREAM_FADVISE_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_FADVISE`
`static boolean`	`GCS_INPUTSTREAM_FAST_FAIL_ON_NOT_FOUND_ENABLE_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_FAST_FAIL_ON_NOT_FOUND_ENABLE`
`static java.lang.String`	`GCS_INPUTSTREAM_FAST_FAIL_ON_NOT_FOUND_ENABLE_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_FAST_FAIL_ON_NOT_FOUND_ENABLE`
`static long`	`GCS_INPUTSTREAM_INPLACE_SEEK_LIMIT_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_INPLACE_SEEK_LIMIT`
`static java.lang.String`	`GCS_INPUTSTREAM_INPLACE_SEEK_LIMIT_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_INPLACE_SEEK_LIMIT`
`static int`	`GCS_INPUTSTREAM_MIN_RANGE_REQUEST_SIZE_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_MIN_RANGE_REQUEST_SIZE`
`static java.lang.String`	`GCS_INPUTSTREAM_MIN_RANGE_REQUEST_SIZE_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_MIN_RANGE_REQUEST_SIZE`
`static java.lang.String`	`GCS_MARKER_FILE_PATTERN_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_MARKER_FILE_PATTERN`
`static java.lang.String`	`GCS_MAX_LIST_ITEMS_PER_CALL` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_MAX_LIST_ITEMS_PER_CALL`
`static long`	`GCS_MAX_LIST_ITEMS_PER_CALL_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_MAX_LIST_ITEMS_PER_CALL`
`static java.lang.String`	`GCS_MAX_REQUESTS_PER_BATCH` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_MAX_REQUESTS_PER_BATCH`
`static long`	`GCS_MAX_REQUESTS_PER_BATCH_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_MAX_REQUESTS_PER_BATCH`
`static int`	`GCS_MAX_WAIT_MILLIS_EMPTY_OBJECT_CREATE_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_MAX_WAIT_MILLIS_EMPTY_OBJECT_CREATE`
`static java.lang.String`	`GCS_MAX_WAIT_MILLIS_EMPTY_OBJECT_CREATE_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_MAX_WAIT_MILLIS_EMPTY_OBJECT_CREATE`
`static java.lang.String`	`GCS_OUTPUTSTREAM_TYPE_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_TYPE`
`static java.lang.String`	`GCS_OUTPUTSTREAM_TYPE_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_TYPE`
`static boolean`	`GCS_PARENT_TIMESTAMP_UPDATE_ENABLE_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_PARENT_TIMESTAMP_UPDATE_ENABLE`
`static java.lang.String`	`GCS_PARENT_TIMESTAMP_UPDATE_ENABLE_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_PARENT_TIMESTAMP_UPDATE_ENABLE`
`static java.lang.String`	`GCS_PARENT_TIMESTAMP_UPDATE_EXCLUDES_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_PARENT_TIMESTAMP_UPDATE_EXCLUDES`
`static java.lang.String`	`GCS_PARENT_TIMESTAMP_UPDATE_EXCLUDES_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_PARENT_TIMESTAMP_UPDATE_EXCLUDES`
`static java.lang.String`	`GCS_PARENT_TIMESTAMP_UPDATE_INCLUDES_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_PARENT_TIMESTAMP_UPDATE_INCLUDES`
`static java.lang.String`	`GCS_PARENT_TIMESTAMP_UPDATE_INCLUDES_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_PARENT_TIMESTAMP_UPDATE_INCLUDES`
`static long`	`GCS_PERFORMANCE_CACHE_DIR_METADATA_PREFETCH_LIMIT_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_DIR_METADATA_PREFETCH_LIMIT`
`static java.lang.String`	`GCS_PERFORMANCE_CACHE_DIR_METADATA_PREFETCH_LIMIT_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_DIR_METADATA_PREFETCH_LIMIT`
`static boolean`	`GCS_PERFORMANCE_CACHE_LIST_CACHING_ENABLE_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_LIST_CACHING_ENABLE`
`static java.lang.String`	`GCS_PERFORMANCE_CACHE_LIST_CACHING_ENABLE_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_LIST_CACHING_ENABLE`
`static long`	`GCS_PERFORMANCE_CACHE_MAX_ENTRY_AGE_MILLIS_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_MAX_ENTRY_AGE_MILLIS`
`static java.lang.String`	`GCS_PERFORMANCE_CACHE_MAX_ENTRY_AGE_MILLIS_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_MAX_ENTRY_AGE_MILLIS`
`static java.lang.String`	`GCS_PROJECT_ID_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_PROJECT_ID`
`static java.lang.String`	`GCS_PROXY_ADDRESS_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_PROXY_ADDRESS`
`static java.lang.String`	`GCS_PROXY_ADDRESS_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_PROXY_ADDRESS`
`static java.lang.String`	`GCS_REQUESTER_PAYS_BUCKETS_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_REQUESTER_PAYS_BUCKETS`
`static java.lang.String`	`GCS_REQUESTER_PAYS_MODE_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_REQUESTER_PAYS_MODE`
`static java.lang.String`	`GCS_REQUESTER_PAYS_PROJECT_ID_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_REQUESTER_PAYS_PROJECT_ID`
`static java.lang.String`	`GCS_SYSTEM_BUCKET_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_SYSTEM_BUCKET`
`static java.lang.String`	`GCS_WORKING_DIRECTORY_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_WORKING_DIRECTORY`
`protected GoogleCloudStorageFileSystem`	`gcsfs` Underlying GCS file system object.
`static java.lang.String`	`GHFS_ID` Identifies this version of the GoogleHadoopFileSystemBase library.
`protected java.net.URI`	`initUri` The URI the File System is passed in initialize.
`protected GoogleHadoopFileSystemBase.ListStatusFileNotFoundBehavior`	`listStatusFileNotFoundBehavior`
`static java.lang.String`	`MR_JOB_HISTORY_DONE_DIR_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.MR_JOB_HISTORY_DONE_DIR_KEY`
`static java.lang.String`	`MR_JOB_HISTORY_INTERMEDIATE_DONE_DIR_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.MR_JOB_HISTORY_INTERMEDIATE_DONE_DIR_KEY`
`static java.lang.String`	`PATH_CODEC_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.PATH_CODEC`
`static java.lang.String`	`PATH_CODEC_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.PATH_CODEC`
`static java.lang.String`	`PATH_CODEC_USE_LEGACY_ENCODING` Use LEGACY_PATH_CODEC.
`static java.lang.String`	`PATH_CODEC_USE_URI_ENCODING` Use new URI_ENCODED_PATH_CODEC.
`static java.lang.String`	`PERMISSIONS_TO_REPORT_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.PERMISSIONS_TO_REPORT`
`static java.lang.String`	`PERMISSIONS_TO_REPORT_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.PERMISSIONS_TO_REPORT`
`static java.lang.String`	`PROPERTIES_FILE` A resource file containing GCS related build properties.
`static short`	`REPLICATION_FACTOR_DEFAULT` Default value of replication factor.
`static java.lang.String`	`SERVICE_ACCOUNT_AUTH_EMAIL_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.AUTH_SERVICE_ACCOUNT_EMAIL`
`static java.lang.String`	`SERVICE_ACCOUNT_AUTH_KEYFILE_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.AUTH_SERVICE_ACCOUNT_KEY_FILE`
`protected java.lang.String`	`systemBucket` Deprecated.
`static java.lang.String`	`UNKNOWN_VERSION` The version returned when one cannot be found in properties.
`static java.lang.String`	`VERSION` Current version.
`static java.lang.String`	`VERSION_PROPERTY` The key in the PROPERTIES_FILE that contains the version built.
`static int`	`WRITE_BUFFERSIZE_DEFAULT` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_BUFFER_SIZE`
`static java.lang.String`	`WRITE_BUFFERSIZE_KEY` Deprecated. use `GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_BUFFER_SIZE`

Fields inherited from class org.apache.hadoop.fs.FileSystem
DEFAULT_FS, FS_DEFAULT_NAME_KEY, LOG, SHUTDOWN_HOOK_PRIORITY, statistics, TRASH_PREFIX, USER_HOME_PREFIX

Constructor Summary

Constructors
Constructor and Description
`GoogleHadoopFileSystemBase()` Constructs an instance of GoogleHadoopFileSystemBase; the internal GoogleCloudStorageFileSystem will be set up with config settings when initialize() is called.
`GoogleHadoopFileSystemBase(GoogleCloudStorageFileSystem gcsfs)` Constructs an instance of GoogleHadoopFileSystemBase using the provided GoogleCloudStorageFileSystem; initialize() will not re-initialize it.

Method Summary

All Methods Instance Methods Abstract Methods Concrete Methods Deprecated Methods
Modifier and Type	Method and Description
`org.apache.hadoop.fs.FSDataOutputStream`	`append(org.apache.hadoop.fs.Path hadoopPath, int bufferSize, org.apache.hadoop.util.Progressable progress)` Appends to an existing file (optional operation).
`protected void`	`checkPath(org.apache.hadoop.fs.Path path)`
`void`	`close()`
`void`	`completeLocalOutput(org.apache.hadoop.fs.Path fsOutputFile, org.apache.hadoop.fs.Path tmpLocalFile)`
`void`	`configureBuckets(java.lang.String systemBucketName, boolean createSystemBucket)` Validates and possibly creates the system bucket.
`void`	`copyFromLocalFile(boolean delSrc, boolean overwrite, org.apache.hadoop.fs.Path[] srcs, org.apache.hadoop.fs.Path dst)`
`void`	`copyFromLocalFile(boolean delSrc, boolean overwrite, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst)`
`void`	`copyToLocalFile(boolean delSrc, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst)`
`org.apache.hadoop.fs.FSDataOutputStream`	`create(org.apache.hadoop.fs.Path hadoopPath, org.apache.hadoop.fs.permission.FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress)` Opens the given file for writing.
`protected com.google.common.collect.ImmutableMap<GoogleHadoopFileSystemBase.Counter,java.util.concurrent.atomic.AtomicLong>`	`createCounterMap()`
`org.apache.hadoop.fs.FSDataOutputStream`	`createNonRecursive(org.apache.hadoop.fs.Path hadoopPath, org.apache.hadoop.fs.permission.FsPermission permission, java.util.EnumSet<org.apache.hadoop.fs.CreateFlag> flags, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress)`
`boolean`	`delete(org.apache.hadoop.fs.Path f)` Deprecated. Use `delete(Path, boolean)` instead
`boolean`	`delete(org.apache.hadoop.fs.Path hadoopPath, boolean recursive)` Deletes the given file or directory.
`boolean`	`deleteOnExit(org.apache.hadoop.fs.Path f)`
`java.lang.String`	`getCanonicalServiceName()`
`org.apache.hadoop.fs.ContentSummary`	`getContentSummary(org.apache.hadoop.fs.Path f)`
`long`	`getDefaultBlockSize()`
`protected int`	`getDefaultPort()` The default port is listed as -1 as an indication that ports are not used.
`short`	`getDefaultReplication()` Gets the default replication factor.
`abstract org.apache.hadoop.fs.Path`	`getDefaultWorkingDirectory()` Gets the default value of working directory.
`org.apache.hadoop.security.token.Token<?>`	`getDelegationToken(java.lang.String renewer)`
`org.apache.hadoop.fs.FileChecksum`	`getFileChecksum(org.apache.hadoop.fs.Path hadoopPath)`
`org.apache.hadoop.fs.FileStatus`	`getFileStatus(org.apache.hadoop.fs.Path hadoopPath)` Gets status of the given path item.
`abstract org.apache.hadoop.fs.Path`	`getFileSystemRoot()` Returns the Hadoop path representing the root of the FileSystem associated with this FileSystemDescriptor.
`GoogleCloudStorageFileSystem`	`getGcsFs()` Gets GCS FS instance.
`abstract java.net.URI`	`getGcsPath(org.apache.hadoop.fs.Path hadoopPath)` Gets GCS path corresponding to the given Hadoop path, which can be relative or absolute, and can have either gs:// or gs:/ forms.
`abstract org.apache.hadoop.fs.Path`	`getHadoopPath(java.net.URI gcsPath)` Gets Hadoop path corresponding to the given GCS path.
`java.lang.String`	`getHadoopScheme()` Deprecated.
`org.apache.hadoop.fs.Path`	`getHomeDirectory()` Returns home directory of the current user.
`protected abstract java.lang.String`	`getHomeDirectorySubpath()` Returns an unqualified path without any leading slash, relative to the filesystem root, which serves as the home directory of the current user; see `getHomeDirectory` for a description of what the home directory means.
`abstract java.lang.String`	`getScheme()` Returns the URI scheme for the Hadoop FileSystem associated with this FileSystemDescriptor.
`java.net.URI`	`getUri()` Returns a URI of the root of this FileSystem.
`long`	`getUsed()`
`org.apache.hadoop.fs.Path`	`getWorkingDirectory()` Gets the current working directory.
`org.apache.hadoop.fs.FileStatus[]`	`globStatus(org.apache.hadoop.fs.Path pathPattern)` Returns an array of FileStatus objects whose path names match pathPattern.
`org.apache.hadoop.fs.FileStatus[]`	`globStatus(org.apache.hadoop.fs.Path pathPattern, org.apache.hadoop.fs.PathFilter filter)` Returns an array of FileStatus objects whose path names match pathPattern and is accepted by the user-supplied path filter.
`void`	`initialize(java.net.URI path, org.apache.hadoop.conf.Configuration config)` See `initialize(URI, Configuration, boolean)` for details; calls with third arg defaulting to 'true' for initializing the superclass.
`void`	`initialize(java.net.URI path, org.apache.hadoop.conf.Configuration config, boolean initSuperclass)` Initializes this file system instance.
`org.apache.hadoop.fs.FileStatus[]`	`listStatus(org.apache.hadoop.fs.Path hadoopPath)` Lists file status.
`org.apache.hadoop.fs.Path`	`makeQualified(org.apache.hadoop.fs.Path path)` Overridden to make root it's own parent.
`boolean`	`mkdirs(org.apache.hadoop.fs.Path hadoopPath, org.apache.hadoop.fs.permission.FsPermission permission)` Makes the given path and all non-existent parents directories.
`org.apache.hadoop.fs.FSDataInputStream`	`open(org.apache.hadoop.fs.Path hadoopPath, int bufferSize)` Opens the given file for reading.
`protected void`	`processDeleteOnExit()`
`boolean`	`rename(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst)` Renames src to dst.
`protected void`	`setListStatusFileNotFoundBehavior(GoogleHadoopFileSystemBase.ListStatusFileNotFoundBehavior behavior)`
`void`	`setOwner(org.apache.hadoop.fs.Path p, java.lang.String username, java.lang.String groupname)`
`void`	`setPermission(org.apache.hadoop.fs.Path p, org.apache.hadoop.fs.permission.FsPermission permission)`
`void`	`setTimes(org.apache.hadoop.fs.Path p, long mtime, long atime)`
`void`	`setVerifyChecksum(boolean verifyChecksum)`
`void`	`setWorkingDirectory(org.apache.hadoop.fs.Path hadoopPath)` Sets the current working directory to the given path.
`org.apache.hadoop.fs.Path`	`startLocalOutput(org.apache.hadoop.fs.Path fsOutputFile, org.apache.hadoop.fs.Path tmpLocalFile)`

Methods inherited from class org.apache.hadoop.fs.FileSystem
access, addDelegationTokens, append, append, appendFile, areSymlinksEnabled, cancelDeleteOnExit, canonicalizeUri, clearStatistics, closeAll, closeAllForUGI, concat, copyFromLocalFile, copyFromLocalFile, copyToLocalFile, copyToLocalFile, create, create, create, create, create, create, create, create, create, create, create, create, createFile, createNewFile, createNonRecursive, createNonRecursive, createSnapshot, createSnapshot, createSymlink, deleteSnapshot, enableSymlinks, exists, fixRelativePart, get, get, get, getAclStatus, getAllStatistics, getAllStoragePolicies, getBlockSize, getCanonicalUri, getChildFileSystems, getDefaultBlockSize, getDefaultReplication, getDefaultUri, getFileBlockLocations, getFileBlockLocations, getFileChecksum, getFileLinkStatus, getFileSystemClass, getFSofPath, getGlobalStorageStatistics, getInitialWorkingDirectory, getLength, getLinkTarget, getLocal, getName, getNamed, getQuotaUsage, getReplication, getServerDefaults, getServerDefaults, getStatistics, getStatistics, getStatus, getStatus, getStoragePolicy, getStorageStatistics, getTrashRoot, getTrashRoots, getUsed, getXAttr, getXAttrs, getXAttrs, isDirectory, isFile, listCorruptFileBlocks, listFiles, listLocatedStatus, listLocatedStatus, listStatus, listStatus, listStatus, listStatusBatch, listStatusIterator, listXAttrs, mkdirs, mkdirs, modifyAclEntries, moveFromLocalFile, moveFromLocalFile, moveToLocalFile, newInstance, newInstance, newInstance, newInstanceLocal, open, primitiveCreate, primitiveMkdir, primitiveMkdir, printStatistics, removeAcl, removeAclEntries, removeDefaultAcl, removeXAttr, rename, renameSnapshot, resolveLink, resolvePath, setAcl, setDefaultUri, setDefaultUri, setReplication, setStoragePolicy, setWriteChecksum, setXAttr, setXAttr, supportsSymlinks, truncate, unsetStoragePolicy

Methods inherited from class org.apache.hadoop.conf.Configured
getConf, setConf

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - PATH_CODEC_USE_URI_ENCODING
```
public static final java.lang.String PATH_CODEC_USE_URI_ENCODING
```
    Use new URI_ENCODED_PATH_CODEC.
    
    See Also:
    
    Constant Field Values
  - PATH_CODEC_USE_LEGACY_ENCODING
```
public static final java.lang.String PATH_CODEC_USE_LEGACY_ENCODING
```
    Use LEGACY_PATH_CODEC.
    
    See Also:
    
    Constant Field Values
  - REPLICATION_FACTOR_DEFAULT
```
public static final short REPLICATION_FACTOR_DEFAULT
```
    Default value of replication factor.
    
    See Also:
    
    Constant Field Values
  - DEFAULT_FILTER
```
public static final org.apache.hadoop.fs.PathFilter DEFAULT_FILTER
```
    Default PathFilter that accepts all paths.
  - AUTHENTICATION_PREFIX
```
public static final java.lang.String AUTHENTICATION_PREFIX
```
    Prefix to use for common authentication keys.
    
    See Also:
    
    Constant Field Values
  - PROPERTIES_FILE
```
public static final java.lang.String PROPERTIES_FILE
```
    A resource file containing GCS related build properties.
    
    See Also:
    
    Constant Field Values
  - VERSION_PROPERTY
```
public static final java.lang.String VERSION_PROPERTY
```
    The key in the PROPERTIES_FILE that contains the version built.
    
    See Also:
    
    Constant Field Values
  - UNKNOWN_VERSION
```
public static final java.lang.String UNKNOWN_VERSION
```
    The version returned when one cannot be found in properties.
    
    See Also:
    
    Constant Field Values
  - VERSION
```
public static final java.lang.String VERSION
```
    Current version.
  - GHFS_ID
```
public static final java.lang.String GHFS_ID
```
    Identifies this version of the GoogleHadoopFileSystemBase library.
  - initUri
```
protected java.net.URI initUri
```
    The URI the File System is passed in initialize.
  - systemBucket
```
@Deprecated
protected java.lang.String systemBucket
```
    Deprecated.
    
    The retrieved configuration value for GoogleHadoopFileSystemConfiguration.GCS_SYSTEM_BUCKET. Used as a fallback for a root bucket, when required.
  - gcsfs
```
protected GoogleCloudStorageFileSystem gcsfs
```
    Underlying GCS file system object.
  - defaultBlockSize
```
protected long defaultBlockSize
```
    Default block size. Note that this is the size that is reported to Hadoop FS clients. It does not modify the actual block size of an underlying GCS object, because GCS JSON API does not allow modifying or querying the value. Modifying this value allows one to control how many mappers are used to process a given file.
  - counters
```
protected final com.google.common.collect.ImmutableMap<GoogleHadoopFileSystemBase.Counter,java.util.concurrent.atomic.AtomicLong> counters
```
    Map of counter values
  - listStatusFileNotFoundBehavior
```
protected GoogleHadoopFileSystemBase.ListStatusFileNotFoundBehavior listStatusFileNotFoundBehavior
```
  - PERMISSIONS_TO_REPORT_KEY
```
@Deprecated
public static final java.lang.String PERMISSIONS_TO_REPORT_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.PERMISSIONS_TO_REPORT
  - PERMISSIONS_TO_REPORT_DEFAULT
```
@Deprecated
public static final java.lang.String PERMISSIONS_TO_REPORT_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.PERMISSIONS_TO_REPORT
  - BUFFERSIZE_KEY
```
@Deprecated
public static final java.lang.String BUFFERSIZE_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_BUFFER_SIZE
  - BUFFERSIZE_DEFAULT
```
@Deprecated
public static final int BUFFERSIZE_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_BUFFER_SIZE
  - WRITE_BUFFERSIZE_KEY
```
@Deprecated
public static final java.lang.String WRITE_BUFFERSIZE_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_BUFFER_SIZE
  - WRITE_BUFFERSIZE_DEFAULT
```
@Deprecated
public static final int WRITE_BUFFERSIZE_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_BUFFER_SIZE
  - BLOCK_SIZE_KEY
```
@Deprecated
public static final java.lang.String BLOCK_SIZE_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.BLOCK_SIZE
  - BLOCK_SIZE_DEFAULT
```
@Deprecated
public static final int BLOCK_SIZE_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.BLOCK_SIZE
  - ENABLE_GCE_SERVICE_ACCOUNT_AUTH_KEY
```
@Deprecated
public static final java.lang.String ENABLE_GCE_SERVICE_ACCOUNT_AUTH_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.AUTH_SERVICE_ACCOUNT_ENABLE
  - SERVICE_ACCOUNT_AUTH_EMAIL_KEY
```
@Deprecated
public static final java.lang.String SERVICE_ACCOUNT_AUTH_EMAIL_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.AUTH_SERVICE_ACCOUNT_EMAIL
  - SERVICE_ACCOUNT_AUTH_KEYFILE_KEY
```
@Deprecated
public static final java.lang.String SERVICE_ACCOUNT_AUTH_KEYFILE_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.AUTH_SERVICE_ACCOUNT_KEY_FILE
  - GCS_PROJECT_ID_KEY
```
@Deprecated
public static final java.lang.String GCS_PROJECT_ID_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_PROJECT_ID
  - GCS_REQUESTER_PAYS_MODE_KEY
```
@Deprecated
public static final java.lang.String GCS_REQUESTER_PAYS_MODE_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_REQUESTER_PAYS_MODE
  - GCS_REQUESTER_PAYS_PROJECT_ID_KEY
```
@Deprecated
public static final java.lang.String GCS_REQUESTER_PAYS_PROJECT_ID_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_REQUESTER_PAYS_PROJECT_ID
  - GCS_REQUESTER_PAYS_BUCKETS_KEY
```
@Deprecated
public static final java.lang.String GCS_REQUESTER_PAYS_BUCKETS_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_REQUESTER_PAYS_BUCKETS
  - GCS_CLIENT_ID_KEY
```
@Deprecated
public static final java.lang.String GCS_CLIENT_ID_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.AUTH_CLIENT_ID
  - GCS_CLIENT_SECRET_KEY
```
@Deprecated
public static final java.lang.String GCS_CLIENT_SECRET_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.AUTH_CLIENT_SECRET
  - GCS_SYSTEM_BUCKET_KEY
```
@Deprecated
public static final java.lang.String GCS_SYSTEM_BUCKET_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_SYSTEM_BUCKET
  - GCS_CREATE_SYSTEM_BUCKET_KEY
```
@Deprecated
public static final java.lang.String GCS_CREATE_SYSTEM_BUCKET_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_CREATE_SYSTEM_BUCKET
  - GCS_CREATE_SYSTEM_BUCKET_DEFAULT
```
@Deprecated
public static final boolean GCS_CREATE_SYSTEM_BUCKET_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_CREATE_SYSTEM_BUCKET
  - GCS_WORKING_DIRECTORY_KEY
```
@Deprecated
public static final java.lang.String GCS_WORKING_DIRECTORY_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_WORKING_DIRECTORY
  - GCS_FILE_SIZE_LIMIT_250GB
```
@Deprecated
public static final java.lang.String GCS_FILE_SIZE_LIMIT_250GB
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_FILE_SIZE_LIMIT_250GB
  - GCS_FILE_SIZE_LIMIT_250GB_DEFAULT
```
@Deprecated
public static final boolean GCS_FILE_SIZE_LIMIT_250GB_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_FILE_SIZE_LIMIT_250GB
  - GCS_MARKER_FILE_PATTERN_KEY
```
@Deprecated
public static final java.lang.String GCS_MARKER_FILE_PATTERN_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_MARKER_FILE_PATTERN
  - GCS_ENABLE_PERFORMANCE_CACHE_KEY
```
@Deprecated
public static final java.lang.String GCS_ENABLE_PERFORMANCE_CACHE_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_ENABLE
  - GCS_ENABLE_PERFORMANCE_CACHE_DEFAULT
```
@Deprecated
public static final boolean GCS_ENABLE_PERFORMANCE_CACHE_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_ENABLE
  - GCS_PERFORMANCE_CACHE_MAX_ENTRY_AGE_MILLIS_KEY
```
@Deprecated
public static final java.lang.String GCS_PERFORMANCE_CACHE_MAX_ENTRY_AGE_MILLIS_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_MAX_ENTRY_AGE_MILLIS
  - GCS_PERFORMANCE_CACHE_MAX_ENTRY_AGE_MILLIS_DEFAULT
```
@Deprecated
public static final long GCS_PERFORMANCE_CACHE_MAX_ENTRY_AGE_MILLIS_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_MAX_ENTRY_AGE_MILLIS
  - GCS_PERFORMANCE_CACHE_LIST_CACHING_ENABLE_KEY
```
@Deprecated
public static final java.lang.String GCS_PERFORMANCE_CACHE_LIST_CACHING_ENABLE_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_LIST_CACHING_ENABLE
  - GCS_PERFORMANCE_CACHE_LIST_CACHING_ENABLE_DEFAULT
```
@Deprecated
public static final boolean GCS_PERFORMANCE_CACHE_LIST_CACHING_ENABLE_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_LIST_CACHING_ENABLE
  - GCS_PERFORMANCE_CACHE_DIR_METADATA_PREFETCH_LIMIT_KEY
```
@Deprecated
public static final java.lang.String GCS_PERFORMANCE_CACHE_DIR_METADATA_PREFETCH_LIMIT_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_DIR_METADATA_PREFETCH_LIMIT
  - GCS_PERFORMANCE_CACHE_DIR_METADATA_PREFETCH_LIMIT_DEFAULT
```
@Deprecated
public static final long GCS_PERFORMANCE_CACHE_DIR_METADATA_PREFETCH_LIMIT_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_DIR_METADATA_PREFETCH_LIMIT
  - GCS_PARENT_TIMESTAMP_UPDATE_ENABLE_KEY
```
@Deprecated
public static final java.lang.String GCS_PARENT_TIMESTAMP_UPDATE_ENABLE_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_PARENT_TIMESTAMP_UPDATE_ENABLE
  - GCS_PARENT_TIMESTAMP_UPDATE_ENABLE_DEFAULT
```
@Deprecated
public static final boolean GCS_PARENT_TIMESTAMP_UPDATE_ENABLE_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_PARENT_TIMESTAMP_UPDATE_ENABLE
  - GCS_PARENT_TIMESTAMP_UPDATE_EXCLUDES_KEY
```
@Deprecated
public static final java.lang.String GCS_PARENT_TIMESTAMP_UPDATE_EXCLUDES_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_PARENT_TIMESTAMP_UPDATE_EXCLUDES
  - GCS_PARENT_TIMESTAMP_UPDATE_EXCLUDES_DEFAULT
```
@Deprecated
public static final java.lang.String GCS_PARENT_TIMESTAMP_UPDATE_EXCLUDES_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_PARENT_TIMESTAMP_UPDATE_EXCLUDES
  - MR_JOB_HISTORY_INTERMEDIATE_DONE_DIR_KEY
```
@Deprecated
public static final java.lang.String MR_JOB_HISTORY_INTERMEDIATE_DONE_DIR_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.MR_JOB_HISTORY_INTERMEDIATE_DONE_DIR_KEY
    
    See Also:
    
    Constant Field Values
  - MR_JOB_HISTORY_DONE_DIR_KEY
```
@Deprecated
public static final java.lang.String MR_JOB_HISTORY_DONE_DIR_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.MR_JOB_HISTORY_DONE_DIR_KEY
    
    See Also:
    
    Constant Field Values
  - GCS_PARENT_TIMESTAMP_UPDATE_INCLUDES_KEY
```
@Deprecated
public static final java.lang.String GCS_PARENT_TIMESTAMP_UPDATE_INCLUDES_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_PARENT_TIMESTAMP_UPDATE_INCLUDES
  - GCS_PARENT_TIMESTAMP_UPDATE_INCLUDES_DEFAULT
```
@Deprecated
public static final java.lang.String GCS_PARENT_TIMESTAMP_UPDATE_INCLUDES_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_PARENT_TIMESTAMP_UPDATE_INCLUDES
  - GCS_ENABLE_REPAIR_IMPLICIT_DIRECTORIES_KEY
```
@Deprecated
public static final java.lang.String GCS_ENABLE_REPAIR_IMPLICIT_DIRECTORIES_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_REPAIR_IMPLICIT_DIRECTORIES_ENABLE
  - GCS_ENABLE_REPAIR_IMPLICIT_DIRECTORIES_DEFAULT
```
@Deprecated
public static final boolean GCS_ENABLE_REPAIR_IMPLICIT_DIRECTORIES_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_REPAIR_IMPLICIT_DIRECTORIES_ENABLE
  - PATH_CODEC_KEY
```
@Deprecated
public static final java.lang.String PATH_CODEC_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.PATH_CODEC
  - PATH_CODEC_DEFAULT
```
@Deprecated
public static final java.lang.String PATH_CODEC_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.PATH_CODEC
  - GCS_ENABLE_INFER_IMPLICIT_DIRECTORIES_KEY
```
@Deprecated
public static final java.lang.String GCS_ENABLE_INFER_IMPLICIT_DIRECTORIES_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_INFER_IMPLICIT_DIRECTORIES_ENABLE
  - GCS_ENABLE_INFER_IMPLICIT_DIRECTORIES_DEFAULT
```
@Deprecated
public static final boolean GCS_ENABLE_INFER_IMPLICIT_DIRECTORIES_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_INFER_IMPLICIT_DIRECTORIES_ENABLE
  - GCS_ENABLE_FLAT_GLOB_KEY
```
@Deprecated
public static final java.lang.String GCS_ENABLE_FLAT_GLOB_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_FLAT_GLOB_ENABLE
  - GCS_ENABLE_FLAT_GLOB_DEFAULT
```
@Deprecated
public static final boolean GCS_ENABLE_FLAT_GLOB_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_FLAT_GLOB_ENABLE
  - GCS_ENABLE_MARKER_FILE_CREATION_KEY
```
@Deprecated
public static final java.lang.String GCS_ENABLE_MARKER_FILE_CREATION_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_MARKER_FILE_CREATION_ENABLE
  - GCS_ENABLE_MARKER_FILE_CREATION_DEFAULT
```
@Deprecated
public static final boolean GCS_ENABLE_MARKER_FILE_CREATION_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_MARKER_FILE_CREATION_ENABLE
  - GCS_ENABLE_COPY_WITH_REWRITE_KEY
```
@Deprecated
public static final java.lang.String GCS_ENABLE_COPY_WITH_REWRITE_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_COPY_WITH_REWRITE_ENABLE
  - GCS_ENABLE_COPY_WITH_REWRITE_DEFAULT
```
@Deprecated
public static final boolean GCS_ENABLE_COPY_WITH_REWRITE_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_COPY_WITH_REWRITE_ENABLE
  - GCS_COPY_MAX_REQUESTS_PER_BATCH
```
@Deprecated
public static final java.lang.String GCS_COPY_MAX_REQUESTS_PER_BATCH
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_COPY_MAX_REQUESTS_PER_BATCH
  - GCS_COPY_MAX_REQUESTS_PER_BATCH_DEFAULT
```
@Deprecated
public static final long GCS_COPY_MAX_REQUESTS_PER_BATCH_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_COPY_MAX_REQUESTS_PER_BATCH
  - GCS_COPY_BATCH_THREADS
```
@Deprecated
public static final java.lang.String GCS_COPY_BATCH_THREADS
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_COPY_BATCH_THREADS
  - GCS_COPY_BATCH_THREADS_DEFAULT
```
@Deprecated
public static final int GCS_COPY_BATCH_THREADS_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_COPY_BATCH_THREADS
  - GCS_MAX_LIST_ITEMS_PER_CALL
```
@Deprecated
public static final java.lang.String GCS_MAX_LIST_ITEMS_PER_CALL
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_MAX_LIST_ITEMS_PER_CALL
  - GCS_MAX_LIST_ITEMS_PER_CALL_DEFAULT
```
@Deprecated
public static final long GCS_MAX_LIST_ITEMS_PER_CALL_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_MAX_LIST_ITEMS_PER_CALL
  - GCS_MAX_REQUESTS_PER_BATCH
```
@Deprecated
public static final java.lang.String GCS_MAX_REQUESTS_PER_BATCH
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_MAX_REQUESTS_PER_BATCH
  - GCS_MAX_REQUESTS_PER_BATCH_DEFAULT
```
@Deprecated
public static final long GCS_MAX_REQUESTS_PER_BATCH_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_MAX_REQUESTS_PER_BATCH
  - GCS_BATCH_THREADS
```
@Deprecated
public static final java.lang.String GCS_BATCH_THREADS
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_BATCH_THREADS
  - GCS_BATCH_THREADS_DEFAULT
```
@Deprecated
public static final int GCS_BATCH_THREADS_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_BATCH_THREADS
  - GCS_HTTP_MAX_RETRY_KEY
```
@Deprecated
public static final java.lang.String GCS_HTTP_MAX_RETRY_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_HTTP_MAX_RETRY
  - GCS_HTTP_MAX_RETRY_DEFAULT
```
@Deprecated
public static final int GCS_HTTP_MAX_RETRY_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_HTTP_MAX_RETRY
  - GCS_HTTP_CONNECT_TIMEOUT_KEY
```
@Deprecated
public static final java.lang.String GCS_HTTP_CONNECT_TIMEOUT_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_HTTP_CONNECT_TIMEOUT
  - GCS_HTTP_CONNECT_TIMEOUT_DEFAULT
```
@Deprecated
public static final int GCS_HTTP_CONNECT_TIMEOUT_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_HTTP_CONNECT_TIMEOUT
  - GCS_HTTP_READ_TIMEOUT_KEY
```
@Deprecated
public static final java.lang.String GCS_HTTP_READ_TIMEOUT_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_HTTP_READ_TIMEOUT
  - GCS_HTTP_READ_TIMEOUT_DEFAULT
```
@Deprecated
public static final int GCS_HTTP_READ_TIMEOUT_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_HTTP_READ_TIMEOUT
  - GCS_PROXY_ADDRESS_KEY
```
@Deprecated
public static final java.lang.String GCS_PROXY_ADDRESS_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_PROXY_ADDRESS
  - GCS_PROXY_ADDRESS_DEFAULT
```
@Deprecated
public static final java.lang.String GCS_PROXY_ADDRESS_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_PROXY_ADDRESS
  - GCS_HTTP_TRANSPORT_KEY
```
@Deprecated
public static final java.lang.String GCS_HTTP_TRANSPORT_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_HTTP_TRANSPORT
  - GCS_HTTP_TRANSPORT_DEFAULT
```
@Deprecated
public static final java.lang.String GCS_HTTP_TRANSPORT_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_HTTP_TRANSPORT
  - GCS_APPLICATION_NAME_SUFFIX_KEY
```
@Deprecated
public static final java.lang.String GCS_APPLICATION_NAME_SUFFIX_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_APPLICATION_NAME_SUFFIX
  - GCS_APPLICATION_NAME_SUFFIX_DEFAULT
```
@Deprecated
public static final java.lang.String GCS_APPLICATION_NAME_SUFFIX_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_APPLICATION_NAME_SUFFIX
  - GCS_MAX_WAIT_MILLIS_EMPTY_OBJECT_CREATE_KEY
```
@Deprecated
public static final java.lang.String GCS_MAX_WAIT_MILLIS_EMPTY_OBJECT_CREATE_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_MAX_WAIT_MILLIS_EMPTY_OBJECT_CREATE
  - GCS_MAX_WAIT_MILLIS_EMPTY_OBJECT_CREATE_DEFAULT
```
@Deprecated
public static final int GCS_MAX_WAIT_MILLIS_EMPTY_OBJECT_CREATE_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_MAX_WAIT_MILLIS_EMPTY_OBJECT_CREATE
  - GCS_OUTPUTSTREAM_TYPE_KEY
```
@Deprecated
public static final java.lang.String GCS_OUTPUTSTREAM_TYPE_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_TYPE
  - GCS_OUTPUTSTREAM_TYPE_DEFAULT
```
@Deprecated
public static final java.lang.String GCS_OUTPUTSTREAM_TYPE_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_TYPE
  - GCS_GENERATION_READ_CONSISTENCY_KEY
```
@Deprecated
public static final java.lang.String GCS_GENERATION_READ_CONSISTENCY_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_GENERATION_READ_CONSISTENCY
  - GCS_GENERATION_READ_CONSISTENCY_DEFAULT
```
@Deprecated
public static final GoogleCloudStorageReadOptions.GenerationReadConsistency GCS_GENERATION_READ_CONSISTENCY_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_GENERATION_READ_CONSISTENCY
  - GCS_INPUTSTREAM_FAST_FAIL_ON_NOT_FOUND_ENABLE_KEY
```
@Deprecated
public static final java.lang.String GCS_INPUTSTREAM_FAST_FAIL_ON_NOT_FOUND_ENABLE_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_FAST_FAIL_ON_NOT_FOUND_ENABLE
  - GCS_INPUTSTREAM_FAST_FAIL_ON_NOT_FOUND_ENABLE_DEFAULT
```
@Deprecated
public static final boolean GCS_INPUTSTREAM_FAST_FAIL_ON_NOT_FOUND_ENABLE_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_FAST_FAIL_ON_NOT_FOUND_ENABLE
  - GCS_INPUTSTREAM_INPLACE_SEEK_LIMIT_KEY
```
@Deprecated
public static final java.lang.String GCS_INPUTSTREAM_INPLACE_SEEK_LIMIT_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_INPLACE_SEEK_LIMIT
  - GCS_INPUTSTREAM_INPLACE_SEEK_LIMIT_DEFAULT
```
@Deprecated
public static final long GCS_INPUTSTREAM_INPLACE_SEEK_LIMIT_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_INPLACE_SEEK_LIMIT
  - GCS_INPUTSTREAM_FADVISE_KEY
```
@Deprecated
public static final java.lang.String GCS_INPUTSTREAM_FADVISE_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_FADVISE
  - GCS_INPUTSTREAM_FADVISE_DEFAULT
```
@Deprecated
public static final GoogleCloudStorageReadOptions.Fadvise GCS_INPUTSTREAM_FADVISE_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_FADVISE
  - GCS_INPUTSTREAM_MIN_RANGE_REQUEST_SIZE_KEY
```
@Deprecated
public static final java.lang.String GCS_INPUTSTREAM_MIN_RANGE_REQUEST_SIZE_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_MIN_RANGE_REQUEST_SIZE
  - GCS_INPUTSTREAM_MIN_RANGE_REQUEST_SIZE_DEFAULT
```
@Deprecated
public static final int GCS_INPUTSTREAM_MIN_RANGE_REQUEST_SIZE_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_MIN_RANGE_REQUEST_SIZE
  - GCE_BUCKET_DELETE_ENABLE_KEY
```
@Deprecated
public static final java.lang.String GCE_BUCKET_DELETE_ENABLE_KEY
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCE_BUCKET_DELETE_ENABLE
  - GCE_BUCKET_DELETE_ENABLE_DEFAULT
```
@Deprecated
public static final boolean GCE_BUCKET_DELETE_ENABLE_DEFAULT
```
    Deprecated. use GoogleHadoopFileSystemConfiguration.GCE_BUCKET_DELETE_ENABLE
- Constructor Detail
  - GoogleHadoopFileSystemBase
```
public GoogleHadoopFileSystemBase()
```
    Constructs an instance of GoogleHadoopFileSystemBase; the internal GoogleCloudStorageFileSystem will be set up with config settings when initialize() is called.
  - GoogleHadoopFileSystemBase
```
public GoogleHadoopFileSystemBase(GoogleCloudStorageFileSystem gcsfs)
```
    Constructs an instance of GoogleHadoopFileSystemBase using the provided GoogleCloudStorageFileSystem; initialize() will not re-initialize it.
- Method Detail
  - createCounterMap
```
protected com.google.common.collect.ImmutableMap<GoogleHadoopFileSystemBase.Counter,java.util.concurrent.atomic.AtomicLong> createCounterMap()
```
  - setListStatusFileNotFoundBehavior
```
protected void setListStatusFileNotFoundBehavior(GoogleHadoopFileSystemBase.ListStatusFileNotFoundBehavior behavior)
```
  - getHomeDirectorySubpath
```
protected abstract java.lang.String getHomeDirectorySubpath()
```
    Returns an unqualified path without any leading slash, relative to the filesystem root, which serves as the home directory of the current user; see getHomeDirectory for a description of what the home directory means.
  - getHadoopPath
```
public abstract org.apache.hadoop.fs.Path getHadoopPath(java.net.URI gcsPath)
```
    Gets Hadoop path corresponding to the given GCS path.
    Parameters:
    gcsPath - Fully-qualified GCS path, of the form gs:///

Class GoogleHadoopFileSystemBase

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.fs.FileSystem

Field Summary

Fields inherited from class org.apache.hadoop.fs.FileSystem

Constructor Summary

Method Summary

Methods inherited from class org.apache.hadoop.fs.FileSystem

Methods inherited from class org.apache.hadoop.conf.Configured

Methods inherited from class java.lang.Object

Field Detail

PATH_CODEC_USE_URI_ENCODING

PATH_CODEC_USE_LEGACY_ENCODING

REPLICATION_FACTOR_DEFAULT

DEFAULT_FILTER

AUTHENTICATION_PREFIX

PROPERTIES_FILE

VERSION_PROPERTY

UNKNOWN_VERSION

VERSION

GHFS_ID

initUri

systemBucket

gcsfs

defaultBlockSize

counters

listStatusFileNotFoundBehavior

PERMISSIONS_TO_REPORT_KEY

PERMISSIONS_TO_REPORT_DEFAULT

BUFFERSIZE_KEY

BUFFERSIZE_DEFAULT

WRITE_BUFFERSIZE_KEY

WRITE_BUFFERSIZE_DEFAULT

BLOCK_SIZE_KEY

BLOCK_SIZE_DEFAULT

ENABLE_GCE_SERVICE_ACCOUNT_AUTH_KEY

SERVICE_ACCOUNT_AUTH_EMAIL_KEY

SERVICE_ACCOUNT_AUTH_KEYFILE_KEY

GCS_PROJECT_ID_KEY

GCS_REQUESTER_PAYS_MODE_KEY

GCS_REQUESTER_PAYS_PROJECT_ID_KEY

GCS_REQUESTER_PAYS_BUCKETS_KEY

GCS_CLIENT_ID_KEY

GCS_CLIENT_SECRET_KEY

GCS_SYSTEM_BUCKET_KEY

GCS_CREATE_SYSTEM_BUCKET_KEY

GCS_CREATE_SYSTEM_BUCKET_DEFAULT

GCS_WORKING_DIRECTORY_KEY

GCS_FILE_SIZE_LIMIT_250GB

GCS_FILE_SIZE_LIMIT_250GB_DEFAULT

GCS_MARKER_FILE_PATTERN_KEY

GCS_ENABLE_PERFORMANCE_CACHE_KEY

GCS_ENABLE_PERFORMANCE_CACHE_DEFAULT

GCS_PERFORMANCE_CACHE_MAX_ENTRY_AGE_MILLIS_KEY

GCS_PERFORMANCE_CACHE_MAX_ENTRY_AGE_MILLIS_DEFAULT

GCS_PERFORMANCE_CACHE_LIST_CACHING_ENABLE_KEY

GCS_PERFORMANCE_CACHE_LIST_CACHING_ENABLE_DEFAULT

GCS_PERFORMANCE_CACHE_DIR_METADATA_PREFETCH_LIMIT_KEY

GCS_PERFORMANCE_CACHE_DIR_METADATA_PREFETCH_LIMIT_DEFAULT

GCS_PARENT_TIMESTAMP_UPDATE_ENABLE_KEY

GCS_PARENT_TIMESTAMP_UPDATE_ENABLE_DEFAULT

GCS_PARENT_TIMESTAMP_UPDATE_EXCLUDES_KEY

GCS_PARENT_TIMESTAMP_UPDATE_EXCLUDES_DEFAULT

MR_JOB_HISTORY_INTERMEDIATE_DONE_DIR_KEY

MR_JOB_HISTORY_DONE_DIR_KEY

GCS_PARENT_TIMESTAMP_UPDATE_INCLUDES_KEY

GCS_PARENT_TIMESTAMP_UPDATE_INCLUDES_DEFAULT

GCS_ENABLE_REPAIR_IMPLICIT_DIRECTORIES_KEY

GCS_ENABLE_REPAIR_IMPLICIT_DIRECTORIES_DEFAULT

PATH_CODEC_KEY

PATH_CODEC_DEFAULT

GCS_ENABLE_INFER_IMPLICIT_DIRECTORIES_KEY

GCS_ENABLE_INFER_IMPLICIT_DIRECTORIES_DEFAULT

GCS_ENABLE_FLAT_GLOB_KEY

GCS_ENABLE_FLAT_GLOB_DEFAULT

GCS_ENABLE_MARKER_FILE_CREATION_KEY

GCS_ENABLE_MARKER_FILE_CREATION_DEFAULT

GCS_ENABLE_COPY_WITH_REWRITE_KEY

GCS_ENABLE_COPY_WITH_REWRITE_DEFAULT

GCS_COPY_MAX_REQUESTS_PER_BATCH