A B C D E F G H I L M N O P Q R S T U V W

A

addModificationTimeToAttributes(Map<String, byte[]>, Clock) - Static method in class com.google.cloud.hadoop.gcsio.FileInfo

Add a key and value representing the current time, as determined by the passed clock, to the passed attributes dictionary.

append(Path, int, Progressable) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Appends to an existing file (optional operation).

apply(String) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.ParentTimestampUpdateIncludePredicate

Determine if updating directory timestamps should be ignored.

apply(Storage.Objects.Insert) - Method in class com.google.cloud.hadoop.gcsio.ObjectWriteConditions

Apply the conditions represented by this object to an Insert operation.

AUTHENTICATION_PREFIX - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

AUTO_REPAIR_IMPLICIT_DIRECTORIES_DEFAULT - Static variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions

Default setting for enabling auto-repair of implicit directories.

B

BatchHelper - Class in com.google.cloud.hadoop.gcsio

BatchHelper abstracts out the logic for maximum requests per batch, and also allows a workaround for the fact that BatchRequest was made a "final class" for some reason, making it impossible to unittest.

BatchHelper() - Constructor for class com.google.cloud.hadoop.gcsio.BatchHelper

BatchHelper.Factory - Class in com.google.cloud.hadoop.gcsio

Since each BatchHelper instance should be tied to a particular related set of requests, use cases will generally interact via an injectable BatchHelper.Factory.

BatchHelper.Factory() - Constructor for class com.google.cloud.hadoop.gcsio.BatchHelper.Factory

BatchHelper.QueueRequestCallback - Interface in com.google.cloud.hadoop.gcsio

Callback that causes a single StorageRequest to be added to the BatchRequest.

BLOCK_SIZE_DEFAULT - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

BLOCK_SIZE_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

BUCKET_EMPTY_MAX_RETRIES - Static variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

BUCKET_EMPTY_WAIT_TIME_MS - Static variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

BUFFERSIZE_DEFAULT - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

BUFFERSIZE_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

build() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystemOptions.Builder

build() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions.Builder

C

cacheBasePath - Variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystemOptions.Builder

cacheConfig - Variable in class com.google.cloud.hadoop.gcsio.DirectoryListCache

CachedBucket - Class in com.google.cloud.hadoop.gcsio

A cache entry for the relevant metadata of a single GCS Bucket, used by DirectoryListCache to supplement "list" operations with buckets known locally, as well as serving as the container for retrieving CachedObjects from their respective cached buckets.

CachedBucket(String) - Constructor for class com.google.cloud.hadoop.gcsio.CachedBucket

Constructs a CachedBucket which has no associated GoogleCloudStorageItemInfo for the bucket.

CachedBucket(GoogleCloudStorageItemInfo) - Constructor for class com.google.cloud.hadoop.gcsio.CachedBucket

Constructs a CachedBucket with the provided GoogleCloudStorageItemInfo for the bucket.

CacheEntry - Class in com.google.cloud.hadoop.gcsio

Container for various pieces of metadata for a single cache entry which may be either a Bucket or a StorageObject; includes GCS API metadata such as StorageResourceId and GoogleCloudStorageItemInfo as well as cache-specific metadata, such as creation time, and last-updated time.

CacheEntry(StorageResourceId) - Constructor for class com.google.cloud.hadoop.gcsio.CacheEntry

Constructs a CacheEntry with no known GoogleCloudStorageItemInfo; callers may have to fetch the associated GoogleCloudStorageItemInfo on-demand.

CacheEntry(StorageResourceId, long) - Constructor for class com.google.cloud.hadoop.gcsio.CacheEntry

Constructs a CacheEntry with no known GoogleCloudStorageItemInfo and an explicit creationTimeMillis; callers may have to fetch the associated GoogleCloudStorageItemInfo on-demand.

CacheEntry(GoogleCloudStorageItemInfo) - Constructor for class com.google.cloud.hadoop.gcsio.CacheEntry

CacheSupplementedGoogleCloudStorage - Class in com.google.cloud.hadoop.gcsio

CacheSupplementedGoogleCloudStorage adds additional book-keeping to a GoogleCloudStorage instance using a DirectoryListCache and wraps the create/copy/delete/list methods to provide immediate same-client consistency for "list" operations following a "create/copy/delete".

CacheSupplementedGoogleCloudStorage(GoogleCloudStorage, DirectoryListCache) - Constructor for class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

Constructs a CacheSupplementedGoogleCloudStorage which should be usable anywhere a GoogleCloudStorage interface is used and which supplements missing listObject/listBucket results from an in-memory cache of known GCS resources which may not have propagated into the eventually-consistent remote "list" index yet.

cacheType - Variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystemOptions.Builder

checkPath(Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem

checkPath(Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

checkPath(Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

Only accept valid AbstractFileSystem and GoogleHadoopFileSystem Paths.

cleanCache(DirectoryListCache) - Static method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemCacheCleaner

Iterates over items in cache, object-first then buckets, allowing the list operations to perform cache-expiration as they run.

clearItemInfo() - Method in class com.google.cloud.hadoop.gcsio.CacheEntry

Clears the GoogleCloudStorageItemInfo stored by this CacheEntry, if any, and sets itemInfoUpdateTimeMillis to 0.

clock - Variable in class com.google.cloud.hadoop.gcsio.DirectoryListCache

close() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

close() - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

Pure pass-through.

close() - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Releases resources used by this instance.

close() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Releases resources used by this instance.

close() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

See close() for details about expected behavior.

close() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageReadChannel

Closes this channel.

close() - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

close() - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

com.google.cloud.hadoop.fs.gcs - package com.google.cloud.hadoop.fs.gcs

com.google.cloud.hadoop.gcsio - package com.google.cloud.hadoop.gcsio

completeLocalOutput(Path, Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

configureBuckets(String, boolean) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem

Validates and possibly creates the system bucket.

configureBuckets(String, boolean) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Validates and possibly creates the system bucket.

containsEntriesForImplicitDirectories() - Method in class com.google.cloud.hadoop.gcsio.DirectoryListCache

Implementations should indicate whether implicit directories are automatically detected and added as CacheEntry items when putResourceId is called without first adding explicit directory objects for the implied parent directories of the added object.

containsEntriesForImplicitDirectories() - Method in class com.google.cloud.hadoop.gcsio.FileSystemBackedDirectoryListCache

Since our backing store has no notion of "implicit" directories, automatically cache directory objects for 'implicit' directories, by virtue of having to create those real directories to insert the child object at all.

containsEntriesForImplicitDirectories() - Method in class com.google.cloud.hadoop.gcsio.InMemoryDirectoryListCache

We don't inspect StorageResourceIds in putResourceId to auto-insert entries for parent directories, nor do we return fake entries in getObjectList when a delimiter implies a pure-prefix match.

convertToDirectoryPath(StorageResourceId) - Static method in class com.google.cloud.hadoop.gcsio.FileInfo

Converts the given resourceId to look like a directory path.

convertToDirectoryPath(URI) - Static method in class com.google.cloud.hadoop.gcsio.FileInfo

Converts the given path to look like a directory path.

convertToFilePath(String) - Static method in class com.google.cloud.hadoop.gcsio.FileInfo

Converts the given object name to look like a file path.

copy(String, List<String>, String, List<String>) - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

Adds the copied destination items to the list cache, without their associated metadata; supplementing with the cache will have to populate the metadata on-demand.

copy(String, List<String>, String, List<String>) - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Copies metadata of the given objects.

copy(String, List<String>, String, List<String>) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

See copy(String, List, String, List) for details about expected behavior.

copy(String, List<String>, String, List<String>) - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

copy(String, List<String>, String, List<String>) - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

copyFromLocalFile(boolean, boolean, Path[], Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

copyFromLocalFile(boolean, boolean, Path, Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

copyToLocalFile(boolean, Path, Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

counters - Variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

create(Path, FsPermission, boolean, int, short, long, Progressable) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Opens the given file for writing.

create(Configuration) - Static method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.ParentTimestampUpdateIncludePredicate

Create a new ParentTimestampUpdateIncludePredicate from the passed Hadoop configuration object.

create(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

Wraps the delegate's returned WritableByteChannel in a helper which will update the resourceCache when close() is called.

create(StorageResourceId, CreateObjectOptions) - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

create(String) - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

Updates cache with bucketName.

create(StorageResourceId) - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Creates and opens an object for writing.

create(StorageResourceId, CreateObjectOptions) - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Creates and opens an object for writing.

create(String) - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Creates a bucket.

create(URI) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Creates and opens an object for writing.

create(URI, CreateFileOptions) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Creates and opens an object for writing.

create(StorageResourceId, CreateObjectOptions) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

create(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

See create(StorageResourceId) for details about expected behavior.

create(String) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

See create(String) for details about expected behavior.

create(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

create(StorageResourceId, CreateObjectOptions) - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

create(String) - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

create(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

create(StorageResourceId, CreateObjectOptions) - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

create(String) - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

CREATE_EMPTY_MARKER_OBJECT_DEFAULT - Static variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions

Default setting for whether or not to create a marker file when beginning file creation.

createCompositeException(List<IOException>) - Static method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageExceptions

Creates a composite IOException out of multiple IOExceptions.

createCounterMap() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

createEmptyObject(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

Records the resourceId after delegating.

createEmptyObject(StorageResourceId, CreateObjectOptions) - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

createEmptyObject(StorageResourceId) - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Creates an empty object, useful for placeholders representing, for example, directories.

createEmptyObject(StorageResourceId, CreateObjectOptions) - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Creates an empty object, useful for placeholders representing, for example, directories.

createEmptyObject(StorageResourceId, CreateObjectOptions) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

createEmptyObject(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

See createEmptyObject(StorageResourceId) for details about expected behavior.

createEmptyObject(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

createEmptyObject(StorageResourceId, CreateObjectOptions) - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

createEmptyObject(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

createEmptyObject(StorageResourceId, CreateObjectOptions) - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

createEmptyObjects(List<StorageResourceId>) - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

Records the resourceIds after delegating.

createEmptyObjects(List<StorageResourceId>, CreateObjectOptions) - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

createEmptyObjects(List<StorageResourceId>) - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Creates a list of empty objects; see createEmptyObject(StorageResourceId) for the single-item version of this method.

createEmptyObjects(List<StorageResourceId>, CreateObjectOptions) - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Creates a list of empty objects; see createEmptyObject(StorageResourceId) for the single-item version of this method.

createEmptyObjects(List<StorageResourceId>, CreateObjectOptions) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

createEmptyObjects(List<StorageResourceId>) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

See createEmptyObjects(List) for details about expected behavior.

createEmptyObjects(List<StorageResourceId>) - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

createEmptyObjects(List<StorageResourceId>, CreateObjectOptions) - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

createEmptyObjects(List<StorageResourceId>) - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

createEmptyObjects(List<StorageResourceId>, CreateObjectOptions) - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

CreateFileOptions - Class in com.google.cloud.hadoop.gcsio

Options that can be specified when creating a file in the GoogleCloudFileSystem.

CreateFileOptions(boolean) - Constructor for class com.google.cloud.hadoop.gcsio.CreateFileOptions

Create a file with empty attributes and optionally overwriting any existing file.

CreateFileOptions(boolean, Map<String, byte[]>) - Constructor for class com.google.cloud.hadoop.gcsio.CreateFileOptions

Create a file with specified attributes and optionally overwriting an existing file.

createInternal(Path, EnumSet<CreateFlag>, FsPermission, int, short, long, Progressable, Options.ChecksumOpt, boolean) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

CreateObjectOptions - Class in com.google.cloud.hadoop.gcsio

Options for creating objects in GCS.

CreateObjectOptions(boolean) - Constructor for class com.google.cloud.hadoop.gcsio.CreateObjectOptions

Construct a new CreateObjectOptions with empty metadata.

CreateObjectOptions(boolean, Map<String, byte[]>) - Constructor for class com.google.cloud.hadoop.gcsio.CreateObjectOptions

createReadableString(String, String) - Static method in class com.google.cloud.hadoop.gcsio.StorageResourceId

Helper for standardizing the way various human-readable messages in logs/exceptions which refer to a bucket/object pair.

createRequest(InputStreamContent) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageWriteChannel

D

DEFAULT - Static variable in class com.google.cloud.hadoop.gcsio.CreateFileOptions

DEFAULT - Static variable in class com.google.cloud.hadoop.gcsio.CreateObjectOptions

DEFAULT - Static variable in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl.BackOffFactory

DEFAULT_BACKOFF_INITIAL_INTERVAL_MILLIS - Static variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageReadChannel

DEFAULT_BACKOFF_MAX_ELAPSED_TIME_MILLIS - Static variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageReadChannel

DEFAULT_BACKOFF_MAX_INTERVAL_MILLIS - Static variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageReadChannel

DEFAULT_BACKOFF_MULTIPLIER - Static variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageReadChannel

DEFAULT_BACKOFF_RANDOMIZATION_FACTOR - Static variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageReadChannel

DEFAULT_FILTER - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

defaultBlockSize - Variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

delete(Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Deprecated.

Use delete(Path, boolean) instead

delete(Path, boolean) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Deletes the given file or directory.

delete(Path, boolean) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

delete(URI, boolean) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Deletes one or more items indicated by the given path.

deleteBuckets(List<String>) - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

Removes buckets from cache, if they exist.

deleteBuckets(List<String>) - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Deletes a list of buckets.

deleteBuckets(List<String>) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

See GoogleCloudStorage#deleteBuckets(List) for details about expected behavior.

deleteBuckets(List<String>) - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

deleteBuckets(List<String>) - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

deleteObjects(List<StorageResourceId>) - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

Removes objects from cache, if they exist.

deleteObjects(List<StorageResourceId>) - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Deletes the given objects.

deleteObjects(List<StorageResourceId>) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

See GoogleCloudStorage#deleteObjects(List) for details about expected behavior.

deleteObjects(List<StorageResourceId>) - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

deleteObjects(List<StorageResourceId>) - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

deleteOnExit(Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

DirectoryListCache - Class in com.google.cloud.hadoop.gcsio

DirectoryListCache is an abstract class providing supplemental accounting of full paths for directories and files created and deleted in GoogleCloudStorageFileSystem.

DirectoryListCache() - Constructor for class com.google.cloud.hadoop.gcsio.DirectoryListCache

DirectoryListCache.Config - Class in com.google.cloud.hadoop.gcsio

Container for various cache-configuration parameters used by a DirectoryListCache when managing expiration/retention policies, etc.

DirectoryListCache.Config() - Constructor for class com.google.cloud.hadoop.gcsio.DirectoryListCache.Config

DirectoryListCache.Type - Enum in com.google.cloud.hadoop.gcsio

Known types of DirectoryListCache implementations.

E

EMPTY_ATTRIBUTES - Static variable in class com.google.cloud.hadoop.gcsio.CreateFileOptions

ENABLE_GCE_SERVICE_ACCOUNT_AUTH_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

enqueue() - Method in interface com.google.cloud.hadoop.gcsio.BatchHelper.QueueRequestCallback

equals(Object) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageItemInfo

equals(Object) - Method in class com.google.cloud.hadoop.gcsio.StorageResourceId

execute() - Method in class com.google.cloud.hadoop.gcsio.OperationWithRetry

Execute the operation until while we have not yet completed successfully and the shouldRetryPredicate returns true and the backOff has not signalled completion.

exists() - Method in class com.google.cloud.hadoop.gcsio.FileInfo

Indicates whether this file or directory exists.

exists(URI) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Indicates whether the given item exists.

exists() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageItemInfo

Indicates whether this item exists.

F

FILE_MODIFICATION_TIMESTAMP_KEY - Static variable in class com.google.cloud.hadoop.gcsio.FileInfo

FileInfo - Class in com.google.cloud.hadoop.gcsio

Contains information about a file or a directory.

FileSystemBackedDirectoryListCache - Class in com.google.cloud.hadoop.gcsio

FileSystemBackedDirectoryListCache mirrors all GCS directory and file creation into a matching tree of directories + empty files inside a configured local filesystem, as accessed via java.nio.Path and java.io.File, all under a base directory configured at construction-time.

FileSystemBackedDirectoryListCache(String) - Constructor for class com.google.cloud.hadoop.gcsio.FileSystemBackedDirectoryListCache

FileSystemDescriptor - Interface in com.google.cloud.hadoop.fs.gcs

FileSystemDescriptor defines the interface containing all the methods which are necessary to differentiate the behavior of different Hadoop FileSystem implementations, including the URI scheme with which an implementation is identified.

flush() - Method in class com.google.cloud.hadoop.gcsio.BatchHelper

Sends any currently remaining requests in the batch; should be caleld at the end of any series of batched requests to ensure everything has been sent.

fromItemInfo(GoogleCloudStorageItemInfo) - Static method in class com.google.cloud.hadoop.gcsio.FileInfo

Handy factory method for constructing a FileInfo from a GoogleCloudStorageItemInfo while potentially returning a singleton instead of really constructing an object for cases like ROOT.

fromItemInfos(List<GoogleCloudStorageItemInfo>) - Static method in class com.google.cloud.hadoop.gcsio.FileInfo

Handy factory method for constructing a list of FileInfo from a list of GoogleCloudStorageItemInfo.

G

GCS_CLIENT_ID_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_CLIENT_SECRET_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_CREATE_SYSTEM_BUCKET_DEFAULT - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_CREATE_SYSTEM_BUCKET_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_ENABLE_FLAT_GLOB_DEFAULT - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_ENABLE_FLAT_GLOB_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_ENABLE_MARKER_FILE_CREATION_DEFAULT - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_ENABLE_MARKER_FILE_CREATION_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_ENABLE_METADATA_CACHE_DEFAULT - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_ENABLE_METADATA_CACHE_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_ENABLE_REPAIR_IMPLICIT_DIRECTORIES_DEFAULT - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_ENABLE_REPAIR_IMPLICIT_DIRECTORIES_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_FILE_SIZE_LIMIT_250GB - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_FILE_SIZE_LIMIT_250GB_DEFAULT - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_METADATA_CACHE_DIRECTORY_DEFAULT - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_METADATA_CACHE_DIRECTORY_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_METADATA_CACHE_TYPE_DEFAULT - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_METADATA_CACHE_TYPE_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_PARENT_TIMESTAMP_UPDATE_ENABLE_DEFAULT - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_PARENT_TIMESTAMP_UPDATE_ENABLE_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_PARENT_TIMESTAMP_UPDATE_EXCLUDES_DEFAULT - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_PARENT_TIMESTAMP_UPDATE_EXCLUDES_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_PARENT_TIMESTAMP_UPDATE_INCLUDES_DEFAULT - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_PARENT_TIMESTAMP_UPDATE_INCLUDES_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_PROJECT_ID_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_ROOT - Static variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

GCS_SYSTEM_BUCKET_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

GCS_WORKING_DIRECTORY_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

gcsfs - Variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

get() - Static method in enum com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.ListStatusFileNotFoundBehavior

Get the ListStatusFileNotFoundBehavior for the currently running Hadoop version.

get(HadoopVersionInfo) - Static method in enum com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.ListStatusFileNotFoundBehavior

Get the ListStatusFileNotFoundBehavior for the given hadoop version/

get(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.CachedBucket

Returns the CacheEntry entry corresponding to resourceId which must be a StorageObject residing inside this CachedBucket, or null if one doesn't exist.

getAppName() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions

getAttributes() - Method in class com.google.cloud.hadoop.gcsio.CreateFileOptions

Extended attributes to set when creating a file.

getAttributes() - Method in class com.google.cloud.hadoop.gcsio.FileInfo

Retrieve file attributes for this file.

getBucketList() - Method in class com.google.cloud.hadoop.gcsio.DirectoryListCache

getBucketList() - Method in class com.google.cloud.hadoop.gcsio.FileSystemBackedDirectoryListCache

getBucketList() - Method in class com.google.cloud.hadoop.gcsio.InMemoryDirectoryListCache

getBucketName() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageItemInfo

Gets bucket name of this item.

getBucketName() - Method in class com.google.cloud.hadoop.gcsio.StorageResourceId

Gets the bucket name component of this resource identifier.

getCacheBasePath() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystemOptions

getCacheEntry(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.DirectoryListCache

Returns the CacheEntry associated with resourceId, or null if it doesn't exist.

getCacheEntry(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.FileSystemBackedDirectoryListCache

getCacheEntry(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.InMemoryDirectoryListCache

getCacheType() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystemOptions

getCanonicalServiceName() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Returns null, because GHFS does not use security tokens.

getCloudStorageOptions() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystemOptions

getCloudStorageOptionsBuilder() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystemOptions.Builder

getContentGeneration() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageItemInfo

Get the content generation of the object.

getContentGenerationMatch() - Method in class com.google.cloud.hadoop.gcsio.ObjectWriteConditions

getContentSummary(Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

getCreationTime() - Method in class com.google.cloud.hadoop.gcsio.FileInfo

Gets creation time of this item.

getCreationTime() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageItemInfo

Gets creation time of this item.

getCreationTimeMillis() - Method in class com.google.cloud.hadoop.gcsio.CacheEntry

Accessor for the creation-time of this CacheEntry, which was set at construction-time and is immutable.

getDefaultBlockSize() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

getDefaultPort() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

The default port is listed as -1 as an indication that ports are not used.

getDefaultReplication() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Gets the default replication factor.

getDefaultWorkingDirectory() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem

Gets the default value of working directory.

getDefaultWorkingDirectory() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Gets the default value of working directory.

getDefaultWorkingDirectory() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopGlobalRootedFileSystem

Deprecated.

Gets the default value of working directory.

getDelegationToken(String) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

getFileBlockLocations(Path, long, long) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

getFileChecksum(Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

getFileChecksum(Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

getFileInfo(URI) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Gets information about the given path item.

getFileInfos(List<URI>) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Gets information about each path in the given list; more efficient than calling getFileInfo() on each path individually in a loop.

getFileNotFoundException(String, String) - Static method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageExceptions

Creates FileNotFoundException with suitable message for a GCS bucket or object.

getFileStatus(Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Gets status of the given path item.

getFileStatus(Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

getFileSystemRoot() - Method in interface com.google.cloud.hadoop.fs.gcs.FileSystemDescriptor

Returns the Hadoop path representing the root of the FileSystem associated with this FileSystemDescriptor.

getFileSystemRoot() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem

getFileSystemRoot() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

getFileSystemRoot() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopGlobalRootedFileSystem

Deprecated.

getFsStatus() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

getGcsPath(Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem

Translates a "gs:/" style hadoopPath (or relative path which is not fully-qualified) into the appropriate GCS path which is compatible with the underlying GcsFs or gsutil.

getGcsPath(Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Gets GCS path corresponding to the given Hadoop path, which can be relative or absolute, and can have either gs:// or gs:/ forms.

getGcsPath(Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopGlobalRootedFileSystem

Deprecated.

getHadoopPath(URI) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem

Validates GCS Path belongs to this file system.

getHadoopPath(URI) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Gets Hadoop path corresponding to the given GCS path.

getHadoopPath(URI) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopGlobalRootedFileSystem

Deprecated.

Gets Hadoop path corresponding to the given GCS path.

getHadoopScheme() - Method in interface com.google.cloud.hadoop.fs.gcs.FileSystemDescriptor

Deprecated.

getHadoopScheme() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Deprecated.

getHomeDirectory() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Returns home directory of the current user.

getHomeDirectorySubpath() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem

Override to allow a homedir subpath which sits directly on our FileSystem root.

getHomeDirectorySubpath() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Returns an unqualified path without any leading slash, relative to the filesystem root, which serves as the home directory of the current user; see getHomeDirectory for a description of what the home directory means.

getHomeDirectorySubpath() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopGlobalRootedFileSystem

Deprecated.

Returns an unqualified path without any leading slash, relative to the filesystem root, which serves as the home directory of the current user; see getHomeDirectory for a description of what the home directory means.

getInstance() - Static method in class com.google.cloud.hadoop.gcsio.InMemoryDirectoryListCache

Accessor for shared singleton instance of DirectoryListCache.

getInternalNumBuckets() - Method in class com.google.cloud.hadoop.gcsio.DirectoryListCache

Gets the internal number of CachedBucket entries, which may not be equal to the size of getBucketList() if there are expired entries.

getInternalNumBuckets() - Method in class com.google.cloud.hadoop.gcsio.FileSystemBackedDirectoryListCache

getInternalNumBuckets() - Method in class com.google.cloud.hadoop.gcsio.InMemoryDirectoryListCache

getInternalNumObjects() - Method in class com.google.cloud.hadoop.gcsio.DirectoryListCache

Gets the internal total count of cached StorageObject entries.

getInternalNumObjects() - Method in class com.google.cloud.hadoop.gcsio.FileSystemBackedDirectoryListCache

getInternalNumObjects() - Method in class com.google.cloud.hadoop.gcsio.InMemoryDirectoryListCache

getItemInfo() - Method in class com.google.cloud.hadoop.gcsio.CacheEntry

Returns the GoogleCloudStorageItemInfo currently held by this CacheEntry; may be null if one was never provided.

getItemInfo(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

Pure pass-through.

getItemInfo() - Method in class com.google.cloud.hadoop.gcsio.FileInfo

Gets information about the underlying item.

getItemInfo(StorageResourceId) - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Gets information about an object or a bucket.

getItemInfo(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

See getItemInfo(StorageResourceId) for details about expected behavior.

getItemInfo(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

Pure fetch from cache.

getItemInfo(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

getItemInfos(List<StorageResourceId>) - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

Pure pass-through.

getItemInfos(List<StorageResourceId>) - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Gets information about multiple objects and/or buckets.

getItemInfos(List<StorageResourceId>) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

See GoogleCloudStorage#getItemInfos(List) for details about expected behavior.

getItemInfos(List<StorageResourceId>) - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

Pure fetch from cache.

getItemInfos(List<StorageResourceId>) - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

getItemInfoUpdateTimeMillis() - Method in class com.google.cloud.hadoop.gcsio.CacheEntry

Accessor for the last time the GoogleCloudStorageItemInfo of this CacheEntry was updated, or 0 if it was never updated.

getLocation() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageItemInfo

Gets location of this item.

getMaxEntryAgeMillis() - Method in class com.google.cloud.hadoop.gcsio.DirectoryListCache.Config

Getter for maxEntryAgeMillis.

getMaxInfoAgeMillis() - Method in class com.google.cloud.hadoop.gcsio.DirectoryListCache.Config

Getter for maxInfoAgeMillis.

getMaxListItemsPerCall() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions

getMaxRequestsPerBatch() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions

getMetadata() - Method in class com.google.cloud.hadoop.gcsio.CreateObjectOptions

Custom metadata to apply to this object.

getMetadata() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageItemInfo

Gets user-supplied metadata for this item.

getMetadata() - Method in class com.google.cloud.hadoop.gcsio.UpdatableItemInfo

getMetaGeneration() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageItemInfo

Get the meta generation of the object.

getMetaGenerationMatch() - Method in class com.google.cloud.hadoop.gcsio.ObjectWriteConditions

getModificationTime() - Method in class com.google.cloud.hadoop.gcsio.FileInfo

Gets the modification time of this file if one is set, otherwise the value of FileInfo.getCreationTime() is returned.

getMutableConfig() - Method in class com.google.cloud.hadoop.gcsio.DirectoryListCache

Returns the Config instance used by this DirectoryListCache instance to determine expiration ages, etc.

getName() - Method in class com.google.cloud.hadoop.gcsio.CachedBucket

Returns the name of the Bucket associated with this CachedBucket.

getNumObjects() - Method in class com.google.cloud.hadoop.gcsio.CachedBucket

getObjectList() - Method in class com.google.cloud.hadoop.gcsio.CachedBucket

getObjectList(String, String, String, Set<String>) - Method in class com.google.cloud.hadoop.gcsio.DirectoryListCache

getObjectList(String, String, String, Set<String>) - Method in class com.google.cloud.hadoop.gcsio.FileSystemBackedDirectoryListCache

getObjectList(String, String, String, Set<String>) - Method in class com.google.cloud.hadoop.gcsio.InMemoryDirectoryListCache

getObjectName() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageItemInfo

Gets object name of this item.

getObjectName() - Method in class com.google.cloud.hadoop.gcsio.StorageResourceId

Gets the object name component of this resource identifier.

getParentPath(URI) - Static method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Gets the parent directory of the given path.

getPath() - Method in class com.google.cloud.hadoop.gcsio.FileInfo

Gets the path of this file or directory.

getPath(String) - Static method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Constructs and returns full path for the given bucket name.

getPath(String, String) - Static method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Constructs and returns full path for the given object name.

getPath(String, String, boolean) - Static method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Constructs and returns full path for the given object name.

getProjectId() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions

getRawBucketList() - Method in class com.google.cloud.hadoop.gcsio.DirectoryListCache

getRawBucketList() - Method in class com.google.cloud.hadoop.gcsio.FileSystemBackedDirectoryListCache

getRawBucketList() - Method in class com.google.cloud.hadoop.gcsio.InMemoryDirectoryListCache

getResourceId() - Method in class com.google.cloud.hadoop.gcsio.CacheEntry

Returns the StorageResourceId associated with this CacheEntry; may identify a Bucket or StorageObject.

getResourceId() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageItemInfo

Gets the resourceId which holds the (possibly null) bucketName and objectName of this object.

getScheme() - Method in interface com.google.cloud.hadoop.fs.gcs.FileSystemDescriptor

Returns the URI scheme for the Hadoop FileSystem associated with this FileSystemDescriptor.

getScheme() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem

As the global-rooted FileSystem, our hadoop-path "scheme" is exactly equal to the general GCS scheme.

getScheme() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

getScheme() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopGlobalRootedFileSystem

Deprecated.

As the global-rooted FileSystem, our hadoop-path "scheme" is distinct from GCS's scheme.

getServerDefaults() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

getShouldIncludeInTimestampUpdatesPredicate() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystemOptions

getSize() - Method in class com.google.cloud.hadoop.gcsio.FileInfo

Gets the size of this file or directory.

getSize() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageItemInfo

Gets size of this item (number of bytes).

getStorageClass() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageItemInfo

Gets storage class of this item.

getStorageResourceId() - Method in class com.google.cloud.hadoop.gcsio.UpdatableItemInfo

getUninitializedInstanceForTest() - Static method in class com.google.cloud.hadoop.gcsio.FileSystemBackedDirectoryListCache

Returns an instance not bound to a basePath for testing purposes.

getUri() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Returns a URI of the root of this FileSystem.

getUri() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

This is overridden to use GoogleHadoopFileSystem's URI, because AbstractFileSystem appends the default port to the authority.

getUriDefaultPort() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

getUsed() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

getWorkingDirectory() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Gets the current working directory.

getWriteChannelOptions() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions

getWriteChannelOptionsBuilder() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions.Builder

GHFS_ID - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

globStatus(Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Returns an array of FileStatus objects whose path names match pathPattern.

globStatus(Path, PathFilter) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Returns an array of FileStatus objects whose path names match pathPattern and is accepted by the user-supplied path filter.

GoogleCloudStorage - Interface in com.google.cloud.hadoop.gcsio

Interface for exposing the Google Cloud Storage API behavior in a way more amenable to writing filesystem semantics on top of it, without having to deal with API-specific considerations such as HttpTransports, credentials, network errors, batching, etc.

GoogleCloudStorageExceptions - Class in com.google.cloud.hadoop.gcsio

Miscellaneous helper methods for standardizing the types of exceptions thrown by the various GCS-based FileSystems.

GoogleCloudStorageExceptions() - Constructor for class com.google.cloud.hadoop.gcsio.GoogleCloudStorageExceptions

GoogleCloudStorageFileSystem - Class in com.google.cloud.hadoop.gcsio

Provides a POSIX like file system layered on top of Google Cloud Storage (GCS).

GoogleCloudStorageFileSystem(Credential, GoogleCloudStorageFileSystemOptions) - Constructor for class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Constructs an instance of GoogleCloudStorageFileSystem.

GoogleCloudStorageFileSystem(GoogleCloudStorage) - Constructor for class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Constructs a GoogleCloudStorageFilesystem based on an already-configured underlying GoogleCloudStorage gcs.

GoogleCloudStorageFileSystem(GoogleCloudStorage, GoogleCloudStorageFileSystemOptions) - Constructor for class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Constructs a GoogleCloudStorageFilesystem based on an already-configured underlying GoogleCloudStorage gcs.

GoogleCloudStorageFileSystemOptions - Class in com.google.cloud.hadoop.gcsio

Configurable options for the GoogleCloudStorageFileSystem class.

GoogleCloudStorageFileSystemOptions(GoogleCloudStorageOptions, boolean, DirectoryListCache.Type, String, Predicate<String>) - Constructor for class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystemOptions

GoogleCloudStorageFileSystemOptions.Builder - Class in com.google.cloud.hadoop.gcsio

Mutable builder for GoogleCloudStorageFileSystemOptions.

GoogleCloudStorageFileSystemOptions.Builder() - Constructor for class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystemOptions.Builder

GoogleCloudStorageImpl - Class in com.google.cloud.hadoop.gcsio

Provides read/write access to Google Cloud Storage (GCS), using Java nio channel semantics.

GoogleCloudStorageImpl(GoogleCloudStorageOptions, Credential) - Constructor for class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

Constructs an instance of GoogleCloudStorageImpl.

GoogleCloudStorageImpl(GoogleCloudStorageOptions, Storage) - Constructor for class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

Constructs an instance of GoogleCloudStorageImpl.

GoogleCloudStorageImpl() - Constructor for class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

GoogleCloudStorageImpl.BackOffFactory - Interface in com.google.cloud.hadoop.gcsio

A factory for producing BackOff objects.

GoogleCloudStorageItemInfo - Class in com.google.cloud.hadoop.gcsio

Contains information about an item in Google Cloud Storage.

GoogleCloudStorageItemInfo(StorageResourceId, long, long, String, String) - Constructor for class com.google.cloud.hadoop.gcsio.GoogleCloudStorageItemInfo

Constructs an instance of GoogleCloudStorageItemInfo.

GoogleCloudStorageItemInfo(StorageResourceId, long, long, String, String, Map<String, byte[]>, long, long) - Constructor for class com.google.cloud.hadoop.gcsio.GoogleCloudStorageItemInfo

Constructs an instance of GoogleCloudStorageItemInfo.

GoogleCloudStorageOptions - Class in com.google.cloud.hadoop.gcsio

Configuration options for the GoogleCloudStorage class.

GoogleCloudStorageOptions(boolean, String, String, long, long, boolean, AsyncWriteChannelOptions) - Constructor for class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions

GoogleCloudStorageOptions.Builder - Class in com.google.cloud.hadoop.gcsio

Mutable builder for the GoogleCloudStorageOptions class.

GoogleCloudStorageOptions.Builder() - Constructor for class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions.Builder

GoogleCloudStorageReadChannel - Class in com.google.cloud.hadoop.gcsio

Provides seekable read access to GCS.

GoogleCloudStorageStrings - Class in com.google.cloud.hadoop.gcsio

Contains helper methods for standardizing String-matching algorithms specific to GCS.

GoogleCloudStorageStrings() - Constructor for class com.google.cloud.hadoop.gcsio.GoogleCloudStorageStrings

GoogleCloudStorageWriteChannel - Class in com.google.cloud.hadoop.gcsio

Implements WritableByteChannel to provide write access to GCS.

GoogleCloudStorageWriteChannel(ExecutorService, Storage, ClientRequestHelper<StorageObject>, String, String, AsyncWriteChannelOptions, ObjectWriteConditions, Map<String, String>) - Constructor for class com.google.cloud.hadoop.gcsio.GoogleCloudStorageWriteChannel

Constructs an instance of GoogleCloudStorageWriteChannel.

GoogleHadoopFileSystem - Class in com.google.cloud.hadoop.fs.gcs

GoogleHadoopFileSystem is a version of GoogleHadoopFileSystemBase which is rooted in a single bucket at initialization time; in this case, Hadoop paths no longer correspond directly to general GCS paths, and all Hadoop operations going through this FileSystem will never touch any GCS bucket other than the bucket on which this FileSystem is rooted.

GoogleHadoopFileSystem() - Constructor for class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem

Constructs an instance of GoogleHadoopFileSystem; the internal GoogleCloudStorageFileSystem will be set up with config settings when initialize() is called.

GoogleHadoopFileSystem(GoogleCloudStorageFileSystem) - Constructor for class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem

Constructs an instance of GoogleHadoopFileSystem using the provided GoogleCloudStorageFileSystem; initialize() will not re-initialize it.

GoogleHadoopFileSystemBase - Class in com.google.cloud.hadoop.fs.gcs

This class provides a Hadoop compatible File System on top of Google Cloud Storage (GCS).

GoogleHadoopFileSystemBase() - Constructor for class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Constructs an instance of GoogleHadoopFileSystemBase; the internal GoogleCloudStorageFileSystem will be set up with config settings when initialize() is called.

GoogleHadoopFileSystemBase(GoogleCloudStorageFileSystem) - Constructor for class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Constructs an instance of GoogleHadoopFileSystemBase using the provided GoogleCloudStorageFileSystem; initialize() will not re-initialize it.

GoogleHadoopFileSystemBase.Counter - Enum in com.google.cloud.hadoop.fs.gcs

Defines names of counters we track for each operation.

GoogleHadoopFileSystemBase.ListStatusFileNotFoundBehavior - Enum in com.google.cloud.hadoop.fs.gcs

Behavior of listStatus when a path is not found.

GoogleHadoopFileSystemBase.ParentTimestampUpdateIncludePredicate - Class in com.google.cloud.hadoop.fs.gcs

A predicate that processes individual directory paths and evaluates the conditions set in fs.gs.parent.timestamp.update.enable, fs.gs.parent.timestamp.update.substrings.include and fs.gs.parent.timestamp.update.substrings.exclude to determine if a path should be ignored when running directory timestamp updates.

GoogleHadoopFileSystemBase.ParentTimestampUpdateIncludePredicate(boolean, List<String>, List<String>) - Constructor for class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.ParentTimestampUpdateIncludePredicate

GoogleHadoopFileSystemCacheCleaner - Class in com.google.cloud.hadoop.fs.gcs

A tool that will perform GC on FileSystemBackedDirectoryListCache backing stores.

GoogleHadoopFileSystemCacheCleaner() - Constructor for class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemCacheCleaner

GoogleHadoopFS - Class in com.google.cloud.hadoop.fs.gcs

GoogleHadoopFS provides a YARN compatible Abstract File System on top of Google Cloud Storage (GCS).

GoogleHadoopFS(URI, Configuration) - Constructor for class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

GoogleHadoopFS(GoogleHadoopFileSystem, URI, Configuration) - Constructor for class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

GoogleHadoopGlobalRootedFileSystem - Class in com.google.cloud.hadoop.fs.gcs

Deprecated.

This implementation is deprecated and relies on the deprecated systemBucket property. Use GoogleHadoopFileSystem instead.

GoogleHadoopGlobalRootedFileSystem() - Constructor for class com.google.cloud.hadoop.fs.gcs.GoogleHadoopGlobalRootedFileSystem

Deprecated.

Constructs an instance of GoogleHadoopGlobalRootedFileSystem; the internal GoogleCloudStorageFileSystem will be set up with config settings when initialize() is called.

GoogleHadoopGlobalRootedFileSystem(GoogleCloudStorageFileSystem) - Constructor for class com.google.cloud.hadoop.fs.gcs.GoogleHadoopGlobalRootedFileSystem

Deprecated.

Constructs an instance of GoogleHadoopGlobalRootedFileSystem using the provided GoogleCloudStorageFileSystem; initialize() will not re-initialize it.

H

handle(String) - Method in enum com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.ListStatusFileNotFoundBehavior

Perform version specific handling for a missing path.

hasContentGenerationMatch() - Method in class com.google.cloud.hadoop.gcsio.ObjectWriteConditions

hashCode() - Method in class com.google.cloud.hadoop.gcsio.StorageResourceId

hasMetaGenerationMatch() - Method in class com.google.cloud.hadoop.gcsio.ObjectWriteConditions

I

initialize(URI, Configuration) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

See GoogleHadoopFileSystemBase.initialize(URI, Configuration, boolean) for details; calls with third arg defaulting to 'true' for initializing the superclass.

initialize(URI, Configuration, boolean) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Initializes this file system instance.

initUri - Variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

InMemoryDirectoryListCache - Class in com.google.cloud.hadoop.gcsio

InMemoryDirectoryListCache provides in-memory accounting of full paths for directories and files created and deleted in GoogleCloudStorageFileSystem within a single running process.

InMemoryDirectoryListCache() - Constructor for class com.google.cloud.hadoop.gcsio.InMemoryDirectoryListCache

Callers should usually only obtain an instance via InMemoryDirectoryListCache.getInstance() so that cache info is shared process-wide, but instances can be created for temporary caches.

isAutoRepairImplicitDirectoriesEnabled() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions

isBucket() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageItemInfo

Indicates whether this item is a bucket.

isBucket() - Method in class com.google.cloud.hadoop.gcsio.StorageResourceId

Returns true if this StorageResourceId represents a GCS Bucket; if true, then getObjectName will return null.

isCacheEntryExpired(CacheEntry) - Method in class com.google.cloud.hadoop.gcsio.DirectoryListCache

Helper for determining whether a CacheEntry is entirely expired and should be removed from the cache.

isDirectory() - Method in class com.google.cloud.hadoop.gcsio.FileInfo

Indicates whether this item is a directory.

isDirectory() - Method in class com.google.cloud.hadoop.gcsio.StorageResourceId

Indicates if this StorageResourceId corresponds to a 'directory'; similar to FileInfo.isDirectory() except deals entirely with pathnames instead of also checking for exists() to be true on a corresponding GoogleCloudStorageItemInfo.

isDirectoryPath(URI) - Static method in class com.google.cloud.hadoop.gcsio.FileInfo

Indicates whether the given path looks like a directory path.

isEmpty() - Method in class com.google.cloud.hadoop.gcsio.BatchHelper

Returns true if there are no currently queued entries in the batch helper.

isGlobalRoot() - Method in class com.google.cloud.hadoop.gcsio.FileInfo

Indicates whether this instance has information about the unique, shared root of the underlying storage system.

isMarkerFileCreationEnabled() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions

isMetadataCacheEnabled() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystemOptions

isOpen() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageReadChannel

Tells whether this channel is open.

isRoot() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageItemInfo

Indicates whether this item refers to the GCS root (gs://).

isRoot() - Method in class com.google.cloud.hadoop.gcsio.StorageResourceId

Returns true if this StorageResourceId represents the GCS root (gs://); if true, then both getBucketName and getObjectName will be null.

isStorageObject() - Method in class com.google.cloud.hadoop.gcsio.StorageResourceId

Returns true if this StorageResourceId represents a GCS StorageObject; if true, both getBucketName and getObjectName will be non-empty and non-null.

isValidName(String) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

Follow HDFS conventions except allow for ':' in paths.

L

listAllFileInfoForPrefix(URI) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Equivalent to a recursive listing of prefix, except that prefix doesn't have to represent an actual object but can just be a partial prefix string, and there is no auto-repair of implicit directories since we can't detect implicit directories without listing by 'delimiter'.

listBucketInfo() - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

Supplements the list returned by the delegate with cached bucket infos; may trigger fetching of any metadata not already available in the cache.

listBucketInfo() - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Gets a list of GoogleCloudStorageItemInfo for all buckets of this project.

listBucketInfo() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

See listBucketInfo() for details about expected behavior.

listBucketInfo() - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

listBucketInfo() - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

listBucketNames() - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

Supplements the list returned by the delegate with cached bucket names; won't trigger any fetching of metadata.

listBucketNames() - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Gets a list of names of buckets in this project.

listBucketNames() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

See listBucketNames() for details about expected behavior.

listBucketNames() - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

listBucketNames() - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

listFileInfo(URI, boolean) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

If the given path points to a directory then the information about its children is returned, otherwise information about the given file is returned.

listFileNames(FileInfo) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

If the given item is a directory then the paths of its immediate children are returned, otherwise the path of the given item is returned.

listFileNames(FileInfo, boolean) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

If the given item is a directory then the paths of its children are returned, otherwise the path of the given item is returned.

listObjectInfo(String, String, String) - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

Supplements the list returned by the delegate with cached object infos; may trigger fetching of any metadata not already available in the cache.

listObjectInfo(String, String, String) - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Same name-matching semantics as listObjectNames except this method retrieves the full GoogleCloudStorageFileInfo for each item as well; generally the info is already available from the same "list()" calls, so the only additional cost is dispatching an extra batch request to retrieve object metadata for all listed *directories*, since these are originally listed as String prefixes without attached metadata.

listObjectInfo(String, String, String) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

See listObjectInfo(String, String, String) for details about expected behavior.

listObjectInfo(String, String, String) - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

Uses shared prefix-matching logic to filter entries from the metadata cache.

listObjectInfo(String, String, String) - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

listObjectNames(String, String, String) - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

Supplements the list returned by the delegate with cached object names; won't trigger any fetching of metadata.

listObjectNames(String, String, String) - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Gets names of objects contained in the given bucket and whose names begin with the given prefix.

listObjectNames(String, String, String) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

See listObjectNames(String, String, String) for details about expected behavior.

listObjectNames(String, String, String) - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

listObjectNames(String, String, String) - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

listStatus(Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Lists file status.

listStatus(Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

listStatusFileNotFoundBehavior - Variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

log - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

log - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemCacheCleaner

log - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

log - Static variable in class com.google.cloud.hadoop.gcsio.FileInfo

log - Static variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

log - Static variable in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

M

main(String[]) - Static method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemCacheCleaner

makeQualified(Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Overridden to make root it's own parent.

matchListPrefix(String, String, String) - Static method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageStrings

Helper which mimics the GCS API behavior for taking an existing objectName and checking if it matches a user-supplied prefix with an optional directory "delimiter".

MAX_LIST_ITEMS_PER_CALL_DEFAULT - Static variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions

Default number of items to return per call to the list* GCS RPCs.

MAX_REQUESTS_PER_BATCH_DEFAULT - Static variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions

Default setting for maximum number of requests per GCS batch.

maybeInvalidateExpiredInfo(CacheEntry) - Method in class com.google.cloud.hadoop.gcsio.DirectoryListCache

Helper for possibly clearing the GoogleCloudStorageItemInfo stored in entry based on cacheConfig settings.

metadataCacheEnabled - Variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystemOptions.Builder

MetadataReadOnlyGoogleCloudStorage - Class in com.google.cloud.hadoop.gcsio

MetadataReadOnlyGoogleCloudStorage holds a collection of Storage object/bucket metadata entries and serves listObjectNames, listObjectInfo, getItemInfos, and getItemInfo exclusively from the in-memory cache.

MetadataReadOnlyGoogleCloudStorage(Collection<GoogleCloudStorageItemInfo>) - Constructor for class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

Constructs a MetadataReadOnlyGoogleCloudStorage which can be used for temporary contexts where only object metadata read operations will be used through the GoogleCloudStorage interface.

mkdir(Path, FsPermission, boolean) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

mkdir(URI) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Creates a directory at the specified path.

mkdirs(Path, FsPermission) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Makes the given path and all non-existent parents directories.

mkdirs(URI) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Creates a directory at the specified path.

MR_JOB_HISTORY_DONE_DIR_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

MR_JOB_HISTORY_INTERMEDIATE_DONE_DIR_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

N

newBackOff() - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl.BackOffFactory

newBatchHelper(HttpRequestInitializer, Storage, long) - Method in class com.google.cloud.hadoop.gcsio.BatchHelper.Factory

newBuilder() - Static method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystemOptions

newBuilder() - Static method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions

NONE - Static variable in class com.google.cloud.hadoop.gcsio.ObjectWriteConditions

No conditions for completing the write.

O

objectOptionsFromFileOptions(CreateFileOptions) - Static method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Convert CreateFileOptions to CreateObjectOptions.

ObjectWriteConditions - Class in com.google.cloud.hadoop.gcsio

Conditions on which a object write should be allowed to continue.

ObjectWriteConditions() - Constructor for class com.google.cloud.hadoop.gcsio.ObjectWriteConditions

ObjectWriteConditions(Optional<Long>, Optional<Long>) - Constructor for class com.google.cloud.hadoop.gcsio.ObjectWriteConditions

open(Path, int) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Opens the given file for reading.

open(Path, int) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

open(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

Pure pass-through.

open(StorageResourceId) - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Opens an object for reading.

open(URI) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Opens an object for reading.

open(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

See open(StorageResourceId) for details about expected behavior.

open(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

open(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

openStreamAndSetSize(long) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageReadChannel

Opens the underlying stream, sets its position to the given value and sets size based on stream content size.

OperationWithRetry<T extends com.google.api.services.storage.StorageRequest<S>,S> - Class in com.google.cloud.hadoop.gcsio

Implements application level operation retries.

OperationWithRetry(Sleeper, BackOff, T, Predicate<IOException>) - Constructor for class com.google.cloud.hadoop.gcsio.OperationWithRetry

overwriteExisting() - Method in class com.google.cloud.hadoop.gcsio.CreateFileOptions

Get the value of overwriteExisting.

overwriteExisting() - Method in class com.google.cloud.hadoop.gcsio.CreateObjectOptions

Get the value of overwriteExisting.

P

PATH_DELIMITER - Static variable in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

PATH_DELIMITER - Static variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

position() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageReadChannel

Returns this channel's current position.

position(long) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageReadChannel

Sets this channel's position.

position() - Method in interface com.google.cloud.hadoop.gcsio.SeekableReadableByteChannel

Returns this channel's position.

position(long) - Method in interface com.google.cloud.hadoop.gcsio.SeekableReadableByteChannel

Sets this channel's position.

processDeleteOnExit() - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

PROPERTIES_FILE - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

put(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.CachedBucket

Adds a CacheEntry entry to this bucket corresponding to the StorageObject for resourceId.

putResourceId(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.DirectoryListCache

Adds the names of the Bucket or StorageObject referenced by resourceId to the cache, with no attached metadata.

putResourceId(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.FileSystemBackedDirectoryListCache

putResourceId(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.InMemoryDirectoryListCache

Q

queue(StorageRequest<T>, JsonBatchCallback<T>) - Method in class com.google.cloud.hadoop.gcsio.BatchHelper

Adds an additional request to the batch, and possibly flushes the current contents of the batch if maxRequestsPerBatch has been reached.

R

read(ByteBuffer) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageReadChannel

Reads from this channel and stores read data in the given buffer.

remove(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.CachedBucket

Removes the CacheEntry entry corresponding to resourceId which must be a StorageObject residing inside this CachedBucket, if it exists.

removeResourceId(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.DirectoryListCache

Removes CacheEntry associated with resourceId, if it exists.

removeResourceId(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.FileSystemBackedDirectoryListCache

removeResourceId(StorageResourceId) - Method in class com.google.cloud.hadoop.gcsio.InMemoryDirectoryListCache

rename(Path, Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Renames src to dst.

rename(URI, URI) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Renames the given item's path.

renameInternal(Path, Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

repairDirs(List<URI>) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Creates the list of directories specified in exactDirPaths; doesn't perform validation and doesn't create their parent dirs if their parent dirs don't already exist.

repairPossibleImplicitDirectory(URI) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Checks that path doesn't already exist as a directory object, and if so, performs an object listing using the full path as the match prefix so that if there are any objects which imply path is a parent directory, we will discover its existence as a returned GCS 'prefix'.

REPLICATION_FACTOR_DEFAULT - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

ROOT - Static variable in class com.google.cloud.hadoop.gcsio.StorageResourceId

ROOT_INFO - Static variable in class com.google.cloud.hadoop.gcsio.FileInfo

ROOT_INFO - Static variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageItemInfo

S

SCHEME - Static variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

SeekableReadableByteChannel - Interface in com.google.cloud.hadoop.gcsio

A readable byte channel that supports seek operation.

SERVICE_ACCOUNT_AUTH_EMAIL_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

SERVICE_ACCOUNT_AUTH_KEYFILE_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

setAppName(String) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions.Builder

setAutoRepairImplicitDirectoriesEnabled(boolean) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions.Builder

setBasePath(String) - Method in class com.google.cloud.hadoop.gcsio.FileSystemBackedDirectoryListCache

Only exposed for testing purposes; shouldn't be called outside of tests.

setCacheBasePath(String) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystemOptions.Builder

setCacheType(DirectoryListCache.Type) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystemOptions.Builder

setClock(Clock) - Static method in class com.google.cloud.hadoop.gcsio.CacheEntry

Sets a custom Clock to be used for computing cache creation and update times.

setClock(Clock) - Method in class com.google.cloud.hadoop.gcsio.DirectoryListCache

Sets the static Clock instance used for calculating expiration times.

setCreateMarkerObjects(boolean) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions.Builder

setIsMetadataCacheEnabled(boolean) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystemOptions.Builder

setItemInfo(GoogleCloudStorageItemInfo) - Method in class com.google.cloud.hadoop.gcsio.CacheEntry

Sets the GoogleCloudStorageItemInfo corresponding to this CacheEntry's StorageResourceId, and updates the itemInfoUpdateTimeMillis.

setListStatusFileNotFoundBehavior(GoogleHadoopFileSystemBase.ListStatusFileNotFoundBehavior) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

setMaxEntryAgeMillis(long) - Method in class com.google.cloud.hadoop.gcsio.DirectoryListCache.Config

Setter for maxEntryAgeMillis.

setMaxInfoAgeMillis(long) - Method in class com.google.cloud.hadoop.gcsio.DirectoryListCache.Config

Setter for maxInfoAgeMillis.

setMaxListItemsPerCall(long) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions.Builder

setMaxRequestsPerBatch(long) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions.Builder

setMaxRetries(int) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageReadChannel

Sets the number of times to automatically retry by re-opening the underlying readChannel whenever an exception occurs while reading from it.

setOwner(Path, String, String) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

setOwner(Path, String, String) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

setPermission(Path, FsPermission) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

setPermission(Path, FsPermission) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

setProjectId(String) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions.Builder

setReplication(Path, short) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

setShouldIncludeInTimestampUpdatesPredicate(Predicate<String>) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystemOptions.Builder

setSize(long) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageReadChannel

Sets size of this channel to the given value.

setTimes(Path, long, long) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

setTimes(Path, long, long) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

setVerifyChecksum(boolean) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

setVerifyChecksum(boolean) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS

setWorkingDirectory(Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Sets the current working directory to the given path.

setWriteChannelOptionsBuilder(AsyncWriteChannelOptions.Builder) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions.Builder

shouldIncludeInTimestampUpdatesPredicate - Variable in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystemOptions.Builder

size() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageReadChannel

Returns size of the object to which this channel is connected.

size() - Method in interface com.google.cloud.hadoop.gcsio.SeekableReadableByteChannel

Returns the current size of entity to which this channel is connected.

startLocalOutput(Path, Path) - Method in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

StorageResourceId - Class in com.google.cloud.hadoop.gcsio

Data struct representing either a GCS StorageObject, a GCS Bucket or the GCS root (gs://).

StorageResourceId(String) - Constructor for class com.google.cloud.hadoop.gcsio.StorageResourceId

Constructor for a StorageResourceId representing a Bucket; getObjectName() will return null for a StorageResourceId which represents a Bucket.

StorageResourceId(String, String) - Constructor for class com.google.cloud.hadoop.gcsio.StorageResourceId

Constructor for a StorageResourceId representing a full StorageObject, including bucketName and objectName.

supportsCacheEntryByReference() - Method in class com.google.cloud.hadoop.gcsio.DirectoryListCache

Implementations should indicate whether or not CacheEntry objects returned from methods are intended to be shared references which are authoritative cache state regarding metadata like cached GoogleCloudStorageItemInfo, etc.

supportsCacheEntryByReference() - Method in class com.google.cloud.hadoop.gcsio.FileSystemBackedDirectoryListCache

We use a real filesystem as our authoritative cache, and thus CacheEntries are not shared references.

supportsCacheEntryByReference() - Method in class com.google.cloud.hadoop.gcsio.InMemoryDirectoryListCache

We use in-memory data structures to hold CacheEntry items, and thus manage them in a shared manner; returned CacheEntry items are shared references, and updating their cached info effectively updates the entry's info for all users of the cache.

systemBucket - Variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

Deprecated.

T

ThrottledGoogleCloudStorage - Class in com.google.cloud.hadoop.gcsio

Throttled GCS implementation that will limit our bucket creation and delete operations to N per second.

ThrottledGoogleCloudStorage(GoogleCloudStorage, RateLimiter) - Constructor for class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

Construct a ThrottledGoogleCloudStorage object that throttles all operations.

ThrottledGoogleCloudStorage(double, GoogleCloudStorage, EnumSet<ThrottledGoogleCloudStorage.StorageOperation>) - Constructor for class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

ThrottledGoogleCloudStorage(RateLimiter, GoogleCloudStorage, EnumSet<ThrottledGoogleCloudStorage.StorageOperation>) - Constructor for class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

ThrottledGoogleCloudStorage.StorageOperation - Enum in com.google.cloud.hadoop.gcsio

Operations that may be throttled.

throwIfNotValid() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystemOptions

throwIfNotValid() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions

toString() - Method in class com.google.cloud.hadoop.gcsio.FileInfo

Gets string representation of this instance.

toString() - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageItemInfo

Gets string representation of this instance.

toString() - Method in class com.google.cloud.hadoop.gcsio.StorageResourceId

Returns a string of the form gs:///.

tryUpdateTimestampsForParentDirectories(List<URI>, List<URI>) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

For each listed modified object, attempt to update the modification time of the parent directory.

U

UNKNOWN_VERSION - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

UpdatableItemInfo - Class in com.google.cloud.hadoop.gcsio

Item information that can be updated without re-writing an entire object.

UpdatableItemInfo(StorageResourceId, Map<String, byte[]>) - Constructor for class com.google.cloud.hadoop.gcsio.UpdatableItemInfo

updateItems(List<UpdatableItemInfo>) - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

updateItems(List<UpdatableItemInfo>) - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Attempt to update metadata of the objects referenced within the passed itemInfo objects.

updateItems(List<UpdatableItemInfo>) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

updateItems(List<UpdatableItemInfo>) - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

updateItems(List<UpdatableItemInfo>) - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

updateTimestampsForParentDirectories(List<URI>, List<URI>) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

For each listed modified object, attempt to update the modification time of the parent directory.

V

validatePathAndGetId(URI, boolean) - Static method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem

Validates the given URI and if valid, returns the associated StorageResourceId.

validatePosition(long) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageReadChannel

Validates that the given position is valid for this channel.

validateResourceId(StorageResourceId) - Static method in class com.google.cloud.hadoop.gcsio.DirectoryListCache

Helper to validate resourceId which may be a Bucket or StorageObject.

valueOf(String) - Static method in enum com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.Counter

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.ListStatusFileNotFoundBehavior

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum com.google.cloud.hadoop.gcsio.DirectoryListCache.Type

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage.StorageOperation

Returns the enum constant of this type with the specified name.

values() - Static method in enum com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.Counter

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.ListStatusFileNotFoundBehavior

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum com.google.cloud.hadoop.gcsio.DirectoryListCache.Type

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage.StorageOperation

Returns an array containing the constants of this enum type, in the order they are declared.

VERSION - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

VERSION_PROPERTY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

W

waitForBucketEmpty(String) - Method in class com.google.cloud.hadoop.gcsio.CacheSupplementedGoogleCloudStorage

Pure pass-through.

waitForBucketEmpty(String) - Method in interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

Waits for the given bucket to be empty.

waitForBucketEmpty(String) - Method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageImpl

See waitForBucketEmpty(String) for details about expected behavior.

waitForBucketEmpty(String) - Method in class com.google.cloud.hadoop.gcsio.MetadataReadOnlyGoogleCloudStorage

waitForBucketEmpty(String) - Method in class com.google.cloud.hadoop.gcsio.ThrottledGoogleCloudStorage

wrapException(IOException, String, String, String) - Static method in class com.google.cloud.hadoop.gcsio.GoogleCloudStorageExceptions

Wraps the given IOException into another IOException, adding the given error message and a reference to the supplied bucket and object.

WRITE_BUFFERSIZE_DEFAULT - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

WRITE_BUFFERSIZE_KEY - Static variable in class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase

A B C D E F G H I L M N O P Q R S T U V W