com.google.cloud.hadoop.util

Class BaseAbstractGoogleAsyncWriteChannel<T>

java.lang.Object

com.google.cloud.hadoop.util.BaseAbstractGoogleAsyncWriteChannel<T>

Type Parameters:

T - The type of the result of the completed upload operation.

All Implemented Interfaces:

Closeable, AutoCloseable, Channel, WritableByteChannel

Direct Known Subclasses:

AbstractGoogleAsyncWriteChannel

public abstract class BaseAbstractGoogleAsyncWriteChannel<T>
extends Object
implements WritableByteChannel

Skeletal implementation of a WritableByteChannel that executes an asynchronous upload operation and optionally handles the result.

Field Summary

Fields

Modifier and Type	Field and Description
protected String	contentType
static int	GCS_UPLOAD_GRANULARITY
protected static com.google.common.flogger.GoogleLogger	logger
protected ExecutorService	threadPool
static int	UPLOAD_CHUNK_SIZE_DEFAULT
static int	UPLOAD_PIPE_BUFFER_SIZE_DEFAULT Deprecated.
protected int	uploadChunkSize
protected Future<T>	uploadOperation

Constructor Summary

Constructors

Constructor and Description
BaseAbstractGoogleAsyncWriteChannel(ExecutorService threadPool, AsyncWriteChannelOptions options) Construct a new channel using the given ExecutorService to run background uploads.

Method Summary

Modifier and Type	Method and Description
void	close() Closes this channel.
T	createResponseFromException(IOException ioe) Derived classes may optionally intercept an IOException thrown from the execute() method of a prepared request that came from #createRequest, and return a reconstituted "response" object if the IOException can be handled as a success; for example, if the caller already has an identifier for an object, and the response is used solely for obtaining the same identifier, and the IOException is a handled "409 Already Exists" type of exception, then the derived class may override this method to return the expected "identifier" response.
protected abstract String	getResourceString()
void	handleResponse(T response) Handle the API response.
void	initialize() Initialize this channel object for writing.
boolean	isDirectUploadEnabled() Returns true if direct media uploads are enabled.
boolean	isOpen() Tells whether this channel is open.
protected void	setContentType(String contentType) Sets the contentType.
void	setDirectUploadEnabled(boolean enableDirectUpload) Enables or disables direct uploads.
void	setUploadBufferSize(int bufferSize) Deprecated.
abstract void	setUploadChunkSize(int chunkSize) Sets size of upload buffer used.
abstract void	startUpload(PipedInputStream pipeSource) Create a new thread which handles the upload.
int	write(ByteBuffer buffer) Writes contents of the given buffer to this channel.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

logger

protected static final com.google.common.flogger.GoogleLogger logger

GCS_UPLOAD_GRANULARITY

public static final int GCS_UPLOAD_GRANULARITY

See Also:

Constant Field Values

UPLOAD_CHUNK_SIZE_DEFAULT

public static final int UPLOAD_CHUNK_SIZE_DEFAULT

UPLOAD_PIPE_BUFFER_SIZE_DEFAULT

@Deprecated
public static final int UPLOAD_PIPE_BUFFER_SIZE_DEFAULT

Deprecated.

See Also:

Constant Field Values

contentType

protected String contentType

threadPool

protected final ExecutorService threadPool

uploadChunkSize

protected int uploadChunkSize

uploadOperation

protected Future<T> uploadOperation

Constructor Detail

BaseAbstractGoogleAsyncWriteChannel

public BaseAbstractGoogleAsyncWriteChannel(ExecutorService threadPool,
                                           AsyncWriteChannelOptions options)

Construct a new channel using the given ExecutorService to run background uploads.

Method Detail

startUpload

public abstract void startUpload(PipedInputStream pipeSource)
                          throws IOException

Create a new thread which handles the upload.

Throws:

IOException

handleResponse

public void handleResponse(T response)
                    throws IOException

Handle the API response.

This method is invoked after the upload has completed on the same thread that invokes close().

Parameters:

response - The API response object.

Throws:

IOException

createResponseFromException

public T createResponseFromException(IOException ioe)

Derived classes may optionally intercept an IOException thrown from the execute() method of a prepared request that came from #createRequest, and return a reconstituted "response" object if the IOException can be handled as a success; for example, if the caller already has an identifier for an object, and the response is used solely for obtaining the same identifier, and the IOException is a handled "409 Already Exists" type of exception, then the derived class may override this method to return the expected "identifier" response. Return null to let the exception propagate through correctly.

setUploadBufferSize

@Deprecated
public void setUploadBufferSize(int bufferSize)

Deprecated.

setUploadChunkSize

public abstract void setUploadChunkSize(int chunkSize)

Sets size of upload buffer used.

setDirectUploadEnabled

public void setDirectUploadEnabled(boolean enableDirectUpload)

Enables or disables direct uploads.

See Also:

MediaHttpUploader.setDirectUploadEnabled(boolean)

isDirectUploadEnabled

public boolean isDirectUploadEnabled()

Returns true if direct media uploads are enabled.

write

public int write(ByteBuffer buffer)
          throws IOException

Writes contents of the given buffer to this channel.

Note: The data that one writes gets written to a pipe which must not block if the pipe has sufficient buffer space. A success code returned from this method does not mean that the specific data was successfully written to the underlying storage. It simply means that there is no error at present. The data upload may encounter an error on a separate thread. Such error is not ignored; it shows up as an exception during a subsequent call to write() or close(). The only way to be sure of successful upload is when the close() method returns successfully.

Specified by:

write in interface WritableByteChannel

Parameters:

buffer - buffer to write

Throws:

IOException - on IO error

isOpen

public boolean isOpen()

Tells whether this channel is open.

Specified by:

isOpen in interface Channel

Returns:

a value indicating whether this channel is open

close

public void close()
           throws IOException

Closes this channel.

Note: The method returns only after all data has been successfully written to GCS or if there is a non-retry-able error.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Channel

Throws:

IOException - on IO error

initialize

public void initialize()
                throws IOException

Initialize this channel object for writing.

Throws:

IOException

setContentType

protected void setContentType(String contentType)

Sets the contentType. This must be called before initialize() for any effect.

getResourceString

protected abstract String getResourceString()