com.microsoft.azure.storage.blob

Class TransferManager

java.lang.Object

com.microsoft.azure.storage.blob.TransferManager

public final class TransferManager
extends Object

This class contains a collection of methods (and structures associated with those methods) which perform higher-level operations. Whereas operations on the URL types guarantee a single REST request and make no assumptions on desired behavior, these methods will often compose several requests to provide a convenient way of performing more complex operations. Further, we will make our own assumptions and optimizations for common cases that may not be ideal for rarer cases.

Field Summary

Fields

Modifier and Type	Field and Description
static int	BLOB_DEFAULT_DOWNLOAD_BLOCK_SIZE The default size of a download chunk for download large blobs.

Constructor Summary

Constructors

Constructor and Description
TransferManager()

Method Summary

Modifier and Type	Method and Description
static io.reactivex.Single<BlobDownloadHeaders>	downloadBlobToFile(AsynchronousFileChannel file, BlobURL blobURL, BlobRange range, TransferManagerDownloadFromBlobOptions options) Downloads a file directly into a file, splitting the download into chunks and parallelizing as necessary.
static io.reactivex.Single<CommonRestResponse>	uploadFileToBlockBlob(AsynchronousFileChannel file, BlockBlobURL blockBlobURL, int blockLength, Integer maxSingleShotSize, TransferManagerUploadToBlockBlobOptions options) Uploads the contents of a file to a block blob in parallel, breaking it into block-size chunks if necessary.
static io.reactivex.Single<BlockBlobCommitBlockListResponse>	uploadFromNonReplayableFlowable(io.reactivex.Flowable<ByteBuffer> source, BlockBlobURL blockBlobURL, int blockSize, int numBuffers, TransferManagerUploadToBlockBlobOptions options) Uploads the contents of an arbitrary Flowable to a block blob.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

BLOB_DEFAULT_DOWNLOAD_BLOCK_SIZE

public static final int BLOB_DEFAULT_DOWNLOAD_BLOCK_SIZE

The default size of a download chunk for download large blobs.

See Also:

Constant Field Values

Constructor Detail

TransferManager

public TransferManager()

Method Detail

uploadFileToBlockBlob

public static io.reactivex.Single<CommonRestResponse> uploadFileToBlockBlob(AsynchronousFileChannel file,
                                                                            BlockBlobURL blockBlobURL,
                                                                            int blockLength,
                                                                            Integer maxSingleShotSize,
                                                                            TransferManagerUploadToBlockBlobOptions options)
                                                                     throws IOException

Uploads the contents of a file to a block blob in parallel, breaking it into block-size chunks if necessary.

Parameters:

file - The file to upload.

blockBlobURL - Points to the blob to which the data should be uploaded.

blockLength - If the data must be broken up into blocks, this value determines what size those blocks will be. This will affect the total number of service requests made as each REST request uploads exactly one block in full. This value will be ignored if the data can be uploaded in a single put-blob operation. Must be between 1 and BlockBlobURL.MAX_STAGE_BLOCK_BYTES. Note as well that fileLength/blockLength must be less than or equal to BlockBlobURL.MAX_BLOCKS.

maxSingleShotSize - If the size of the data is less than or equal to this value, it will be uploaded in a single put rather than broken up into chunks. If the data is uploaded in a single shot, the block size will be ignored. Some constraints to consider are that more requests cost more, but several small or mid-sized requests may sometimes perform better. Must be greater than 0. May be null to accept default behavior.

options - TransferManagerUploadToBlockBlobOptions

Returns:

Emits the successful response.

Throws:

IOException

downloadBlobToFile

public static io.reactivex.Single<BlobDownloadHeaders> downloadBlobToFile(AsynchronousFileChannel file,
                                                                          BlobURL blobURL,
                                                                          BlobRange range,
                                                                          TransferManagerDownloadFromBlobOptions options)

Downloads a file directly into a file, splitting the download into chunks and parallelizing as necessary.

Parameters:

file - The destination file to which the blob will be written.

blobURL - The URL to the blob to download.

range - BlobRange

options - TransferManagerDownloadFromBlobOptions

Returns:

A Completable that will signal when the download is complete.

uploadFromNonReplayableFlowable

public static io.reactivex.Single<BlockBlobCommitBlockListResponse> uploadFromNonReplayableFlowable(io.reactivex.Flowable<ByteBuffer> source,
                                                                                                    BlockBlobURL blockBlobURL,
                                                                                                    int blockSize,
                                                                                                    int numBuffers,
                                                                                                    TransferManagerUploadToBlockBlobOptions options)

Uploads the contents of an arbitrary Flowable to a block blob. This Flowable need not be replayable and therefore it may have as its source a network stream or any other data for which the replay behavior is unknown (non-replayable meaning the Flowable may not return the exact same data on each subscription). To eliminate the need for replayability on the source, the client must perform some buffering in order to ensure the actual data passed to the network is replayable. This is important in order to support retries, which are crucial for reliable data transfer. Typically, the greater the number of buffers used, the greater the possible parallelism. Larger buffers means we will have to stage fewer blocks. The tradeoffs between these values are context-dependent, so some experimentation may be required to optimize inputs for a given scenario. Note that buffering must be strictly sequential. Only the upload portion of this operation may be parallelized; the reads cannot be. Therefore, this method is not as optimal as uploadFileToBlockBlob(AsynchronousFileChannel, BlockBlobURL, int, Integer, TransferManagerUploadToBlockBlobOptions) and if the source is known to be a file, that method should be preferred.

Parameters:

source - Contains the data to upload. Unlike other upload methods in this library, this method does not require that the Flowable be replayable.

blockBlobURL - Points to the blob to which the data should be uploaded.

blockSize - The size of each block that will be staged. This value also determines the size that each buffer used by this method will be and determines the number of requests that need to be made. The amount of memory consumed by this method may be up to blockSize * numBuffers. If block size is large, this method will make fewer network calls, but each individual call will send more data and will therefore take longer.

numBuffers - The maximum number of buffers this method should allocate. Must be at least two. Generally this value should have some relationship to the value for parallelism passed via the options. If the number of available buffers is smaller than the level of parallelism, then this method will not be able to make full use of the available parallelism. It is unlikely that the value need be more than two times the level of parallelism as such a value means that (assuming buffering is fast enough) there are enough available buffers to have both one occupied for each worker and one ready for all workers should they all complete the current request at approximately the same time. The amount of memory consumed by this method may be up to blockSize * numBuffers.

options - TransferManagerUploadToBlockBlobOptions

Returns:

Emits the successful response.