public abstract class BlobStoreRepository extends AbstractLifecycleComponent<Repository> implements Repository, BlobStoreIndexShardRepository.RateLimiterListener
This repository works with any BlobStore
implementation. The blobStore should be initialized in the derived
class before doStart()
is called.
BlobStoreRepository maintains the following structure in the blob store
STORE_ROOT
|- index - list of all snapshot name as JSON array
|- snapshot-20131010 - JSON serialized Snapshot for snapshot "20131010"
|- meta-20131010.dat - JSON serialized MetaData for snapshot "20131010" (includes only global metadata)
|- snapshot-20131011 - JSON serialized Snapshot for snapshot "20131011"
|- meta-20131011.dat - JSON serialized MetaData for snapshot "20131011"
.....
|- indices/ - data for all indices
|- foo/ - data for index "foo"
| |- meta-20131010.dat - JSON Serialized IndexMetaData for index "foo"
| |- 0/ - data for shard "0" of index "foo"
| | |- __1 \
| | |- __2 |
| | |- __3 |- files from different segments see snapshot-* for their mappings to real segment files
| | |- __4 |
| | |- __5 /
| | .....
| | |- snap-20131010.dat - JSON serialized BlobStoreIndexShardSnapshot for snapshot "20131010"
| | |- snap-20131011.dat - JSON serialized BlobStoreIndexShardSnapshot for snapshot "20131011"
| | |- list-123 - JSON serialized BlobStoreIndexShardSnapshot for snapshot "20131011"
| |
| |- 1/ - data for shard "1" of index "foo"
| | |- __1
| | .....
| |
| |-2/
| ......
|
|- bar/ - data for index bar
......
Modifier and Type | Field and Description |
---|---|
protected String |
repositoryName |
lifecycle
deprecationLogger, logger, settings
Modifier | Constructor and Description |
---|---|
protected |
BlobStoreRepository(String repositoryName,
RepositorySettings repositorySettings,
IndexShardRepository indexShardRepository)
Constructs new BlobStoreRepository
|
Modifier and Type | Method and Description |
---|---|
protected abstract BlobPath |
basePath()
Returns base path of the repository
|
protected abstract BlobStore |
blobStore()
Returns initialized and ready to use BlobStore
|
protected ByteSizeValue |
chunkSize()
Returns data file chunk size.
|
void |
deleteSnapshot(SnapshotId snapshotId)
Deletes snapshot
|
protected void |
doClose() |
protected void |
doStart() |
protected void |
doStop() |
void |
endVerification(String seed)
Called at the end of repository verification process.
|
Snapshot |
finalizeSnapshot(SnapshotId snapshotId,
List<String> indices,
long startTime,
String failure,
int totalShards,
List<SnapshotShardFailure> shardFailures)
Finalizes snapshotting process
|
void |
initializeSnapshot(SnapshotId snapshotId,
List<String> indices,
MetaData metaData)
Starts snapshotting process
|
protected boolean |
isCompress()
Returns true if metadata and snapshot files should be compressed
|
static boolean |
legacyMetaData(Version version)
In v2.0.0 we changed the matadata file format
|
void |
onRestorePause(long nanos) |
void |
onSnapshotPause(long nanos) |
boolean |
readOnly()
Returns true if the repository supports only read operations
|
Snapshot |
readSnapshot(SnapshotId snapshotId)
Reads snapshot description from repository.
|
protected List<SnapshotId> |
readSnapshotList()
Reads snapshot index file
|
MetaData |
readSnapshotMetaData(SnapshotId snapshotId,
Snapshot snapshot,
List<String> indices)
Returns global metadata associate with the snapshot.
|
long |
restoreThrottleTimeInNanos()
Returns restore throttle time in nanoseconds
|
List<SnapshotId> |
snapshots()
Returns the list of snapshots currently stored in the repository
|
long |
snapshotThrottleTimeInNanos()
Returns snapshot throttle time in nanoseconds
|
String |
startVerification()
Verifies repository on the master node and returns the verification token.
|
static String |
testBlobPrefix(String seed) |
protected void |
writeSnapshotList(List<SnapshotId> snapshots)
Writes snapshot index file
|
addLifecycleListener, close, lifecycleState, removeLifecycleListener, start, stop
logDeprecatedSetting, logRemovedSetting, nodeName
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
addLifecycleListener, lifecycleState, removeLifecycleListener, start, stop
close
protected final String repositoryName
protected BlobStoreRepository(String repositoryName, RepositorySettings repositorySettings, IndexShardRepository indexShardRepository)
repositoryName
- repository namerepositorySettings
- repository settingsindexShardRepository
- an instance of IndexShardRepositoryprotected void doStart()
doStart
in class AbstractLifecycleComponent<Repository>
protected void doStop()
doStop
in class AbstractLifecycleComponent<Repository>
protected void doClose()
doClose
in class AbstractLifecycleComponent<Repository>
protected abstract BlobStore blobStore()
This method is first called in the doStart()
method.
protected abstract BlobPath basePath()
protected boolean isCompress()
protected ByteSizeValue chunkSize()
This method should return null if no chunking is needed.
public void initializeSnapshot(SnapshotId snapshotId, List<String> indices, MetaData metaData)
initializeSnapshot
in interface Repository
snapshotId
- snapshot idindices
- list of indices to be snapshottedmetaData
- cluster metadatapublic void deleteSnapshot(SnapshotId snapshotId)
deleteSnapshot
in interface Repository
snapshotId
- snapshot idpublic Snapshot finalizeSnapshot(SnapshotId snapshotId, List<String> indices, long startTime, String failure, int totalShards, List<SnapshotShardFailure> shardFailures)
This method is called on master after all shards are snapshotted.
finalizeSnapshot
in interface Repository
snapshotId
- snapshot idfailure
- global failure reason or nulltotalShards
- total number of shardsshardFailures
- list of shard failurespublic List<SnapshotId> snapshots()
snapshots
in interface Repository
public MetaData readSnapshotMetaData(SnapshotId snapshotId, Snapshot snapshot, List<String> indices) throws IOException
The returned meta data contains global metadata as well as metadata for all indices listed in the indices parameter.
readSnapshotMetaData
in interface Repository
snapshot
- snapshotindices
- list of indicesIOException
public Snapshot readSnapshot(SnapshotId snapshotId)
readSnapshot
in interface Repository
snapshotId
- snapshot IDpublic static boolean legacyMetaData(Version version)
protected void writeSnapshotList(List<SnapshotId> snapshots) throws IOException
This file can be used by read-only repositories that are unable to list files in the repository
snapshots
- list of snapshot idsIOException
- I/O errorsprotected List<SnapshotId> readSnapshotList() throws IOException
This file can be used by read-only repositories that are unable to list files in the repository
IOException
- I/O errorspublic void onRestorePause(long nanos)
onRestorePause
in interface BlobStoreIndexShardRepository.RateLimiterListener
public void onSnapshotPause(long nanos)
onSnapshotPause
in interface BlobStoreIndexShardRepository.RateLimiterListener
public long snapshotThrottleTimeInNanos()
Repository
snapshotThrottleTimeInNanos
in interface Repository
public long restoreThrottleTimeInNanos()
Repository
restoreThrottleTimeInNanos
in interface Repository
public String startVerification()
Repository
If the verification token is not null, it's passed to all data nodes for verification. If it's null - no additional verification is required
startVerification
in interface Repository
public void endVerification(String seed)
Repository
This method should perform all necessary cleanup of the temporary files created in the repository
endVerification
in interface Repository
seed
- verification request generated by Repository.startVerification()
commandpublic boolean readOnly()
Repository
readOnly
in interface Repository
Copyright © 2009–2016. All rights reserved.