org.elasticsearch.index.shard

Class IndexShard

java.lang.Object

org.elasticsearch.index.shard.AbstractIndexShardComponent

org.elasticsearch.index.shard.IndexShard

All Implemented Interfaces:

IndexShardComponent

Direct Known Subclasses:

ShadowIndexShard

public class IndexShard
extends AbstractIndexShardComponent

Field Summary

Fields

Modifier and Type	Field and Description
protected AtomicReference<Engine>	currentEngineReference
protected EngineFactory	engineFactory
static String	INDEX_FLUSH_ON_CLOSE Index setting to control if a flush is executed before engine is closed This setting is realtime updateable.
static String	INDEX_REFRESH_INTERVAL
protected ShardRouting	shardRouting
protected IndexShardState	state

Fields inherited from class org.elasticsearch.index.shard.AbstractIndexShardComponent

deprecationLogger, indexSettings, logger, shardId

Constructor Summary

Constructors

Constructor and Description

IndexShard(ShardId shardId, IndexSettingsService indexSettingsService, IndicesLifecycle indicesLifecycle, Store store, StoreRecoveryService storeRecoveryService, ThreadPool threadPool, MapperService mapperService, IndexQueryParserService queryParserService, IndexCache indexCache, IndexAliasesService indexAliasesService, IndicesQueryCache indicesQueryCache, ShardPercolateService shardPercolateService, CodecService codecService, ShardTermVectorsService termVectorsService, IndexFieldDataService indexFieldDataService, IndexService indexService, IndicesWarmer warmer, SnapshotDeletionPolicy deletionPolicy, SimilarityService similarityService, EngineFactory factory, ClusterService clusterService, ShardPath path, BigArrays bigArrays, IndexSearcherWrappingService wrappingService, IndexingMemoryController indexingMemoryController)

Method Summary

Methods

Modifier and Type	Method and Description
Engine.Searcher	acquireSearcher(String source)
void	addFailedEngineListener(Engine.FailedEngineListener failedEngineListener)
boolean	allowsPrimaryPromotion() Returns true iff this shard allows primary promotion, otherwise false
boolean	canIndex() returns true if this shard supports indexing (i.e., write) operations.
boolean	checkIdle(long inactiveTimeNS) Called by IndexingMemoryController to check whether more than inactiveTimeNS has passed since the last indexing operation, and become inactive (reducing indexing and translog buffers to tiny values) if so.
void	close(String reason, boolean flushEngine)
CommitStats	commitStats()
CompletionStats	completionStats(String... fields)
void	create(Engine.Create create)
void	decrementOperationCounter()
void	delete(Engine.Delete delete)
void	deleteShardState() Deletes the shards metadata state.
DocsStats	docStats()
Engine	engine()
protected Engine	engineUnsafe() NOTE: returns null if engine is not yet started (e.g.
void	failShard(String reason, Throwable e) Fails the shard and marks the shard store as corrupted if e is caused by index corruption
ShardFieldData	fieldData()
FieldDataStats	fieldDataStats(String... fields)
void	finalizeRecovery() perform the last stages of recovery once all translog operations are done.
Engine.CommitId	flush(FlushRequest request)
FlushStats	flushStats()
void	forceMerge(ForceMergeRequest forceMerge)
Engine.GetResult	get(Engine.Get get)
boolean	getActive() Returns true if this shard is active (has seen indexing ops in the last IndexingMemoryController.SHARD_INACTIVE_TIME_SETTING (default 5 minutes), else false.
ByteSizeValue	getIndexingBufferSize()
int	getOperationsCount()
org.apache.lucene.search.QueryCachingPolicy	getQueryCachingPolicy()
ShardGetService	getService()
GetStats	getStats()
ShardSuggestMetric	getSuggestMetric()
Translog.Durabilty	getTranslogDurability() Returns the current translog durability mode
boolean	ignoreRecoveryAttempt() Returns true if this shard can ignore a recovery attempt made to it (since the already doing/done it)
void	incrementOperationCounter()
boolean	index(Engine.Index index) Index a document and return whether it was created, as opposed to just updated.
IndexFieldDataService	indexFieldDataService()
ShardIndexingService	indexingService()
IndexingStats	indexingStats(String... types)
IndexService	indexService()
boolean	isFlushOnClose()
MapperService	mapperService()
MergeStats	mergeStats()
org.apache.lucene.util.Version	minimumCompatibleVersion()
protected Engine	newEngine(boolean skipTranslogRecovery, EngineConfig config)
PercolatorQueriesRegistry	percolateRegistry()
int	performBatchRecovery(Iterable<Translog.Operation> operations) Applies all operations in the iterable to the current engine and returns the number of operations applied.
void	performRecoveryRestart() called if recovery has to be restarted after network error / delay **
Map<String,Mapping>	performTranslogRecovery(boolean indexExists) After the store has been recovered, we need to start the engine in order to apply operations
IndexShard	postRecovery(String reason)
Engine.Create	prepareCreateOnPrimary(SourceToParse source, long version, VersionType versionType, boolean canHaveDuplicates, boolean autoGeneratedId)
Engine.Create	prepareCreateOnReplica(SourceToParse source, long version, VersionType versionType, boolean canHaveDuplicates, boolean autoGeneratedId)
Engine.Delete	prepareDeleteOnPrimary(String type, String id, long version, VersionType versionType)
Engine.Delete	prepareDeleteOnReplica(String type, String id, long version, VersionType versionType)
void	prepareForIndexRecovery() called before starting to copy index files over
Engine.Index	prepareIndexOnPrimary(SourceToParse source, long version, VersionType versionType, boolean canHaveDuplicates)
Engine.Index	prepareIndexOnReplica(SourceToParse source, long version, VersionType versionType, boolean canHaveDuplicates)
QueryCacheStats	queryCacheStats()
void	readAllowed()
void	recoverFromStore(ShardRouting shard, StoreRecoveryService.RecoveryListener recoveryListener)
IndexShardState	recovering(String reason, RecoveryState.Type type, DiscoveryNode sourceNode) Marks the shard as recovering based on a remote or local node, fails with exception is recovering is not allowed to be set.
IndexShardState	recovering(String reason, RecoveryState.Type type, RestoreSource restoreSource) Marks the shard as recovering based on a restore, fails with exception is recovering is not allowed to be set.
RecoveryState	recoveryState() Returns the current RecoveryState if this shard is recovering or has been recovering.
RecoveryStats	recoveryStats() returns stats about ongoing recoveries, both source and target
void	refresh(String source)
RefreshStats	refreshStats()
IndexShard	relocated(String reason)
ShardRequestCache	requestCache()
ShardRouting	routingEntry() Returns the latest cluster routing entry received with this shard.
ShardSearchStats	searchService()
SearchStats	searchStats(String... groups)
SegmentsStats	segmentStats()
ShardBitsetFilterCache	shardBitsetFilterCache()
ShardPath	shardPath()
ShardPercolateService	shardPercolateService()
void	skipTranslogRecovery() After the store has been recovered, we need to start the engine.
SnapshotIndexCommit	snapshotIndex(boolean flushFirst)
IndexShardState	state()
Store	store()
StoreStats	storeStats()
SuggestStats	suggestStats()
void	sync(Translog.Location location) Syncs the given location with the underlying storage unless already synced.
Engine.SyncedFlushResult	syncFlush(String syncId, Engine.CommitId expectedCommitId)
ShardTermVectorsService	termVectorsService()
TranslogStats	translogStats()
void	updateBufferSize(ByteSizeValue shardIndexingBufferSize, ByteSizeValue shardTranslogBufferSize) Change the indexing and translog buffer sizes.
void	updateRoutingEntry(ShardRouting newRouting, boolean persistState) Updates the shards routing entry.
org.apache.lucene.util.Version	upgrade(UpgradeRequest upgrade) Upgrades the shard to the current version of Lucene and returns the minimum segment version
protected void	verifyStarted()
protected void	verifyStartedOrRecovering()
ShardIndexWarmerService	warmerService()
WarmerStats	warmerStats()

Methods inherited from class org.elasticsearch.index.shard.AbstractIndexShardComponent

indexSettings, nodeName, shardId

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

shardRouting

protected volatile ShardRouting shardRouting

state

protected volatile IndexShardState state

currentEngineReference

protected final AtomicReference<Engine> currentEngineReference

engineFactory

protected final EngineFactory engineFactory

INDEX_FLUSH_ON_CLOSE

public static final String INDEX_FLUSH_ON_CLOSE

Index setting to control if a flush is executed before engine is closed This setting is realtime updateable.

See Also:

Constant Field Values

INDEX_REFRESH_INTERVAL

public static final String INDEX_REFRESH_INTERVAL

See Also:

Constant Field Values

Constructor Detail

IndexShard

@Inject
public IndexShard(ShardId shardId,
                 IndexSettingsService indexSettingsService,
                 IndicesLifecycle indicesLifecycle,
                 Store store,
                 StoreRecoveryService storeRecoveryService,
                 ThreadPool threadPool,
                 MapperService mapperService,
                 IndexQueryParserService queryParserService,
                 IndexCache indexCache,
                 IndexAliasesService indexAliasesService,
                 IndicesQueryCache indicesQueryCache,
                 ShardPercolateService shardPercolateService,
                 CodecService codecService,
                 ShardTermVectorsService termVectorsService,
                 IndexFieldDataService indexFieldDataService,
                 IndexService indexService,
                 @Nullable
                 IndicesWarmer warmer,
                 SnapshotDeletionPolicy deletionPolicy,
                 SimilarityService similarityService,
                 EngineFactory factory,
                 ClusterService clusterService,
                 ShardPath path,
                 BigArrays bigArrays,
                 IndexSearcherWrappingService wrappingService,
                 IndexingMemoryController indexingMemoryController)

Method Detail

store

public Store store()

canIndex

public boolean canIndex()

returns true if this shard supports indexing (i.e., write) operations.

indexingService

public ShardIndexingService indexingService()

getService

public ShardGetService getService()

termVectorsService

public ShardTermVectorsService termVectorsService()

getSuggestMetric

public ShardSuggestMetric getSuggestMetric()

shardBitsetFilterCache

public ShardBitsetFilterCache shardBitsetFilterCache()

indexFieldDataService

public IndexFieldDataService indexFieldDataService()

mapperService

public MapperService mapperService()

indexService

public IndexService indexService()

searchService

public ShardSearchStats searchService()

warmerService

public ShardIndexWarmerService warmerService()

requestCache

public ShardRequestCache requestCache()

fieldData

public ShardFieldData fieldData()

routingEntry

public ShardRouting routingEntry()

Returns the latest cluster routing entry received with this shard. Might be null if the shard was just created.

getQueryCachingPolicy

public org.apache.lucene.search.QueryCachingPolicy getQueryCachingPolicy()

updateRoutingEntry

public void updateRoutingEntry(ShardRouting newRouting,
                      boolean persistState)

Updates the shards routing entry. This mutate the shards internal state depending on the changes that get introduced by the new routing value. This method will persist shard level metadata unless explicitly disabled.

recovering

public IndexShardState recovering(String reason,
                         RecoveryState.Type type,
                         DiscoveryNode sourceNode)
                           throws IndexShardStartedException,
                                  IndexShardRelocatedException,
                                  IndexShardRecoveringException,
                                  IndexShardClosedException

Marks the shard as recovering based on a remote or local node, fails with exception is recovering is not allowed to be set.

Throws:

IndexShardStartedException

IndexShardRelocatedException

IndexShardRecoveringException

IndexShardClosedException

recovering

public IndexShardState recovering(String reason,
                         RecoveryState.Type type,
                         RestoreSource restoreSource)
                           throws IndexShardStartedException

Marks the shard as recovering based on a restore, fails with exception is recovering is not allowed to be set.

Throws:

IndexShardStartedException

relocated

public IndexShard relocated(String reason)
                     throws IndexShardNotStartedException

Throws:

IndexShardNotStartedException

state

public IndexShardState state()

prepareCreateOnPrimary

public Engine.Create prepareCreateOnPrimary(SourceToParse source,
                                   long version,
                                   VersionType versionType,
                                   boolean canHaveDuplicates,
                                   boolean autoGeneratedId)

prepareCreateOnReplica

public Engine.Create prepareCreateOnReplica(SourceToParse source,
                                   long version,
                                   VersionType versionType,
                                   boolean canHaveDuplicates,
                                   boolean autoGeneratedId)

create

public void create(Engine.Create create)

prepareIndexOnPrimary

public Engine.Index prepareIndexOnPrimary(SourceToParse source,
                                 long version,
                                 VersionType versionType,
                                 boolean canHaveDuplicates)

prepareIndexOnReplica

public Engine.Index prepareIndexOnReplica(SourceToParse source,
                                 long version,
                                 VersionType versionType,
                                 boolean canHaveDuplicates)

index

public boolean index(Engine.Index index)

Index a document and return whether it was created, as opposed to just updated.

prepareDeleteOnPrimary

public Engine.Delete prepareDeleteOnPrimary(String type,
                                   String id,
                                   long version,
                                   VersionType versionType)

prepareDeleteOnReplica

public Engine.Delete prepareDeleteOnReplica(String type,
                                   String id,
                                   long version,
                                   VersionType versionType)

delete

public void delete(Engine.Delete delete)

get

public Engine.GetResult get(Engine.Get get)

refresh

public void refresh(String source)

refreshStats

public RefreshStats refreshStats()

flushStats

public FlushStats flushStats()

docStats

public DocsStats docStats()

commitStats

@Nullable
public CommitStats commitStats()

Returns:

CommitStats if engine is open, otherwise null

indexingStats

public IndexingStats indexingStats(String... types)

searchStats

public SearchStats searchStats(String... groups)

getStats

public GetStats getStats()

storeStats

public StoreStats storeStats()

mergeStats

public MergeStats mergeStats()

segmentStats

public SegmentsStats segmentStats()

warmerStats

public WarmerStats warmerStats()

queryCacheStats

public QueryCacheStats queryCacheStats()

fieldDataStats

public FieldDataStats fieldDataStats(String... fields)

percolateRegistry

public PercolatorQueriesRegistry percolateRegistry()

shardPercolateService

public ShardPercolateService shardPercolateService()

translogStats

public TranslogStats translogStats()

suggestStats

public SuggestStats suggestStats()

completionStats

public CompletionStats completionStats(String... fields)

syncFlush

public Engine.SyncedFlushResult syncFlush(String syncId,
                                 Engine.CommitId expectedCommitId)

flush

public Engine.CommitId flush(FlushRequest request)
                      throws ElasticsearchException

Throws:

ElasticsearchException

forceMerge

public void forceMerge(ForceMergeRequest forceMerge)
                throws IOException

Throws:

IOException

upgrade

public org.apache.lucene.util.Version upgrade(UpgradeRequest upgrade)
                                       throws IOException

Upgrades the shard to the current version of Lucene and returns the minimum segment version

Throws:

IOException

minimumCompatibleVersion

public org.apache.lucene.util.Version minimumCompatibleVersion()

snapshotIndex

public SnapshotIndexCommit snapshotIndex(boolean flushFirst)
                                  throws EngineException

Throws:

EngineException

failShard

public void failShard(String reason,
             @Nullable
             Throwable e)

Fails the shard and marks the shard store as corrupted if e is caused by index corruption

acquireSearcher

public Engine.Searcher acquireSearcher(String source)

close

public void close(String reason,
         boolean flushEngine)
           throws IOException

Throws:

IOException

postRecovery

public IndexShard postRecovery(String reason)
                        throws IndexShardStartedException,
                               IndexShardRelocatedException,
                               IndexShardClosedException

Throws:

IndexShardStartedException

IndexShardRelocatedException

IndexShardClosedException

prepareForIndexRecovery

public void prepareForIndexRecovery()

called before starting to copy index files over

performBatchRecovery

public int performBatchRecovery(Iterable<Translog.Operation> operations)

Applies all operations in the iterable to the current engine and returns the number of operations applied. This operation will stop applying operations once an operation failed to apply. Note: This method is typically used in peer recovery to replay remote transaction log entries.

getIndexingBufferSize

public ByteSizeValue getIndexingBufferSize()

performTranslogRecovery

public Map<String,Mapping> performTranslogRecovery(boolean indexExists)

After the store has been recovered, we need to start the engine in order to apply operations

skipTranslogRecovery

public void skipTranslogRecovery()
                          throws IOException

After the store has been recovered, we need to start the engine. This method starts a new engine but skips the replay of the transaction log which is required in cases where we restore a previous index or recover from a remote peer.

Throws:

IOException

performRecoveryRestart

public void performRecoveryRestart()
                            throws IOException

called if recovery has to be restarted after network error / delay **

Throws:

IOException

recoveryStats

public RecoveryStats recoveryStats()

returns stats about ongoing recoveries, both source and target

recoveryState

public RecoveryState recoveryState()

Returns the current RecoveryState if this shard is recovering or has been recovering. Returns null if the recovery has not yet started or shard was not recovered (created via an API).

finalizeRecovery

public void finalizeRecovery()

perform the last stages of recovery once all translog operations are done. note that you should still call postRecovery(String).

ignoreRecoveryAttempt

public boolean ignoreRecoveryAttempt()

Returns true if this shard can ignore a recovery attempt made to it (since the already doing/done it)

readAllowed

public void readAllowed()
                 throws IllegalIndexShardStateException

Throws:

IllegalIndexShardStateException

verifyStartedOrRecovering

protected final void verifyStartedOrRecovering()
                                        throws IllegalIndexShardStateException

Throws:

IllegalIndexShardStateException

verifyStarted

protected final void verifyStarted()
                            throws IllegalIndexShardStateException

Throws:

IllegalIndexShardStateException

addFailedEngineListener

public void addFailedEngineListener(Engine.FailedEngineListener failedEngineListener)

updateBufferSize

public void updateBufferSize(ByteSizeValue shardIndexingBufferSize,
                    ByteSizeValue shardTranslogBufferSize)

Change the indexing and translog buffer sizes. If IndexWriter is currently using more than the new buffering indexing size then we do a refresh to free up the heap.

checkIdle

public boolean checkIdle(long inactiveTimeNS)

Called by IndexingMemoryController to check whether more than inactiveTimeNS has passed since the last indexing operation, and become inactive (reducing indexing and translog buffers to tiny values) if so. This returns true if the shard is inactive.

getActive

public boolean getActive()

Returns true if this shard is active (has seen indexing ops in the last IndexingMemoryController.SHARD_INACTIVE_TIME_SETTING (default 5 minutes), else false.

isFlushOnClose

public final boolean isFlushOnClose()

deleteShardState

public void deleteShardState()
                      throws IOException

Deletes the shards metadata state. This method can only be executed if the shard is not active.

Throws:

IOException - if the delete fails

shardPath

public ShardPath shardPath()

recoverFromStore

public void recoverFromStore(ShardRouting shard,
                    StoreRecoveryService.RecoveryListener recoveryListener)

engine

public Engine engine()

engineUnsafe

protected Engine engineUnsafe()

NOTE: returns null if engine is not yet started (e.g. recovery phase 1, copying over index files, is still running), or if engine is closed.

newEngine

protected Engine newEngine(boolean skipTranslogRecovery,
               EngineConfig config)

allowsPrimaryPromotion

public boolean allowsPrimaryPromotion()

Returns true iff this shard allows primary promotion, otherwise false

incrementOperationCounter

public void incrementOperationCounter()

decrementOperationCounter

public void decrementOperationCounter()

getOperationsCount

public int getOperationsCount()

sync

public void sync(Translog.Location location)

Syncs the given location with the underlying storage unless already synced.

getTranslogDurability

public Translog.Durabilty getTranslogDurability()

Returns the current translog durability mode