org.elasticsearch.indices.memory

Class IndexingMemoryController

java.lang.Object

org.elasticsearch.common.component.AbstractComponent

org.elasticsearch.common.component.AbstractLifecycleComponent<IndexingMemoryController>

org.elasticsearch.indices.memory.IndexingMemoryController

All Implemented Interfaces:

AutoCloseable, LifecycleComponent<IndexingMemoryController>, Releasable

public class IndexingMemoryController
extends AbstractLifecycleComponent<IndexingMemoryController>

Field Summary

Fields

Modifier and Type	Field and Description
static ByteSizeValue	INACTIVE_SHARD_INDEXING_BUFFER Once a shard becomes inactive, we reduce the IndexWriter buffer to this value (500 KB) to let active shards use the heap instead.
static ByteSizeValue	INACTIVE_SHARD_TRANSLOG_BUFFER Once a shard becomes inactive, we reduce the Translog buffer to this value (1 KB) to let active shards use the heap instead.
static String	INDEX_BUFFER_SIZE_SETTING How much heap (% or bytes) we will share across all actively indexing shards on this node (default: 10%).
static String	MAX_INDEX_BUFFER_SIZE_SETTING Only applies when indices.memory.index_buffer_size is a %, to set a ceiling on the actual size in bytes (default: not set).
static String	MAX_SHARD_INDEX_BUFFER_SIZE_SETTING Sets a ceiling on the per-shard index buffer size (default: 512 MB).
static String	MAX_SHARD_TRANSLOG_BUFFER_SIZE_SETTING Sets a ceiling on the per-shard translog buffer size (default: 64 KB).
static String	MAX_TRANSLOG_BUFFER_SIZE_SETTING Only applies when indices.memory.translog_buffer_size is a %, to set a ceiling on the actual size in bytes (default: not set).
static String	MIN_INDEX_BUFFER_SIZE_SETTING Only applies when indices.memory.index_buffer_size is a %, to set a floor on the actual size in bytes (default: 48 MB).
static String	MIN_SHARD_INDEX_BUFFER_SIZE_SETTING Sets a floor on the per-shard index buffer size (default: 4 MB).
static String	MIN_SHARD_TRANSLOG_BUFFER_SIZE_SETTING Sets a floor on the per-shard translog buffer size (default: 2 KB).
static String	MIN_TRANSLOG_BUFFER_SIZE_SETTING Only applies when indices.memory.translog_buffer_size is a %, to set a floor on the actual size in bytes (default: 256 KB).
static String	SHARD_INACTIVE_INTERVAL_TIME_SETTING How frequently we check shards to find inactive ones (default: 30 seconds).
static String	SHARD_INACTIVE_TIME_SETTING If we see no indexing operations after this much time for a given shard, we consider that shard inactive (default: 5 minutes).
static String	TRANSLOG_BUFFER_SIZE_SETTING How much heap (% or bytes) we will share across all actively indexing shards for the translog buffer (default: 1%).

Fields inherited from class org.elasticsearch.common.component.AbstractLifecycleComponent

lifecycle

Fields inherited from class org.elasticsearch.common.component.AbstractComponent

deprecationLogger, logger, settings

Constructor Summary

Constructors

Modifier	Constructor and Description
	IndexingMemoryController(Settings settings, ThreadPool threadPool, IndicesService indicesService)
protected	IndexingMemoryController(Settings settings, ThreadPool threadPool, IndicesService indicesService, long jvmMemoryInBytes)

Method Summary

Modifier and Type	Method and Description
protected List<IndexShard>	availableShards()
protected boolean	checkIdle(IndexShard shard, long inactiveTimeNS) ask this shard to check now whether it is inactive, and reduces its indexing and translog buffers if so.
protected long	currentTimeInNanos()
protected void	doClose()
protected void	doStart()
protected void	doStop()
void	forceCheck() check if any shards active status changed, now.
ByteSizeValue	indexingBufferSize() returns the current budget for the total amount of indexing buffers of active shards on this node
protected boolean	shardAvailable(IndexShard shard) returns true if shard exists and is availabe for updates
ByteSizeValue	translogBufferSize() returns the current budget for the total amount of translog buffers of active shards on this node
protected void	updateShardBuffers(IndexShard shard, ByteSizeValue shardIndexingBufferSize, ByteSizeValue shardTranslogBufferSize) set new indexing and translog buffers on this shard.

Methods inherited from class org.elasticsearch.common.component.AbstractLifecycleComponent

addLifecycleListener, close, lifecycleState, removeLifecycleListener, start, stop

Methods inherited from class org.elasticsearch.common.component.AbstractComponent

logDeprecatedSetting, logRemovedSetting, nodeName

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

INDEX_BUFFER_SIZE_SETTING

public static final String INDEX_BUFFER_SIZE_SETTING

How much heap (% or bytes) we will share across all actively indexing shards on this node (default: 10%).

See Also:

Constant Field Values

MIN_INDEX_BUFFER_SIZE_SETTING

public static final String MIN_INDEX_BUFFER_SIZE_SETTING

Only applies when indices.memory.index_buffer_size is a %, to set a floor on the actual size in bytes (default: 48 MB).

See Also:

Constant Field Values

MAX_INDEX_BUFFER_SIZE_SETTING

public static final String MAX_INDEX_BUFFER_SIZE_SETTING

Only applies when indices.memory.index_buffer_size is a %, to set a ceiling on the actual size in bytes (default: not set).

See Also:

Constant Field Values

MIN_SHARD_INDEX_BUFFER_SIZE_SETTING

public static final String MIN_SHARD_INDEX_BUFFER_SIZE_SETTING

Sets a floor on the per-shard index buffer size (default: 4 MB).

See Also:

Constant Field Values

MAX_SHARD_INDEX_BUFFER_SIZE_SETTING

public static final String MAX_SHARD_INDEX_BUFFER_SIZE_SETTING

Sets a ceiling on the per-shard index buffer size (default: 512 MB).

See Also:

Constant Field Values

TRANSLOG_BUFFER_SIZE_SETTING

public static final String TRANSLOG_BUFFER_SIZE_SETTING

How much heap (% or bytes) we will share across all actively indexing shards for the translog buffer (default: 1%).

See Also:

Constant Field Values

MIN_TRANSLOG_BUFFER_SIZE_SETTING

public static final String MIN_TRANSLOG_BUFFER_SIZE_SETTING

Only applies when indices.memory.translog_buffer_size is a %, to set a floor on the actual size in bytes (default: 256 KB).

See Also:

Constant Field Values

MAX_TRANSLOG_BUFFER_SIZE_SETTING

public static final String MAX_TRANSLOG_BUFFER_SIZE_SETTING

Only applies when indices.memory.translog_buffer_size is a %, to set a ceiling on the actual size in bytes (default: not set).

See Also:

Constant Field Values

MIN_SHARD_TRANSLOG_BUFFER_SIZE_SETTING

public static final String MIN_SHARD_TRANSLOG_BUFFER_SIZE_SETTING

Sets a floor on the per-shard translog buffer size (default: 2 KB).

See Also:

Constant Field Values

MAX_SHARD_TRANSLOG_BUFFER_SIZE_SETTING

public static final String MAX_SHARD_TRANSLOG_BUFFER_SIZE_SETTING

Sets a ceiling on the per-shard translog buffer size (default: 64 KB).

See Also:

Constant Field Values

SHARD_INACTIVE_TIME_SETTING

public static final String SHARD_INACTIVE_TIME_SETTING

If we see no indexing operations after this much time for a given shard, we consider that shard inactive (default: 5 minutes).

See Also:

Constant Field Values

SHARD_INACTIVE_INTERVAL_TIME_SETTING

public static final String SHARD_INACTIVE_INTERVAL_TIME_SETTING

How frequently we check shards to find inactive ones (default: 30 seconds).

See Also:

Constant Field Values

INACTIVE_SHARD_INDEXING_BUFFER

public static final ByteSizeValue INACTIVE_SHARD_INDEXING_BUFFER

Once a shard becomes inactive, we reduce the IndexWriter buffer to this value (500 KB) to let active shards use the heap instead.

INACTIVE_SHARD_TRANSLOG_BUFFER

public static final ByteSizeValue INACTIVE_SHARD_TRANSLOG_BUFFER

Once a shard becomes inactive, we reduce the Translog buffer to this value (1 KB) to let active shards use the heap instead.

Constructor Detail

IndexingMemoryController

@Inject
public IndexingMemoryController(Settings settings,
                                        ThreadPool threadPool,
                                        IndicesService indicesService)

IndexingMemoryController

protected IndexingMemoryController(Settings settings,
                                   ThreadPool threadPool,
                                   IndicesService indicesService,
                                   long jvmMemoryInBytes)

Method Detail

doStart

protected void doStart()

Specified by:

doStart in class AbstractLifecycleComponent<IndexingMemoryController>

doStop

protected void doStop()

Specified by:

doStop in class AbstractLifecycleComponent<IndexingMemoryController>

doClose

protected void doClose()

Specified by:

doClose in class AbstractLifecycleComponent<IndexingMemoryController>

indexingBufferSize

public ByteSizeValue indexingBufferSize()

returns the current budget for the total amount of indexing buffers of active shards on this node

translogBufferSize

public ByteSizeValue translogBufferSize()

returns the current budget for the total amount of translog buffers of active shards on this node

availableShards

protected List<IndexShard> availableShards()

shardAvailable

protected boolean shardAvailable(@Nullable
                                 IndexShard shard)

returns true if shard exists and is availabe for updates

updateShardBuffers

protected void updateShardBuffers(IndexShard shard,
                                  ByteSizeValue shardIndexingBufferSize,
                                  ByteSizeValue shardTranslogBufferSize)

set new indexing and translog buffers on this shard. this may cause the shard to refresh to free up heap.

forceCheck

public void forceCheck()

check if any shards active status changed, now.

currentTimeInNanos

protected long currentTimeInNanos()

checkIdle

protected boolean checkIdle(IndexShard shard,
                            long inactiveTimeNS)

ask this shard to check now whether it is inactive, and reduces its indexing and translog buffers if so. return false if the shard is not idle, otherwise true