Package org.elasticsearch.env
Class NodeEnvironment
java.lang.Object
org.elasticsearch.env.NodeEnvironment
- All Implemented Interfaces:
java.io.Closeable
,java.lang.AutoCloseable
public final class NodeEnvironment
extends java.lang.Object
implements java.io.Closeable
A component that holds all data paths for a single node.
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
NodeEnvironment.NodeLock
static class
NodeEnvironment.NodePath
static interface
NodeEnvironment.ShardLocker
A functional interface that people can use to referenceshardLock(ShardId, String, long)
-
Field Summary
Fields Modifier and Type Field Description static Setting<java.lang.Boolean>
ENABLE_LUCENE_SEGMENT_INFOS_TRACE_SETTING
If true the [verbose] SegmentInfos.infoStream logging is sent to System.out.static java.lang.String
INDICES_FOLDER
static Setting<java.lang.Integer>
MAX_LOCAL_STORAGE_NODES_SETTING
Maximum number of data nodes that should run in an environment.static Setting<java.lang.Long>
NODE_ID_SEED_SETTING
Seed for determining a persisted unique uuid of this node.static java.lang.String
NODE_LOCK_FILENAME
static java.lang.String
NODES_FOLDER
-
Constructor Summary
Constructors Constructor Description NodeEnvironment(Settings settings, Environment environment)
Setup the environment. -
Method Summary
Modifier and Type Method Description static void
acquireFSLockForPaths(IndexSettings indexSettings, java.nio.file.Path... shardPaths)
Acquires, then releases, allwrite.lock
files in the given shard paths.java.util.Set<java.lang.String>
availableIndexFolders()
Returns all folder names in ${data.paths}/nodes/{node.id}/indices folderjava.util.Set<java.lang.String>
availableIndexFolders(java.util.function.Predicate<java.lang.String> excludeIndexPathIdsPredicate)
Returns folder names in ${data.paths}/nodes/{node.id}/indices folder that don't match the given predicate.java.util.Set<java.lang.String>
availableIndexFoldersForPath(NodeEnvironment.NodePath nodePath)
Return all directory names in the nodes/{node.id}/indices directory for the given node path.java.util.Set<java.lang.String>
availableIndexFoldersForPath(NodeEnvironment.NodePath nodePath, java.util.function.Predicate<java.lang.String> excludeIndexPathIdsPredicate)
Return directory names in the nodes/{node.id}/indices directory for the given node path that don't match the given predicate.java.nio.file.Path[]
availableShardPaths(ShardId shardId)
Returns all shard paths excluding custom shard path.void
close()
void
deleteIndexDirectorySafe(Index index, long lockTimeoutMS, IndexSettings indexSettings)
Deletes an indexes data directory recursively iff all of the indexes shards locks were successfully acquired.void
deleteIndexDirectoryUnderLock(Index index, IndexSettings indexSettings)
Deletes an indexes data directory recursively.void
deleteShardDirectorySafe(ShardId shardId, IndexSettings indexSettings)
Deletes a shard data directory iff the shards locks were successfully acquired.void
deleteShardDirectoryUnderLock(ShardLock lock, IndexSettings indexSettings)
Deletes a shard data directory.java.util.Set<ShardId>
findAllShardIds(Index index)
Tries to find all allocated shards for the given index on the current node.static java.lang.String
generateNodeId(Settings settings)
int
getNodeLockId()
boolean
hasNodeFile()
java.nio.file.Path[]
indexPaths(Index index)
Returns all index paths.java.util.List<ShardLock>
lockAllForIndex(Index index, IndexSettings settings, java.lang.String lockDetails, long lockTimeoutMS)
Tries to lock all local shards for the given index.java.util.Set<ShardId>
lockedShards()
Returns all currently lock shards.java.nio.file.Path[]
nodeDataPaths()
Returns an array of all of the nodes data locations.java.lang.String
nodeId()
returns the unique uuid describing this node.NodeEnvironment.NodePath[]
nodePaths()
Returns an array of all of theNodeEnvironment.NodePath
s.static java.nio.file.Path
resolveBaseCustomLocation(java.lang.String customDataPath, java.nio.file.Path sharedDataPath, int nodeLockId)
Resolve the custom path for a index's shard.java.nio.file.Path
resolveCustomLocation(java.lang.String customDataPath, ShardId shardId)
Resolve the custom path for a index's shard.static java.nio.file.Path
resolveCustomLocation(java.lang.String customDataPath, ShardId shardId, java.nio.file.Path sharedDataPath, int nodeLockId)
java.nio.file.Path[]
resolveIndexFolder(java.lang.String indexFolderName)
Resolves all existing paths toindexFolderName
in ${data.paths}/nodes/{node.id}/indicesstatic java.nio.file.Path
resolveNodePath(java.nio.file.Path path, int nodeLockId)
Resolve a specific nodes/{node.id} path for the specified path and node lock id.java.util.Map<NodeEnvironment.NodePath,java.lang.Long>
shardCountPerPath(Index index)
Find all the shards for this index, returning a map of theNodePath
to the number of shards on that pathShardLock
shardLock(ShardId id, java.lang.String details)
Tries to lock the given shards ID.ShardLock
shardLock(ShardId shardId, java.lang.String details, long lockTimeoutMS)
Tries to lock the given shards ID.static java.nio.file.Path
shardStatePathToDataPath(java.nio.file.Path shardPath)
Returns theNodePath.path
for this shard.java.nio.file.Path
sharedDataPath()
Returns shared data path for this node environment
-
Field Details
-
MAX_LOCAL_STORAGE_NODES_SETTING
Maximum number of data nodes that should run in an environment. -
NODE_ID_SEED_SETTING
Seed for determining a persisted unique uuid of this node. If the node has already a persisted uuid on disk, this seed will be ignored and the uuid from disk will be reused. -
ENABLE_LUCENE_SEGMENT_INFOS_TRACE_SETTING
If true the [verbose] SegmentInfos.infoStream logging is sent to System.out. -
NODES_FOLDER
public static final java.lang.String NODES_FOLDER- See Also:
- Constant Field Values
-
INDICES_FOLDER
public static final java.lang.String INDICES_FOLDER- See Also:
- Constant Field Values
-
NODE_LOCK_FILENAME
public static final java.lang.String NODE_LOCK_FILENAME- See Also:
- Constant Field Values
-
-
Constructor Details
-
NodeEnvironment
Setup the environment.- Parameters:
settings
- settings from elasticsearch.yml- Throws:
java.io.IOException
-
-
Method Details
-
resolveNodePath
public static java.nio.file.Path resolveNodePath(java.nio.file.Path path, int nodeLockId)Resolve a specific nodes/{node.id} path for the specified path and node lock id.- Parameters:
path
- the pathnodeLockId
- the node lock id- Returns:
- the resolved path
-
generateNodeId
-
deleteShardDirectorySafe
public void deleteShardDirectorySafe(ShardId shardId, IndexSettings indexSettings) throws java.io.IOException, ShardLockObtainFailedExceptionDeletes a shard data directory iff the shards locks were successfully acquired.- Parameters:
shardId
- the id of the shard to delete to delete- Throws:
java.io.IOException
- if an IOException occursShardLockObtainFailedException
-
acquireFSLockForPaths
public static void acquireFSLockForPaths(IndexSettings indexSettings, java.nio.file.Path... shardPaths) throws java.io.IOExceptionAcquires, then releases, allwrite.lock
files in the given shard paths. The "write.lock" file is assumed to be under the shard path's "index" directory as used by Elasticsearch.- Throws:
org.apache.lucene.store.LockObtainFailedException
- if any of the locks could not be acquiredjava.io.IOException
-
deleteShardDirectoryUnderLock
public void deleteShardDirectoryUnderLock(ShardLock lock, IndexSettings indexSettings) throws java.io.IOExceptionDeletes a shard data directory. Note: this method assumes that the shard lock is acquired. This method will also attempt to acquire the write locks for the shard's paths before deleting the data, but this is best effort, as the lock is released before the deletion happens in order to allow the folder to be deleted- Parameters:
lock
- the shards lock- Throws:
java.io.IOException
- if an IOException occursElasticsearchException
- if the write.lock is not acquirable
-
deleteIndexDirectorySafe
public void deleteIndexDirectorySafe(Index index, long lockTimeoutMS, IndexSettings indexSettings) throws java.io.IOException, ShardLockObtainFailedExceptionDeletes an indexes data directory recursively iff all of the indexes shards locks were successfully acquired. If any of the indexes shard directories can't be locked non of the shards will be deleted- Parameters:
index
- the index to deletelockTimeoutMS
- how long to wait for acquiring the indices shard locksindexSettings
- settings for the index being deleted- Throws:
java.io.IOException
- if any of the shards data directories can't be locked or deletedShardLockObtainFailedException
-
deleteIndexDirectoryUnderLock
public void deleteIndexDirectoryUnderLock(Index index, IndexSettings indexSettings) throws java.io.IOExceptionDeletes an indexes data directory recursively. Note: this method assumes that the shard lock is acquired- Parameters:
index
- the index to deleteindexSettings
- settings for the index being deleted- Throws:
java.io.IOException
-
lockAllForIndex
public java.util.List<ShardLock> lockAllForIndex(Index index, IndexSettings settings, java.lang.String lockDetails, long lockTimeoutMS) throws ShardLockObtainFailedExceptionTries to lock all local shards for the given index. If any of the shard locks can't be acquired aShardLockObtainFailedException
is thrown and all previously acquired locks are released.- Parameters:
index
- the index to lock shards forlockTimeoutMS
- how long to wait for acquiring the indices shard locks- Returns:
- the
ShardLock
instances for this index. - Throws:
ShardLockObtainFailedException
-
shardLock
public ShardLock shardLock(ShardId id, java.lang.String details) throws ShardLockObtainFailedExceptionTries to lock the given shards ID. A shard lock is required to perform any kind of write operation on a shards data directory like deleting files, creating a new index writer or recover from a different shard instance into it. If the shard lock can not be acquired aShardLockObtainFailedException
is thrown. Note: this method will return immediately if the lock can't be acquired.- Parameters:
id
- the shard ID to lockdetails
- information about why the shard is being locked- Returns:
- the shard lock. Call
ShardLock.close()
to release the lock - Throws:
ShardLockObtainFailedException
-
shardLock
public ShardLock shardLock(ShardId shardId, java.lang.String details, long lockTimeoutMS) throws ShardLockObtainFailedExceptionTries to lock the given shards ID. A shard lock is required to perform any kind of write operation on a shards data directory like deleting files, creating a new index writer or recover from a different shard instance into it. If the shard lock can not be acquired aShardLockObtainFailedException
is thrown- Parameters:
shardId
- the shard ID to lockdetails
- information about why the shard is being lockedlockTimeoutMS
- the lock timeout in milliseconds- Returns:
- the shard lock. Call
ShardLock.close()
to release the lock - Throws:
ShardLockObtainFailedException
-
lockedShards
Returns all currently lock shards. Note: the shard ids return do not contain a valid Index UUID -
hasNodeFile
public boolean hasNodeFile() -
nodeDataPaths
public java.nio.file.Path[] nodeDataPaths()Returns an array of all of the nodes data locations.- Throws:
java.lang.IllegalStateException
- if the node is not configured to store local locations
-
sharedDataPath
public java.nio.file.Path sharedDataPath()Returns shared data path for this node environment -
nodeId
public java.lang.String nodeId()returns the unique uuid describing this node. The uuid is persistent in the data folder of this node and remains across restarts. -
nodePaths
Returns an array of all of theNodeEnvironment.NodePath
s. -
getNodeLockId
public int getNodeLockId() -
indexPaths
Returns all index paths. -
availableShardPaths
Returns all shard paths excluding custom shard path. Note: Shards are only allocated on one of the returned paths. The returned array may contain paths to non-existing directories. -
availableIndexFolders
public java.util.Set<java.lang.String> availableIndexFolders() throws java.io.IOExceptionReturns all folder names in ${data.paths}/nodes/{node.id}/indices folder- Throws:
java.io.IOException
-
availableIndexFolders
public java.util.Set<java.lang.String> availableIndexFolders(java.util.function.Predicate<java.lang.String> excludeIndexPathIdsPredicate) throws java.io.IOExceptionReturns folder names in ${data.paths}/nodes/{node.id}/indices folder that don't match the given predicate.- Parameters:
excludeIndexPathIdsPredicate
- folder names to exclude- Throws:
java.io.IOException
-
availableIndexFoldersForPath
public java.util.Set<java.lang.String> availableIndexFoldersForPath(NodeEnvironment.NodePath nodePath) throws java.io.IOExceptionReturn all directory names in the nodes/{node.id}/indices directory for the given node path.- Parameters:
nodePath
- the path- Returns:
- all directories that could be indices for the given node path.
- Throws:
java.io.IOException
- if an I/O exception occurs traversing the filesystem
-
availableIndexFoldersForPath
public java.util.Set<java.lang.String> availableIndexFoldersForPath(NodeEnvironment.NodePath nodePath, java.util.function.Predicate<java.lang.String> excludeIndexPathIdsPredicate) throws java.io.IOExceptionReturn directory names in the nodes/{node.id}/indices directory for the given node path that don't match the given predicate.- Parameters:
nodePath
- the pathexcludeIndexPathIdsPredicate
- folder names to exclude- Returns:
- all directories that could be indices for the given node path.
- Throws:
java.io.IOException
- if an I/O exception occurs traversing the filesystem
-
resolveIndexFolder
public java.nio.file.Path[] resolveIndexFolder(java.lang.String indexFolderName)Resolves all existing paths toindexFolderName
in ${data.paths}/nodes/{node.id}/indices -
findAllShardIds
Tries to find all allocated shards for the given index on the current node. NOTE: This methods is prone to race-conditions on the filesystem layer since it might not see directories created concurrently or while it's traversing.- Parameters:
index
- the index to filter shards- Returns:
- a set of shard IDs
- Throws:
java.io.IOException
- if an IOException occurs
-
shardCountPerPath
public java.util.Map<NodeEnvironment.NodePath,java.lang.Long> shardCountPerPath(Index index) throws java.io.IOExceptionFind all the shards for this index, returning a map of theNodePath
to the number of shards on that path- Parameters:
index
- the index by which to filter shards- Returns:
- a map of NodePath to count of the shards for the index on that path
- Throws:
java.io.IOException
- if an IOException occurs
-
close
public void close()- Specified by:
close
in interfacejava.lang.AutoCloseable
- Specified by:
close
in interfacejava.io.Closeable
-
resolveBaseCustomLocation
public static java.nio.file.Path resolveBaseCustomLocation(java.lang.String customDataPath, java.nio.file.Path sharedDataPath, int nodeLockId)Resolve the custom path for a index's shard. -
resolveCustomLocation
Resolve the custom path for a index's shard. Uses theIndexMetadata.SETTING_DATA_PATH
setting to determine the root path for the index.- Parameters:
customDataPath
- the custom data pathshardId
- shard to resolve the path to
-
resolveCustomLocation
public static java.nio.file.Path resolveCustomLocation(java.lang.String customDataPath, ShardId shardId, java.nio.file.Path sharedDataPath, int nodeLockId) -
shardStatePathToDataPath
public static java.nio.file.Path shardStatePathToDataPath(java.nio.file.Path shardPath)Returns theNodePath.path
for this shard.
-