Module org.elasticsearch.server
Package org.elasticsearch.indices

Class SystemIndices

java.lang.Object
org.elasticsearch.indices.SystemIndices
public class SystemIndices extends Object
Provides information about system-owned indices and data streams for Elasticsearch and Elasticsearch plugins.

Various Elasticsearch features such as Security or Watcher keep their state in their own indices. We keep these indices separate from the user index space for a few reasons. In some cases, the indices contain information that should be hidden from users. But, more generally, we want to protect these indices and data streams from being inadvertently modified or deleted.

The system resources are grouped by feature, using the SystemIndices.Feature class. Most features will be loaded from instances of SystemIndexPlugin; any other features will be described in this class. Features may be retrieved by name or iterated over (see getFeature(String) and getFeatures()). Each Feature provides collections of SystemIndexDescriptors or SystemDataStreamDescriptors. These descriptors define their resources by means of patterns. Any index name or data stream name that matches one of these patterns is considered a system resource for * that descriptor, and user access to it will be restricted. These patterns are gathered and validated so that the SystemIndices object can provide information about system resources: for example, whether a particular string will be considered a “system name” (see isSystemIndex(String)).

For more information about the expected behavior of system indices, see SystemIndexDescriptor. For more information about the expected behavior of system data streams, see SystemDataStreamDescriptor.

The SystemIndices object is constructed during Node startup, and is not modified after construction. In other words, the set of system resources will be consistent over the lifetime of a node.

System resources will specify thread pools for reads, writes, and searches. This can ensure that system-critical operations, such as user authentication, are not blocked by heavy thread contention from user activities. getExecutorSelector() provides an object for convenient look-ups of these thread pools.

There are a few requirements for system features, collectively:

  1. The feature names must be distinct.
  2. System index patterns must not overlap.
  3. Aliases for system indices must be distinct.
  4. Feature names must not be reserved names. Right now, the only reserved name is “none”.

System index access is currently controlled by Security role index permissions. However, we have deprecated general rest access to system indices. This class provides checks for system index “access levels” (see getSystemIndexAccessLevel(ThreadContext)). If a request has the wrong access level for a system index it is targeting, then we will issue a deprecation warning. In the future, we will block access. The non-deprecated way to access certain external system indices is to use the correct request headers. This behavior is already in place in SystemDataStreamDescriptor and “net-new” system indices (see SystemIndexDescriptor.isNetNew()).

The implementation of the system index name checks makes heavy use of the Lucene Automaton class. At a high level, an automaton is a kind of matcher that can be created from a regex. Lucene Automata give us the ability to check for overlapping patterns, and to create efficient unions of patterns.

  • Field Details

  • Constructor Details

    • SystemIndices

      public SystemIndices(List<SystemIndices.Feature> pluginAndModuleFeatures)
      Initialize the SystemIndices object
      Parameters:
      pluginAndModuleFeatures - A list of features from which we will load system indices. These features come from plugins and modules. Non-plugin system features such as Tasks will be added automatically.

  • Method Details

    • isSystemName

      public boolean isSystemName(String name)
      Checks whether the given name matches a reserved name or pattern that is intended for use by a system component. The name is checked against index names, aliases, data stream names, and the names of indices that back a system data stream.

    • isSystemIndex

      public boolean isSystemIndex(Index index)
      Determines whether a given index is a system index by comparing its name to the collection of loaded SystemIndexDescriptors
      Parameters:
      index - the Index object to check against loaded SystemIndexDescriptors
      Returns:
      true if the Index's name matches a pattern from a SystemIndexDescriptor

    • isSystemIndex

      public boolean isSystemIndex(String indexName)
      Determines whether a given index is a system index by comparing its name to the collection of loaded SystemIndexDescriptors. This will also match alias names that belong to system indices.
      Parameters:
      indexName - the index name to check against loaded SystemIndexDescriptors
      Returns:
      true if the index name matches a pattern from a SystemIndexDescriptor

    • isSystemDataStream

      public boolean isSystemDataStream(String name)
      Determines whether the provided name matches that of a system data stream that has been defined by a SystemDataStreamDescriptor

    • isSystemIndexBackingDataStream

      public boolean isSystemIndexBackingDataStream(String name)
      Determines whether the provided name matches that of an index that backs a system data stream. Backing indices for system data streams are marked as "system" in their metadata (see SystemIndexMetadataUpgradeService) and receive the same protections as the system data stream.

    • getSystemNameAutomaton

      public org.apache.lucene.util.automaton.Automaton getSystemNameAutomaton()
      The Elasticsearch security plugin can use the automaton that matches all system resource names to efficiently authorize requests.
      Returns:
      An Automaton that tests whether strings are names of system indices, aliases, or data streams.

    • isNetNewSystemIndex

      public boolean isNetNewSystemIndex(String indexName)
      Checks whether an index is a net-new system index, meaning we can apply non-BWC behavior to it. See SystemIndexDescriptor.isNetNew().
      Parameters:
      indexName - The index name to check.
      Returns:
      true if the given index is covered by a net-new system index descriptor, false otherwise.

    • getExecutorSelector

      public ExecutorSelector getExecutorSelector()
      Used to determine which executor should be used for operations on this index. See ExecutorSelector docs for details.

    • findMatchingDescriptor

      @Nullable public SystemIndexDescriptor findMatchingDescriptor(String name)
      Finds a single matching SystemIndexDescriptor, if any, for the given index name.
      Parameters:
      name - the name of the index
      Returns:
      The matching SystemIndexDescriptor or null if no descriptor is found

    • findMatchingDataStreamDescriptor

      @Nullable public SystemDataStreamDescriptor findMatchingDataStreamDescriptor(String name)
      Finds a single matching SystemDataStreamDescriptor, if any, for the given DataStream name.
      Parameters:
      name - the name of the DataStream
      Returns:
      The matching SystemDataStreamDescriptor or null if no descriptor is found

    • getProductSystemIndexNamePredicate

      public Predicate<String> getProductSystemIndexNamePredicate(ThreadContext threadContext)
      Builds a predicate that tests whether a system index should be accessible for a given ThreadContext. We allow guaranteed (that is, non-deprecated) external access to system indices based on special request headers in addition to security roles. If those headers are present and provide a product name with access to the index, they will be added to the thread context and checked here. Without these headers, we will add deprecation warnings to the response. In future versions, we will deny access altogether.
      Parameters:
      threadContext - the threadContext containing headers used for system index access
      Returns:
      Predicate to check external system index names with

    • getFeatureNames

      public Set<String> getFeatureNames()
      Get a set of feature names. This is useful for checking whether particular features are present on the node.
      Returns:
      A set of all feature names

    • getFeature

      public SystemIndices.Feature getFeature(String name)
      Get a feature by name.
      Parameters:
      name - Name of a feature.
      Returns:
      The corresponding feature if it exists on this node, null otherwise.

    • getFeatures

      public Collection<SystemIndices.Feature> getFeatures()
      Get a collection of the Features this SystemIndices object is managing.
      Returns:
      A collection of Features.

    • validateDataStreamAccess

      public SystemDataStreamDescriptor validateDataStreamAccess(String dataStreamName, ThreadContext threadContext)

    • dataStreamAccessException

      public static IllegalArgumentException dataStreamAccessException(ThreadContext threadContext, Collection<String> names)

    • netNewSystemIndexAccessException

      public static IllegalArgumentException netNewSystemIndexAccessException(ThreadContext threadContext, Collection<String> names)

    • getSystemIndexAccessLevel

      public static SystemIndices.SystemIndexAccessLevel getSystemIndexAccessLevel(ThreadContext threadContext)
      Determines what level of system index access should be allowed in the current context. For system data streams and "net-new" system indices (see SystemIndexDescriptor.isNetNew()), access levels should be used to reject requests entirely. For non-net-new, backwards-compatible system indices, these access levels should be used for deprecation warnings.
      Parameters:
      threadContext - the current thread context that has headers associated with the current request
      Returns:
      SystemIndices.SystemIndexAccessLevel.ALL if unrestricted system index access should be allowed, SystemIndices.SystemIndexAccessLevel.RESTRICTED if a subset of system index access should be allowed, or SystemIndices.SystemIndexAccessLevel.NONE if no system index access should be allowed.

    • getMappingsVersions

      public Map<String,SystemIndexDescriptor.MappingsVersion> getMappingsVersions()

    • validateFeatureName

      public static void validateFeatureName(String name, String plugin)
      Check that a feature name is not reserved
      Parameters:
      name - Name of feature
      plugin - Name of plugin providing the feature