public final class MergePolicyConfig extends Object implements IndexSettingsService.Listener
Merges select segments of approximately equal size, subject to an allowed number of segments per tier. The merge policy is able to merge non-adjacent segments, and separates how many segments are merged at once from how many segments are allowed per tier. It also does not over-merge (i.e., cascade merges).
All merge policy settings are dynamic and can be updated on a live index. The merge policy has the following settings:
index.merge.policy.expunge_deletes_allowed
:
When expungeDeletes is called, we only merge away a segment if its delete
percentage is over this threshold. Default is 10
.
index.merge.policy.floor_segment
:
Segments smaller than this are "rounded up" to this size, i.e. treated as
equal (floor) size for merge selection. This is to prevent frequent
flushing of tiny segments, thus preventing a long tail in the index. Default
is 2mb
.
index.merge.policy.max_merge_at_once
:
Maximum number of segments to be merged at a time during "normal" merging.
Default is 10
.
index.merge.policy.max_merge_at_once_explicit
:
Maximum number of segments to be merged at a time, during optimize or
expungeDeletes. Default is 30
.
index.merge.policy.max_merged_segment
:
Maximum sized segment to produce during normal merging (not explicit
optimize). This setting is approximate: the estimate of the merged segment
size is made by summing sizes of to-be-merged segments (compensating for
percent deleted docs). Default is 5gb
.
index.merge.policy.segments_per_tier
:
Sets the allowed number of segments per tier. Smaller values mean more
merging but fewer segments. Default is 10
. Note, this value needs to be
>= than the max_merge_at_once
otherwise you'll force too many merges to
occur.
index.merge.policy.reclaim_deletes_weight
:
Controls how aggressively merges that reclaim more deletions are favored.
Higher values favor selecting merges that reclaim deletions. A value of
0.0
means deletions don't impact merge selection. Defaults to 2.0
.
For normal merging, the policy first computes a "budget" of how many segments are allowed to be in the index. If the index is over-budget, then the policy sorts segments by decreasing size (proportionally considering percent deletes), and then finds the least-cost merge. Merge cost is measured by a combination of the "skew" of the merge (size of largest seg divided by smallest seg), total merge size and pct deletes reclaimed, so that merges with lower skew, smaller size and those reclaiming more deletes, are favored.
If a merge will produce a segment that's larger than
max_merged_segment
then the policy will merge fewer segments (down to
1 at once, if that one has deletions) to keep the segment size under
budget.
Note, this can mean that for large shards that holds many gigabytes of
data, the default of max_merged_segment
(5gb
) can cause for many
segments to be in an index, and causing searches to be slower. Use the
indices segments API to see the segments that an index has, and
possibly either increase the max_merged_segment
or issue an optimize
call for the index (try and aim to issue it on a low traffic time).
Modifier and Type | Field and Description |
---|---|
static double |
DEFAULT_EXPUNGE_DELETES_ALLOWED |
static ByteSizeValue |
DEFAULT_FLOOR_SEGMENT |
static int |
DEFAULT_MAX_MERGE_AT_ONCE |
static int |
DEFAULT_MAX_MERGE_AT_ONCE_EXPLICIT |
static ByteSizeValue |
DEFAULT_MAX_MERGED_SEGMENT |
static double |
DEFAULT_RECLAIM_DELETES_WEIGHT |
static double |
DEFAULT_SEGMENTS_PER_TIER |
static String |
INDEX_COMPOUND_FORMAT |
static String |
INDEX_MERGE_ENABLED |
static String |
INDEX_MERGE_POLICY_EXPUNGE_DELETES_ALLOWED |
static String |
INDEX_MERGE_POLICY_FLOOR_SEGMENT |
static String |
INDEX_MERGE_POLICY_MAX_MERGE_AT_ONCE |
static String |
INDEX_MERGE_POLICY_MAX_MERGE_AT_ONCE_EXPLICIT |
static String |
INDEX_MERGE_POLICY_MAX_MERGED_SEGMENT |
static String |
INDEX_MERGE_POLICY_RECLAIM_DELETES_WEIGHT |
static String |
INDEX_MERGE_POLICY_SEGMENTS_PER_TIER |
Constructor and Description |
---|
MergePolicyConfig(ESLogger logger,
Settings indexSettings) |
Modifier and Type | Method and Description |
---|---|
static String |
formatNoCFSRatio(double ratio) |
org.apache.lucene.index.MergePolicy |
getMergePolicy() |
void |
onRefreshSettings(Settings settings) |
static double |
parseNoCFSRatio(String noCFSRatio) |
public static final double DEFAULT_EXPUNGE_DELETES_ALLOWED
public static final ByteSizeValue DEFAULT_FLOOR_SEGMENT
public static final int DEFAULT_MAX_MERGE_AT_ONCE
public static final int DEFAULT_MAX_MERGE_AT_ONCE_EXPLICIT
public static final ByteSizeValue DEFAULT_MAX_MERGED_SEGMENT
public static final double DEFAULT_SEGMENTS_PER_TIER
public static final double DEFAULT_RECLAIM_DELETES_WEIGHT
public static final String INDEX_COMPOUND_FORMAT
public static final String INDEX_MERGE_POLICY_EXPUNGE_DELETES_ALLOWED
public static final String INDEX_MERGE_POLICY_FLOOR_SEGMENT
public static final String INDEX_MERGE_POLICY_MAX_MERGE_AT_ONCE
public static final String INDEX_MERGE_POLICY_MAX_MERGE_AT_ONCE_EXPLICIT
public static final String INDEX_MERGE_POLICY_MAX_MERGED_SEGMENT
public static final String INDEX_MERGE_POLICY_SEGMENTS_PER_TIER
public static final String INDEX_MERGE_POLICY_RECLAIM_DELETES_WEIGHT
public static final String INDEX_MERGE_ENABLED
public org.apache.lucene.index.MergePolicy getMergePolicy()
public void onRefreshSettings(Settings settings)
onRefreshSettings
in interface IndexSettingsService.Listener
public static double parseNoCFSRatio(String noCFSRatio)
public static String formatNoCFSRatio(double ratio)
Copyright © 2009–2016. All rights reserved.