Class TooManyDeletesCompactionStrategy
- java.lang.Object
-
- org.apache.accumulo.tserver.compaction.CompactionStrategy
-
- org.apache.accumulo.tserver.compaction.DefaultCompactionStrategy
-
- org.apache.accumulo.tserver.compaction.strategies.TooManyDeletesCompactionStrategy
-
public class TooManyDeletesCompactionStrategy extends DefaultCompactionStrategy
This compaction strategy works in concert with theDeletesSummarizer
. Using the statistics from DeleteSummarizer this strategy will compact all files in a table when the number of deletes/non-deletes exceeds a threshold.This strategy has two options. First the "threshold" option allows setting the point at which a compaction will be triggered. This options defaults to ".25" and must be in the range (0.0, 1.0]. The second option is "proceed_zero_no_summary" which determines if the strategy should proceed when a bulk imported file has no summary information.
If the delete summarizer was configured on a table that already had files, then those files will have not summary information. This strategy can still proceed in this situation. It will fall back to using Accumulo's estimated entries per file in this case. For the files without summary information the estimated number of deletes will be zero. This fall back method will underestimate deletes which will not lead to false positives, except for the case of bulk imported files. Accumulo estimates that bulk imported files have zero entires. The second option "proceed_zero_no_summary" determines if this strategy should proceed when it sees bulk imported files that do not have summary data. This option defaults to "false".
Bulk files can be generated with summary information by calling
AccumuloFileOutputFormat#setSummarizers(JobConf, SummarizerConfiguration...)
orRFile.WriterOptions.withSummarizers(SummarizerConfiguration...)
When this strategy does not decide to compact based on the number of deletes, then it will defer the decision to the
DefaultCompactionStrategy
.Configuring this compaction strategy for a table will cause it to always queue compactions, even though it may not decide to compact. These queued compactions may show up on the Accumulo monitor page. This is because summary data can not be read until after compaction is queued and dequeued. When the compaction is dequeued it can then decide not to compact. See ACCUMULO-4573
- Since:
- 2.0.0
-
-
Field Summary
Fields Modifier and Type Field Description static String
PROCEED_ZERO_NO_SUMMARY_OPT
static String
PROCEED_ZERO_NO_SUMMARY_OPT_DEFAULT
static String
THRESHOLD_OPT
This option should be a floating point number between 1 and 0.static String
THRESHOLD_OPT_DEFAULT
The default threshold.
-
Constructor Summary
Constructors Constructor Description TooManyDeletesCompactionStrategy()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
gatherInformation(MajorCompactionRequest request)
Called prior to obtaining the tablet lock, useful for examining metadata or indexes.CompactionPlan
getCompactionPlan(MajorCompactionRequest request)
Get the plan for compacting a tablets files.void
init(Map<String,String> options)
The settings for the compaction strategy pulled from zookeeper.boolean
shouldCompact(MajorCompactionRequest request)
Determine if this tablet is eligible for a major compaction.
-
-
-
Field Detail
-
THRESHOLD_OPT
public static final String THRESHOLD_OPT
This option should be a floating point number between 1 and 0.- See Also:
- Constant Field Values
-
THRESHOLD_OPT_DEFAULT
public static final String THRESHOLD_OPT_DEFAULT
The default threshold.- See Also:
- Constant Field Values
-
PROCEED_ZERO_NO_SUMMARY_OPT
public static final String PROCEED_ZERO_NO_SUMMARY_OPT
- See Also:
- Constant Field Values
-
PROCEED_ZERO_NO_SUMMARY_OPT_DEFAULT
public static final String PROCEED_ZERO_NO_SUMMARY_OPT_DEFAULT
- See Also:
- Constant Field Values
-
-
Method Detail
-
init
public void init(Map<String,String> options)
Description copied from class:CompactionStrategy
The settings for the compaction strategy pulled from zookeeper. Thetable.compacations.major.strategy.opts
part of the setting will be removed.- Overrides:
init
in classCompactionStrategy
-
shouldCompact
public boolean shouldCompact(MajorCompactionRequest request)
Description copied from class:CompactionStrategy
Determine if this tablet is eligible for a major compaction. It's ok if it later determines (throughCompactionStrategy.gatherInformation(MajorCompactionRequest)
andCompactionStrategy.getCompactionPlan(MajorCompactionRequest)
) that it does not need to. Any state stored during shouldCompact will no longer exist whenCompactionStrategy.gatherInformation(MajorCompactionRequest)
andCompactionStrategy.getCompactionPlan(MajorCompactionRequest)
are called.Called while holding the tablet lock, so it should not be doing any blocking.
Since no blocking should be done in this method, then its unexpected that this method will throw IOException. However since its in the API, it can not be easily removed.
- Overrides:
shouldCompact
in classDefaultCompactionStrategy
-
gatherInformation
public void gatherInformation(MajorCompactionRequest request) throws IOException
Description copied from class:CompactionStrategy
Called prior to obtaining the tablet lock, useful for examining metadata or indexes. State collected during this method will be available during the call theCompactionStrategy.getCompactionPlan(MajorCompactionRequest)
.- Overrides:
gatherInformation
in classCompactionStrategy
- Parameters:
request
- basic details about the tablet- Throws:
IOException
-
getCompactionPlan
public CompactionPlan getCompactionPlan(MajorCompactionRequest request)
Description copied from class:CompactionStrategy
Get the plan for compacting a tablets files. Called while holding the tablet lock, so it should not be doing any blocking.Since no blocking should be done in this method, then its unexpected that this method will throw IOException. However since its in the API, it can not be easily removed.
- Overrides:
getCompactionPlan
in classDefaultCompactionStrategy
- Parameters:
request
- basic details about the tablet- Returns:
- the plan for a major compaction, or null to cancel the compaction.
-
-