org.apache.cassandra.db.compaction

Class AbstractCompactionStrategy

java.lang.Object

org.apache.cassandra.db.compaction.AbstractCompactionStrategy

Direct Known Subclasses:

DateTieredCompactionStrategy, LeveledCompactionStrategy, SizeTieredCompactionStrategy, WrappingCompactionStrategy

public abstract class AbstractCompactionStrategy
extends java.lang.Object

Pluggable compaction strategy determines how SSTables get merged. There are two main goals: - perform background compaction constantly as needed; this typically makes a tradeoff between i/o done by compaction, and merging done at read time. - perform a full (maximum possible) compaction if requested by the user

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AbstractCompactionStrategy.ScannerList

Field Summary

Fields

Modifier and Type	Field and Description
protected ColumnFamilyStore	cfs
protected static java.lang.String	COMPACTION_ENABLED
protected static long	DEFAULT_TOMBSTONE_COMPACTION_INTERVAL
protected static float	DEFAULT_TOMBSTONE_THRESHOLD
protected static boolean	DEFAULT_UNCHECKED_TOMBSTONE_COMPACTION_OPTION
protected boolean	disableTombstoneCompactions
protected boolean	enabled
protected boolean	isActive pause/resume/getNextBackgroundTask must synchronize.
java.util.Map<java.lang.String,java.lang.String>	options
protected static java.lang.String	TOMBSTONE_COMPACTION_INTERVAL_OPTION
protected static java.lang.String	TOMBSTONE_THRESHOLD_OPTION
protected long	tombstoneCompactionInterval
protected float	tombstoneThreshold
protected static java.lang.String	UNCHECKED_TOMBSTONE_COMPACTION_OPTION
protected boolean	uncheckedTombstoneCompaction

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractCompactionStrategy(ColumnFamilyStore cfs, java.util.Map<java.lang.String,java.lang.String> options)

Method Summary

Methods

Modifier and Type	Method and Description
abstract void	addSSTable(SSTableReader added)
void	disable()
void	enable()
java.util.List<SSTableReader>	filterSSTablesForReads(java.util.List<SSTableReader> sstables)
static java.lang.Iterable<SSTableReader>	filterSuspectSSTables(java.lang.Iterable<SSTableReader> originalCandidates) Filters SSTables that are to be blacklisted from the given collection
AbstractCompactionTask	getCompactionTask(LifecycleTransaction txn, int gcBefore, long maxSSTableBytes)
abstract int	getEstimatedRemainingTasks()
abstract java.util.Collection<AbstractCompactionTask>	getMaximalTask(int gcBefore, boolean splitOutput)
abstract long	getMaxSSTableBytes()
long	getMemtableReservedSize() If not affected by MeteredFlusher (and handling flushing on its own), override to tell MF how much space to reserve for this CF, i.e., how much space to subtract from `memtable_total_space_in_mb` when deciding if other memtables should be flushed or not.
java.lang.String	getName()
abstract AbstractCompactionTask	getNextBackgroundTask(int gcBefore)
AbstractCompactionStrategy.ScannerList	getScanners(java.util.Collection<SSTableReader> toCompact)
AbstractCompactionStrategy.ScannerList	getScanners(java.util.Collection<SSTableReader> sstables, Range<Token> range) Returns a list of KeyScanners given sstables and a range on which to scan.
abstract AbstractCompactionTask	getUserDefinedTask(java.util.Collection<SSTableReader> sstables, int gcBefore)
java.util.Collection<java.util.Collection<SSTableReader>>	groupSSTablesForAntiCompaction(java.util.Collection<SSTableReader> sstablesToGroup) Method for grouping similar SSTables together, This will be used by anti-compaction to determine which SSTables should be anitcompacted as a group.
boolean	isAffectedByMeteredFlusher()
boolean	isEnabled()
void	pause() For internal, temporary suspension of background compactions so that we can do exceptional things like truncate or major compaction
abstract void	removeSSTable(SSTableReader sstable)
void	replaceFlushed(Memtable memtable, SSTableReader sstable) Handle a flushed memtable.
void	replaceSSTables(java.util.Collection<SSTableReader> removed, java.util.Collection<SSTableReader> added)
void	resume() For internal, temporary suspension of background compactions so that we can do exceptional things like truncate or major compaction
boolean	shouldBeEnabled()
boolean	shouldDefragment()
void	shutdown() Releases any resources if this strategy is shutdown (when the CFS is reloaded after a schema change).
void	startup() Performs any extra initialization required
static java.util.Map<java.lang.String,java.lang.String>	validateOptions(java.util.Map<java.lang.String,java.lang.String> options)
protected boolean	worthDroppingTombstones(SSTableReader sstable, int gcBefore) Check if given sstable is worth dropping tombstones at gcBefore.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_TOMBSTONE_THRESHOLD

protected static final float DEFAULT_TOMBSTONE_THRESHOLD

See Also:

Constant Field Values

DEFAULT_TOMBSTONE_COMPACTION_INTERVAL

protected static final long DEFAULT_TOMBSTONE_COMPACTION_INTERVAL

See Also:

Constant Field Values

DEFAULT_UNCHECKED_TOMBSTONE_COMPACTION_OPTION

protected static final boolean DEFAULT_UNCHECKED_TOMBSTONE_COMPACTION_OPTION

See Also:

Constant Field Values

TOMBSTONE_THRESHOLD_OPTION

protected static final java.lang.String TOMBSTONE_THRESHOLD_OPTION

See Also:

Constant Field Values

TOMBSTONE_COMPACTION_INTERVAL_OPTION

protected static final java.lang.String TOMBSTONE_COMPACTION_INTERVAL_OPTION

See Also:

Constant Field Values

UNCHECKED_TOMBSTONE_COMPACTION_OPTION

protected static final java.lang.String UNCHECKED_TOMBSTONE_COMPACTION_OPTION

See Also:

Constant Field Values

COMPACTION_ENABLED

protected static final java.lang.String COMPACTION_ENABLED

See Also:

Constant Field Values

options

public java.util.Map<java.lang.String,java.lang.String> options

cfs

protected final ColumnFamilyStore cfs

tombstoneThreshold

protected float tombstoneThreshold

tombstoneCompactionInterval

protected long tombstoneCompactionInterval

uncheckedTombstoneCompaction

protected boolean uncheckedTombstoneCompaction

disableTombstoneCompactions

protected boolean disableTombstoneCompactions

isActive

protected boolean isActive

pause/resume/getNextBackgroundTask must synchronize. This guarantees that after pause completes, no new tasks will be generated; or put another way, pause can't run until in-progress tasks are done being created. This allows runWithCompactionsDisabled to be confident that after pausing, once in-progress tasks abort, it's safe to proceed with truncate/cleanup/etc. See CASSANDRA-3430

enabled

protected volatile boolean enabled

Constructor Detail

AbstractCompactionStrategy

protected AbstractCompactionStrategy(ColumnFamilyStore cfs,
                          java.util.Map<java.lang.String,java.lang.String> options)

Method Detail

pause

public void pause()

For internal, temporary suspension of background compactions so that we can do exceptional things like truncate or major compaction

resume

public void resume()

For internal, temporary suspension of background compactions so that we can do exceptional things like truncate or major compaction

startup

public void startup()

Performs any extra initialization required

shutdown

public void shutdown()

Releases any resources if this strategy is shutdown (when the CFS is reloaded after a schema change).

getNextBackgroundTask

public abstract AbstractCompactionTask getNextBackgroundTask(int gcBefore)

Parameters:

gcBefore - throw away tombstones older than this

Returns:

the next background/minor compaction task to run; null if nothing to do. Is responsible for marking its sstables as compaction-pending.

getMaximalTask

public abstract java.util.Collection<AbstractCompactionTask> getMaximalTask(int gcBefore,
                                                          boolean splitOutput)

Parameters:

gcBefore - throw away tombstones older than this

Returns:

a compaction task that should be run to compact this columnfamilystore as much as possible. Null if nothing to do. Is responsible for marking its sstables as compaction-pending.

getUserDefinedTask

public abstract AbstractCompactionTask getUserDefinedTask(java.util.Collection<SSTableReader> sstables,
                                        int gcBefore)

Parameters:

sstables - SSTables to compact. Must be marked as compacting.

gcBefore - throw away tombstones older than this

Returns:

a compaction task corresponding to the requested sstables. Will not be null. (Will throw if user requests an invalid compaction.) Is responsible for marking its sstables as compaction-pending.

getCompactionTask

public AbstractCompactionTask getCompactionTask(LifecycleTransaction txn,
                                       int gcBefore,
                                       long maxSSTableBytes)

getEstimatedRemainingTasks

public abstract int getEstimatedRemainingTasks()

Returns:

the number of background tasks estimated to still be needed for this columnfamilystore

getMaxSSTableBytes

public abstract long getMaxSSTableBytes()

Returns:

size in bytes of the largest sstables for this strategy

isEnabled

public boolean isEnabled()

enable

public void enable()

disable

public void disable()

isAffectedByMeteredFlusher

public boolean isAffectedByMeteredFlusher()

Returns:

whether or not MeteredFlusher should be able to trigger memtable flushes for this CF.

getMemtableReservedSize

public long getMemtableReservedSize()

If not affected by MeteredFlusher (and handling flushing on its own), override to tell MF how much space to reserve for this CF, i.e., how much space to subtract from `memtable_total_space_in_mb` when deciding if other memtables should be flushed or not.

replaceFlushed

public void replaceFlushed(Memtable memtable,
                  SSTableReader sstable)

Handle a flushed memtable.

Parameters:

memtable - the flushed memtable

sstable - the written sstable. can be null if the memtable was clean.

filterSSTablesForReads

public java.util.List<SSTableReader> filterSSTablesForReads(java.util.List<SSTableReader> sstables)

Returns:

a subset of the suggested sstables that are relevant for read requests.

filterSuspectSSTables

public static java.lang.Iterable<SSTableReader> filterSuspectSSTables(java.lang.Iterable<SSTableReader> originalCandidates)

Filters SSTables that are to be blacklisted from the given collection

Parameters:

originalCandidates - The collection to check for blacklisted SSTables

Returns:

list of the SSTables with blacklisted ones filtered out

getScanners

public AbstractCompactionStrategy.ScannerList getScanners(java.util.Collection<SSTableReader> sstables,
                                                 Range<Token> range)

Returns a list of KeyScanners given sstables and a range on which to scan. The default implementation simply grab one SSTableScanner per-sstable, but overriding this method allow for a more memory efficient solution if we know the sstable don't overlap (see LeveledCompactionStrategy for instance).

shouldDefragment

public boolean shouldDefragment()

getName

public java.lang.String getName()

replaceSSTables

public void replaceSSTables(java.util.Collection<SSTableReader> removed,
                   java.util.Collection<SSTableReader> added)

addSSTable

public abstract void addSSTable(SSTableReader added)

removeSSTable

public abstract void removeSSTable(SSTableReader sstable)

getScanners

public AbstractCompactionStrategy.ScannerList getScanners(java.util.Collection<SSTableReader> toCompact)

worthDroppingTombstones

protected boolean worthDroppingTombstones(SSTableReader sstable,
                              int gcBefore)

Check if given sstable is worth dropping tombstones at gcBefore. Check is skipped if tombstone_compaction_interval time does not elapse since sstable creation and returns false.

Parameters:

sstable - SSTable to check

gcBefore - time to drop tombstones

Returns:

true if given sstable's tombstones are expected to be removed

validateOptions

public static java.util.Map<java.lang.String,java.lang.String> validateOptions(java.util.Map<java.lang.String,java.lang.String> options)
                                                                        throws ConfigurationException

Throws:

ConfigurationException

shouldBeEnabled

public boolean shouldBeEnabled()

groupSSTablesForAntiCompaction

public java.util.Collection<java.util.Collection<SSTableReader>> groupSSTablesForAntiCompaction(java.util.Collection<SSTableReader> sstablesToGroup)

Method for grouping similar SSTables together, This will be used by anti-compaction to determine which SSTables should be anitcompacted as a group. If a given compaction strategy creates sstables which cannot be merged due to some constraint it must override this method.