org.apache.cassandra.db.compaction

Class CompactionStrategyManager

java.lang.Object

org.apache.cassandra.db.compaction.CompactionStrategyManager

All Implemented Interfaces:

INotificationConsumer

public class CompactionStrategyManager
extends java.lang.Object
implements INotificationConsumer

Manages the compaction strategies. SSTables are isolated from each other based on their incremental repair status (repaired, unrepaired, or pending repair) and directory (determined by their starting token). This class handles the routing between AbstractStrategyHolder instances based on repair status, and the AbstractStrategyHolder instances have separate compaction strategies for each directory, which it routes sstables to. Note that PendingRepairHolder also divides sstables on their pending repair id. Operations on this class are guarded by a ReentrantReadWriteLock. This lock performs mutual exclusion on reads and writes to the following variables: this#repaired, this#unrepaired, this#isActive, this#params, this#currentBoundaries. Whenever performing reads on these variables, the this#readLock should be acquired. Likewise, updates to these variables should be guarded by this#writeLock. Whenever the DiskBoundaries change, the compaction strategies must be reloaded, so in order to ensure the compaction strategy placement reflect most up-to-date disk boundaries, call this#maybeReloadDiskBoundaries() before acquiring the read lock to acess the strategies.

Field Summary

Fields

Modifier and Type	Field and Description
CompactionLogger	compactionLogger

Constructor Summary

Constructors

Constructor and Description
CompactionStrategyManager(ColumnFamilyStore cfs)
CompactionStrategyManager(ColumnFamilyStore cfs, java.util.function.Supplier<DiskBoundaries> boundariesSupplier, boolean partitionSSTablesByTokenRange)

Method Summary

Modifier and Type	Method and Description
SSTableMultiWriter	createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, java.util.UUID pendingRepair, boolean isTransient, MetadataCollector collector, SerializationHeader header, java.util.Collection<Index> indexes, LifecycleNewTracker lifecycleNewTracker)
void	disable()
void	enable()
CompactionParams	getCompactionParams()
AbstractCompactionStrategy	getCompactionStrategyFor(SSTableReader sstable) return the compaction strategy for the given sstable returns differently based on the repaired status and which vnode the compaction strategy belongs to
AbstractCompactionTask	getCompactionTask(LifecycleTransaction txn, int gcBefore, long maxSSTableBytes)
int	getEstimatedRemainingTasks()
int	getLevelFanoutSize()
CompactionTasks	getMaximalTasks(int gcBefore, boolean splitOutput)
long	getMaxSSTableBytes()
java.lang.String	getName()
AbstractCompactionTask	getNextBackgroundTask(int gcBefore) Return the next background task Returns a task for the compaction strategy that needs it the most (most estimated remaining tasks)
AbstractCompactionStrategy.ScannerList	getScanners(java.util.Collection<SSTableReader> sstables)
AbstractCompactionStrategy.ScannerList	getScanners(java.util.Collection<SSTableReader> sstables, java.util.Collection<Range<Token>> ranges)
int[]	getSSTableCountPerLevel()
java.util.List<java.util.List<AbstractCompactionStrategy>>	getStrategies()
java.util.List<java.lang.String>	getStrategyFolders(AbstractCompactionStrategy strategy)
int	getUnleveledSSTables()
CompactionTasks	getUserDefinedTasks(java.util.Collection<SSTableReader> sstables, int gcBefore) Return a list of compaction tasks corresponding to the sstables requested.
java.util.List<AbstractStrategyHolder.GroupedSSTableContainer>	groupSSTables(java.lang.Iterable<SSTableReader> sstables) Split sstables into a list of grouped sstable containers, the list index an sstable lives in matches the list index of the holder that's responsible for it
java.util.Collection<java.util.Collection<SSTableReader>>	groupSSTablesForAntiCompaction(java.util.Collection<SSTableReader> sstablesToGroup)
void	handleNotification(INotification notification, java.lang.Object sender)
boolean	hasDataForPendingRepair(java.util.UUID sessionID)
boolean	isActive()
boolean	isEnabled()
boolean	isRepaired(AbstractCompactionStrategy strategy)
AbstractCompactionStrategy.ScannerList	maybeGetScanners(java.util.Collection<SSTableReader> sstables, java.util.Collection<Range<Token>> ranges) Create ISSTableScanners from the given sstables Delegates the call to the compaction strategies to allow LCS to create a scanner
void	maybeReload(TableMetadata metadata)
protected void	maybeReloadDiskBoundaries() Checks if the disk boundaries changed and reloads the compaction strategies to reflect the most up-to-date disk boundaries.
void	mutateRepaired(java.util.Collection<SSTableReader> sstables, long repairedAt, java.util.UUID pendingRepair, boolean isTransient) Mutates sstable repairedAt times and notifies listeners of the change with the writeLock held.
boolean	onlyPurgeRepairedTombstones()
void	pause() pause compaction while we cancel all ongoing compactions Separate call from enable/disable to not have to save the enabled-state externally
CleanupSummary	releaseRepairData(java.util.Collection<java.util.UUID> sessions)
void	resume()
void	setNewLocalCompactionStrategy(CompactionParams params)
boolean	shouldBeEnabled()
void	shutdown()
boolean	supportsEarlyOpen()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

compactionLogger

public final CompactionLogger compactionLogger

Constructor Detail

CompactionStrategyManager

public CompactionStrategyManager(ColumnFamilyStore cfs)

CompactionStrategyManager

public CompactionStrategyManager(ColumnFamilyStore cfs,
                                 java.util.function.Supplier<DiskBoundaries> boundariesSupplier,
                                 boolean partitionSSTablesByTokenRange)

Method Detail

getNextBackgroundTask

public AbstractCompactionTask getNextBackgroundTask(int gcBefore)

Return the next background task Returns a task for the compaction strategy that needs it the most (most estimated remaining tasks)

isEnabled

public boolean isEnabled()

isActive

public boolean isActive()

resume

public void resume()

pause

public void pause()

pause compaction while we cancel all ongoing compactions Separate call from enable/disable to not have to save the enabled-state externally

getCompactionStrategyFor

public AbstractCompactionStrategy getCompactionStrategyFor(SSTableReader sstable)

return the compaction strategy for the given sstable returns differently based on the repaired status and which vnode the compaction strategy belongs to

Parameters:

sstable -

Returns:

hasDataForPendingRepair

public boolean hasDataForPendingRepair(java.util.UUID sessionID)

shutdown

public void shutdown()

maybeReload

public void maybeReload(TableMetadata metadata)

maybeReloadDiskBoundaries

protected void maybeReloadDiskBoundaries()

Checks if the disk boundaries changed and reloads the compaction strategies to reflect the most up-to-date disk boundaries. This is typically called before acquiring the this#readLock to ensure the most up-to-date disk locations and boundaries are used. This should *never* be called inside by a thread holding the this#readLock, since it will potentially acquire the this#writeLock to update the compaction strategies what can cause a deadlock.

getUnleveledSSTables

public int getUnleveledSSTables()

getLevelFanoutSize

public int getLevelFanoutSize()

getSSTableCountPerLevel

public int[] getSSTableCountPerLevel()

groupSSTables

public java.util.List<AbstractStrategyHolder.GroupedSSTableContainer> groupSSTables(java.lang.Iterable<SSTableReader> sstables)

Split sstables into a list of grouped sstable containers, the list index an sstable lives in matches the list index of the holder that's responsible for it

handleNotification

public void handleNotification(INotification notification,
                               java.lang.Object sender)

Specified by:

handleNotification in interface INotificationConsumer

enable

public void enable()

disable

public void disable()

maybeGetScanners

public AbstractCompactionStrategy.ScannerList maybeGetScanners(java.util.Collection<SSTableReader> sstables,
                                                               java.util.Collection<Range<Token>> ranges)

Create ISSTableScanners from the given sstables Delegates the call to the compaction strategies to allow LCS to create a scanner

Parameters:

sstables -

ranges -

Returns:

getScanners

public AbstractCompactionStrategy.ScannerList getScanners(java.util.Collection<SSTableReader> sstables,
                                                          java.util.Collection<Range<Token>> ranges)

getScanners

public AbstractCompactionStrategy.ScannerList getScanners(java.util.Collection<SSTableReader> sstables)

groupSSTablesForAntiCompaction

public java.util.Collection<java.util.Collection<SSTableReader>> groupSSTablesForAntiCompaction(java.util.Collection<SSTableReader> sstablesToGroup)

getMaxSSTableBytes

public long getMaxSSTableBytes()

getCompactionTask

public AbstractCompactionTask getCompactionTask(LifecycleTransaction txn,
                                                int gcBefore,
                                                long maxSSTableBytes)

getMaximalTasks

public CompactionTasks getMaximalTasks(int gcBefore,
                                       boolean splitOutput)

getUserDefinedTasks

public CompactionTasks getUserDefinedTasks(java.util.Collection<SSTableReader> sstables,
                                           int gcBefore)

Return a list of compaction tasks corresponding to the sstables requested. Split the sstables according to whether they are repaired or not, and by disk location. Return a task per disk location and repair status group.

Parameters:

sstables - the sstables to compact

gcBefore - gc grace period, throw away tombstones older than this

Returns:

a list of compaction tasks corresponding to the sstables requested

getEstimatedRemainingTasks

public int getEstimatedRemainingTasks()

shouldBeEnabled

public boolean shouldBeEnabled()

getName

public java.lang.String getName()

getStrategies

public java.util.List<java.util.List<AbstractCompactionStrategy>> getStrategies()

setNewLocalCompactionStrategy

public void setNewLocalCompactionStrategy(CompactionParams params)

getCompactionParams

public CompactionParams getCompactionParams()

onlyPurgeRepairedTombstones

public boolean onlyPurgeRepairedTombstones()

createSSTableMultiWriter

public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor,
                                                   long keyCount,
                                                   long repairedAt,
                                                   java.util.UUID pendingRepair,
                                                   boolean isTransient,
                                                   MetadataCollector collector,
                                                   SerializationHeader header,
                                                   java.util.Collection<Index> indexes,
                                                   LifecycleNewTracker lifecycleNewTracker)

isRepaired

public boolean isRepaired(AbstractCompactionStrategy strategy)

getStrategyFolders

public java.util.List<java.lang.String> getStrategyFolders(AbstractCompactionStrategy strategy)

supportsEarlyOpen

public boolean supportsEarlyOpen()

mutateRepaired

public void mutateRepaired(java.util.Collection<SSTableReader> sstables,
                           long repairedAt,
                           java.util.UUID pendingRepair,
                           boolean isTransient)
                    throws java.io.IOException

Mutates sstable repairedAt times and notifies listeners of the change with the writeLock held. Prevents races with other processes between when the metadata is changed and when sstables are moved between strategies.

Throws:

java.io.IOException

releaseRepairData

public CleanupSummary releaseRepairData(java.util.Collection<java.util.UUID> sessions)