Package org.apache.cassandra.db.compaction

Class CompactionStrategyManager

  • All Implemented Interfaces:
    INotificationConsumer
    public class CompactionStrategyManager
extends java.lang.Object
implements INotificationConsumer
    Manages the compaction strategies. SSTables are isolated from each other based on their incremental repair status (repaired, unrepaired, or pending repair) and directory (determined by their starting token). This class handles the routing between AbstractStrategyHolder instances based on repair status, and the AbstractStrategyHolder instances have separate compaction strategies for each directory, which it routes sstables to. Note that PendingRepairHolder also divides sstables on their pending repair id. Operations on this class are guarded by a ReentrantReadWriteLock. This lock performs mutual exclusion on reads and writes to the following variables: {@link this#repaired}, {@link this#unrepaired}, {@link this#isActive}, {@link this#params}, {@link this#currentBoundaries}. Whenever performing reads on these variables, the {@link this#readLock} should be acquired. Likewise, updates to these variables should be guarded by {@link this#writeLock}. Whenever the DiskBoundaries change, the compaction strategies must be reloaded, so in order to ensure the compaction strategy placement reflect most up-to-date disk boundaries, call {@link this#maybeReloadDiskBoundaries()} before acquiring the read lock to acess the strategies.

    • Field Detail

      • TWCS_BUCKET_COUNT_MAX

        public static int TWCS_BUCKET_COUNT_MAX

    • Constructor Detail

      • CompactionStrategyManager

        public CompactionStrategyManager​(ColumnFamilyStore cfs)

      • CompactionStrategyManager

        public CompactionStrategyManager​(ColumnFamilyStore cfs,
                                 java.util.function.Supplier<DiskBoundaries> boundariesSupplier,
                                 boolean partitionSSTablesByTokenRange)

    • Method Detail

      • getNextBackgroundTask

        public AbstractCompactionTask getNextBackgroundTask​(long gcBefore)
        Return the next background task Returns a task for the compaction strategy that needs it the most (most estimated remaining tasks)

      • isEnabled

        public boolean isEnabled()

      • isActive

        public boolean isActive()

      • resume

        public void resume()

      • pause

        public void pause()
        pause compaction while we cancel all ongoing compactions Separate call from enable/disable to not have to save the enabled-state externally

      • getCompactionStrategyFor

        public AbstractCompactionStrategy getCompactionStrategyFor​(SSTableReader sstable)
        return the compaction strategy for the given sstable returns differently based on the repaired status and which vnode the compaction strategy belongs to
        Parameters:
        sstable -
        Returns:

      • hasDataForPendingRepair

        public boolean hasDataForPendingRepair​(TimeUUID sessionID)

      • hasPendingRepairSSTable

        public boolean hasPendingRepairSSTable​(TimeUUID sessionID,
                                       SSTableReader sstable)

      • shutdown

        public void shutdown()

      • maybeReloadParamsFromSchema

        public void maybeReloadParamsFromSchema​(CompactionParams params)
        Maybe reload the compaction strategies. Called after changing configuration.

      • maybeReloadDiskBoundaries

        protected void maybeReloadDiskBoundaries()
        Checks if the disk boundaries changed and reloads the compaction strategies to reflect the most up-to-date disk boundaries.

        This is typically called before acquiring the {@link this#readLock} to ensure the most up-to-date disk locations and boundaries are used.

        This should *never* be called inside by a thread holding the {@link this#readLock}, since it will potentially acquire the {@link this#writeLock} to update the compaction strategies what can cause a deadlock.

        TODO: improve this to reload after receiving a notification rather than trying to reload on every operation

      • getUnleveledSSTables

        public int getUnleveledSSTables()

      • getLevelFanoutSize

        public int getLevelFanoutSize()

      • getSSTableCountPerLevel

        public int[] getSSTableCountPerLevel()

      • getPerLevelSizeBytes

        public long[] getPerLevelSizeBytes()

      • isLeveledCompaction

        public boolean isLeveledCompaction()

      • getSSTableCountPerTWCSBucket

        public int[] getSSTableCountPerTWCSBucket()

      • groupSSTables

        public java.util.List<AbstractStrategyHolder.GroupedSSTableContainer> groupSSTables​(java.lang.Iterable<SSTableReader> sstables)
        Split sstables into a list of grouped sstable containers, the list index an sstable lives in matches the list index of the holder that's responsible for it

      • enable

        public void enable()

      • disable

        public void disable()

      • maybeGetScanners

        public AbstractCompactionStrategy.ScannerList maybeGetScanners​(java.util.Collection<SSTableReader> sstables,
                                                               java.util.Collection<Range<Token>> ranges)
        Create ISSTableScanners from the given sstables Delegates the call to the compaction strategies to allow LCS to create a scanner
        Parameters:
        sstables -
        ranges -
        Returns:

      • groupSSTablesForAntiCompaction

        public java.util.Collection<java.util.Collection<SSTableReader>> groupSSTablesForAntiCompaction​(java.util.Collection<SSTableReader> sstablesToGroup)

      • getMaxSSTableBytes

        public long getMaxSSTableBytes()

      • getUserDefinedTasks

        public CompactionTasks getUserDefinedTasks​(java.util.Collection<SSTableReader> sstables,
                                           long gcBefore)
        Return a list of compaction tasks corresponding to the sstables requested. Split the sstables according to whether they are repaired or not, and by disk location. Return a task per disk location and repair status group.
        Parameters:
        sstables - the sstables to compact
        gcBefore - gc grace period, throw away tombstones older than this
        Returns:
        a list of compaction tasks corresponding to the sstables requested

      • getEstimatedRemainingTasks

        public int getEstimatedRemainingTasks()

      • getEstimatedRemainingTasks

        public int getEstimatedRemainingTasks​(int additionalSSTables,
                                      long additionalBytes,
                                      boolean isIncremental)

      • shouldBeEnabled

        public boolean shouldBeEnabled()

      • getName

        public java.lang.String getName()

      • overrideLocalParams

        public void overrideLocalParams​(CompactionParams params)

      • onlyPurgeRepairedTombstones

        public boolean onlyPurgeRepairedTombstones()

      • supportsEarlyOpen

        public boolean supportsEarlyOpen()

      • mutateRepaired

        public void mutateRepaired​(java.util.Collection<SSTableReader> sstables,
                           long repairedAt,
                           TimeUUID pendingRepair,
                           boolean isTransient)
                    throws java.io.IOException
        Mutates sstable repairedAt times and notifies listeners of the change with the writeLock held. Prevents races with other processes between when the metadata is changed and when sstables are moved between strategies.
        Throws:
        java.io.IOException