Package org.apache.cassandra.db.compaction.unified

Class Controller

  • public class Controller
extends java.lang.Object
    The controller provides compaction parameters to the unified compaction strategy

    • Field Detail

      • logger

        protected static final org.slf4j.Logger logger

      • DEFAULT_BASE_SHARD_COUNT

        public static final int DEFAULT_BASE_SHARD_COUNT
        Default base shard count, used when a base count is not explicitly supplied. This value applies as long as the table is not a system one, and directories are not defined. For others a base count of 1 is used as system tables are usually small and do not need as much compaction parallelism, while having directories defined provides for parallelism in a different way.

      • DEFAULT_TARGET_SSTABLE_SIZE

        public static final long DEFAULT_TARGET_SSTABLE_SIZE

      • survivalFactors

        protected final double[] survivalFactors

      • minSSTableSize

        protected volatile long minSSTableSize

      • flushSizeOverride

        protected final long flushSizeOverride

      • currentFlushSize

        protected volatile long currentFlushSize

      • maxSSTablesToCompact

        protected final int maxSSTablesToCompact

      • expiredSSTableCheckFrequency

        protected final long expiredSSTableCheckFrequency

      • ignoreOverlapsInExpirationCheck

        protected final boolean ignoreOverlapsInExpirationCheck

      • baseShardCount

        protected final int baseShardCount

      • targetSSTableSize

        protected final double targetSSTableSize

      • sstableGrowthModifier

        protected final double sstableGrowthModifier

    • Method Detail

      • getScalingParameter

        public int getScalingParameter​(int index)
        Parameters:
        index -
        Returns:
        the scaling parameter W

      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object

      • getFanout

        public int getFanout​(int index)

      • getThreshold

        public int getThreshold​(int index)

      • getNumShards

        public int getNumShards​(double localDensity)
        Calculate the number of shards to split the local token space in for the given SSTable density. This is calculated as a power-of-two multiple of baseShardCount, so that the expected size of resulting SSTables is between sqrt(0.5) and sqrt(2) times the target size, which is calculated from targetSSTableSize to grow at the given sstableGrowthModifier of the exponential growth of the density.

        Additionally, if a minimum SSTable size is set, we can go below the baseShardCount when that would result in SSTables smaller than that minimum. Note that in the case of a non-power-of-two base count, we will only split to divisors of baseShardCount.

        Note that to get the SSTables resulting from this splitting within the bounds, the density argument must be normalized to the span that is being split. In other words, if no disks are defined, the density should be scaled by the token coverage of the locally-owned ranges. If multiple data directories are defined, the density should be scaled by the token coverage of the respective data directory. That is, localDensity = size / span, where the span is normalized so that span = 1 when the data covers the range that is being split.

      • getSurvivalFactor

        public double getSurvivalFactor​(int index)
        Parameters:
        index -
        Returns:
        the survival factor o

      • getFlushSizeBytes

        public long getFlushSizeBytes()
        Return the flush sstable size in bytes. This is usually obtained from the observed sstable flush sizes, refreshed when it differs significantly from the current values. It can also be set by the user in the options.
        Returns:
        the flush size in bytes.

      • getIgnoreOverlapsInExpirationCheck

        public boolean getIgnoreOverlapsInExpirationCheck()
        Returns:
        whether is allowed to drop expired SSTables without checking if partition keys appear in other SSTables. Same behavior as in TWCS.

      • getExpiredSSTableCheckFrequency

        public long getExpiredSSTableCheckFrequency()

      • fromOptions

        public static Controller fromOptions​(ColumnFamilyStore cfs,
                                     java.util.Map<java.lang.String,​java.lang.String> options)

      • validateOptions

        public static java.util.Map<java.lang.String,​java.lang.String> validateOptions​(java.util.Map<java.lang.String,​java.lang.String> options)
                                                                              throws ConfigurationException
        Throws:
        ConfigurationException

      • getBaseSstableSize

        public double getBaseSstableSize​(int F)

      • getMaxLevelDensity

        public double getMaxLevelDensity​(int index,
                                 double minSize)

      • maxThroughput

        public double maxThroughput()

      • maxConcurrentCompactions

        public int maxConcurrentCompactions()

      • maxSSTablesToCompact

        public int maxSSTablesToCompact()

      • random

        public java.util.Random random()
        Random number generator to be used for the selection of tasks. Replaced by some tests.

      • overlapInclusionMethod

        public Overlaps.InclusionMethod overlapInclusionMethod()
        Return the overlap inclusion method to use when combining overlap sections into a bucket. For example, with SSTables A(0, 5), B(2, 9), C(6, 12), D(10, 12) whose overlap sections calculation returns [AB, BC, CD], - NONE means no sections are to be merged. AB, BC and CD will be separate buckets, compactions AB, BC and CD will be added separately, thus some SSTables will be partially used / single-source compacted, likely to be recompacted again with the next selected bucket. - SINGLE means only overlaps of the sstables in the selected bucket will be added. AB+BC will be one bucket, and CD will be another (as BC is already used). A middle ground of sorts, should reduce overcompaction but still has some. - TRANSITIVE means a transitive closure of overlapping sstables will be selected. AB+BC+CD will be in the same bucket, selected compactions will apply to all overlapping sstables and no overcompaction will be done, at the cost of reduced compaction parallelism and increased length of the operation. TRANSITIVE is the default and makes most sense. NONE is a closer approximation to operation of legacy UCS. The option is exposed for experimentation.

      • parseScalingParameters

        public static int[] parseScalingParameters​(java.lang.String str)

      • printScalingParameters

        public static java.lang.String printScalingParameters​(int[] parameters)