Packages

object CompactibleFileStreamLog

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. CompactibleFileStreamLog
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. val COMPACT_FILE_SUFFIX: String
  2. val COMPACT_LATENCY_WARN_THRESHOLD_MS: Int
  3. def deriveCompactInterval(defaultInterval: Int, latestCompactBatchId: Int): Int

    Derives a compact interval from the latest compact batch id and a default compact interval.

  4. def getAllValidBatches(batchId: Long, compactInterval: Long): Seq[Long]

    Returns all necessary logs before batchId (inclusive).

    Returns all necessary logs before batchId (inclusive). If batchId is a compaction, just return itself. Otherwise, it will find the previous compaction batch and return all batches between it and batchId.

  5. def getBatchIdFromFileName(fileName: String): Long
  6. def getValidBatchesBeforeCompactionBatch(compactionBatchId: Long, compactInterval: Int): Seq[Long]

    Returns all valid batches before the specified compactionBatchId.

    Returns all valid batches before the specified compactionBatchId. They contain all logs we need to do a new compaction.

    E.g., if compactInterval is 3 and compactionBatchId is 5, this method should returns Seq(2, 3, 4) (Note: it includes the previous compaction batch 2).

  7. def isCompactionBatch(batchId: Long, compactInterval: Int): Boolean

    Returns if this is a compaction batch.

    Returns if this is a compaction batch. FileStreamSinkLog will compact old logs every compactInterval commits.

    E.g., if compactInterval is 3, then 2, 5, 8, ... are all compaction batches.

  8. def nextCompactionBatchId(batchId: Long, compactInterval: Long): Long

    Returns the next compaction batch id after batchId.