Package org.apache.accumulo.tserver.compaction

Class MajorCompactionRequest

java.lang.Object

org.apache.accumulo.tserver.compaction.MajorCompactionRequest

All Implemented Interfaces:

Cloneable

public class MajorCompactionRequest
extends Object
implements Cloneable

Information that can be used to determine how a tablet is to be major compacted, if needed.

Constructor Summary

Constructors

Constructor	Description
MajorCompactionRequest(KeyExtent extent, MajorCompactionReason reason, AccumuloConfiguration tabletConfig, ServerContext context)
MajorCompactionRequest(KeyExtent extent, MajorCompactionReason reason, VolumeManager manager, AccumuloConfiguration tabletConfig, BlockCache summaryCache, BlockCache indexCache, com.google.common.cache.Cache<String,Long> fileLenCache, ServerContext context)
MajorCompactionRequest(MajorCompactionRequest mcr)

Method Summary

Modifier and Type	Method	Description
MajorCompactionRequest	clone()
Map<FileRef,DataFileValue>	getFiles()
int	getMaxFilesPerTablet()
MajorCompactionReason	getReason()
List<Summary>	getSummaries(Collection<FileRef> files, Predicate<SummarizerConfiguration> summarySelector)	Returns all summaries present in each file.
String	getTableConfig(String key)
Map<String,String>	getTableProperties()
TabletId	getTabletId()
FileSKVIterator	openReader(FileRef ref)
void	setFiles(Map<FileRef,DataFileValue> update)

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MajorCompactionRequest

public MajorCompactionRequest(KeyExtent extent,
                              MajorCompactionReason reason,
                              VolumeManager manager,
                              AccumuloConfiguration tabletConfig,
                              BlockCache summaryCache,
                              BlockCache indexCache,
                              com.google.common.cache.Cache<String,Long> fileLenCache,
                              ServerContext context)

MajorCompactionRequest

public MajorCompactionRequest(KeyExtent extent,
                              MajorCompactionReason reason,
                              AccumuloConfiguration tabletConfig,
                              ServerContext context)

MajorCompactionRequest

public MajorCompactionRequest(MajorCompactionRequest mcr)

Method Detail

getTabletId

public TabletId getTabletId()

getReason

public MajorCompactionReason getReason()

getFiles

public Map<FileRef,DataFileValue> getFiles()

getSummaries

public List<Summary> getSummaries(Collection<FileRef> files,
                                  Predicate<SummarizerConfiguration> summarySelector)

Returns all summaries present in each file.

This method can only be called from CompactionStrategy.gatherInformation(MajorCompactionRequest). Unfortunately, gatherInformation() is not called before CompactionStrategy.shouldCompact(MajorCompactionRequest). Therefore shouldCompact() should just return true when a compactions strategy wants to use summary information.

When using summaries to make compaction decisions, its important to ensure that all summary data fits in the tablet server summary cache. The size of this cache is configured by code tserver.cache.summary.size}. Also its important to use the summarySelector predicate to only retrieve the needed summary data. Otherwise uneeded summary data could be brought into the cache.

Some files may contain data outside of a tablets range. When Summarizer's generate small amounts of summary data, multiple summaries may be stored within a file for different row ranges. This will allow more accurate summaries to be returned for the case where a file has data outside a tablets range. However, some summary data outside of the tablets range may still be included. When this happens Summary.FileStatistics.getExtra() will be non zero. Also, its good to be aware of the other potential causes of inaccuracies Summary.FileStatistics.getInaccurate()

When this method is called with multiple files, it will automatically merge summary data using Summarizer.Combiner.merge(Map, Map). If summary information is needed for each file, then just call this method for each file.

Writing a compaction strategy that uses summary information is a bit tricky. See the source code for TooManyDeletesCompactionStrategy as an example of a compaction strategy.

See Also:

Summarizer, TableOperations.addSummarizers(String, SummarizerConfiguration...), RFile.WriterOptions.withSummarizers(SummarizerConfiguration...)

setFiles

public void setFiles(Map<FileRef,DataFileValue> update)

openReader

public FileSKVIterator openReader(FileRef ref)
                           throws IOException

Throws:

IOException

getTableProperties

public Map<String,String> getTableProperties()

getTableConfig

public String getTableConfig(String key)

getMaxFilesPerTablet

public int getMaxFilesPerTablet()

clone

public MajorCompactionRequest clone()
                             throws CloneNotSupportedException

Overrides:

clone in class Object

Throws:

CloneNotSupportedException