- All Implemented Interfaces:
- java.lang.Iterable<AbstractCompactedRow>
public class ParallelCompactionIterable
extends AbstractCompactionIterable
A class to run compaction taking advantage of multiple-core processes:
One Deserializer thread per input sstable performs read + deserialize (a row at a time).
The resulting ColumnFamilies are added to a queue, which is fed to the merge Reducer.
The merge Reducer creates MergeTasks on a thread-per-core Executor, and returns AsyncPrecompactedRow objects.
The main complication is in handling larger-than-memory rows. When one is encountered, no further deserialization
is done until that row is merged and written -- creating a pipeline stall, as it were. Thus, this is intended
to be useful with mostly-in-memory row sizes, but preserves correctness in the face of occasional exceptions.