Class UnorderedWorkAssigner
- java.lang.Object
-
- org.apache.accumulo.master.replication.DistributedWorkQueueWorkAssigner
-
- org.apache.accumulo.master.replication.UnorderedWorkAssigner
-
- All Implemented Interfaces:
WorkAssigner
public class UnorderedWorkAssigner extends DistributedWorkQueueWorkAssigner
Read work records from the replication table, create work entries for other nodes to complete.Uses the DistributedWorkQueue to make the work available for any tserver. This approach does not consider the locality of the tabletserver performing the work in relation to the data being replicated (local HDFS blocks).
The implementation allows for multiple tservers to concurrently replicate data to peer(s), however it is possible that data for a table is replayed on the peer in a different order than the master. The
SequentialWorkAssigner
should be used if this must be guaranteed at the cost of replication throughput.
-
-
Field Summary
-
Fields inherited from class org.apache.accumulo.master.replication.DistributedWorkQueueWorkAssigner
conf, conn, maxQueueSize, workQueue, zooCache
-
-
Constructor Summary
Constructors Constructor Description UnorderedWorkAssigner()
UnorderedWorkAssigner(AccumuloConfiguration conf, Connector conn)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected void
cleanupFinishedWork()
Iterate over the queued work to remove entries that have been completed.String
getName()
protected Set<String>
getQueuedWork()
protected Set<String>
getQueuedWork(ReplicationTarget target)
protected int
getQueueSize()
protected void
initializeQueuedWork()
Initialize the queuedWork set with the work already sent outprotected boolean
queueWork(org.apache.hadoop.fs.Path path, ReplicationTarget target)
Distribute the work for the given path with filenameprotected void
removeQueuedWork(ReplicationTarget target, String queueKey)
Remove the given work from the internal stateprotected void
setQueuedWork(Set<String> queuedWork)
protected boolean
shouldQueueWork(ReplicationTarget target)
-
Methods inherited from class org.apache.accumulo.master.replication.DistributedWorkQueueWorkAssigner
assignWork, configure, createWork, getConf, getConnector, getMaxQueueSize, getWorkQueue, getZooCache, initializeWorkQueue, isWorkRequired, setConf, setConnector, setMaxQueueSize, setWorkQueue, setZooCache
-
-
-
-
Constructor Detail
-
UnorderedWorkAssigner
public UnorderedWorkAssigner()
-
UnorderedWorkAssigner
public UnorderedWorkAssigner(AccumuloConfiguration conf, Connector conn)
-
-
Method Detail
-
getName
public String getName()
-
initializeQueuedWork
protected void initializeQueuedWork()
Initialize the queuedWork set with the work already sent out- Specified by:
initializeQueuedWork
in classDistributedWorkQueueWorkAssigner
-
queueWork
protected boolean queueWork(org.apache.hadoop.fs.Path path, ReplicationTarget target)
Distribute the work for the given path with filename- Specified by:
queueWork
in classDistributedWorkQueueWorkAssigner
- Parameters:
path
- Path to the file being replicatedtarget
- Target for the file to be replicated to- Returns:
- True if the work was queued, false otherwise
-
cleanupFinishedWork
protected void cleanupFinishedWork()
Iterate over the queued work to remove entries that have been completed.- Specified by:
cleanupFinishedWork
in classDistributedWorkQueueWorkAssigner
-
shouldQueueWork
protected boolean shouldQueueWork(ReplicationTarget target)
- Specified by:
shouldQueueWork
in classDistributedWorkQueueWorkAssigner
- Returns:
- Can replication work for the given
ReplicationTarget
be submitted to be worked on.
-
getQueueSize
protected int getQueueSize()
- Specified by:
getQueueSize
in classDistributedWorkQueueWorkAssigner
- Returns:
- the size of the queued work
-
getQueuedWork
protected Set<String> getQueuedWork(ReplicationTarget target)
- Specified by:
getQueuedWork
in classDistributedWorkQueueWorkAssigner
- Parameters:
target
- Target for the work- Returns:
- Queued work for the given target
-
removeQueuedWork
protected void removeQueuedWork(ReplicationTarget target, String queueKey)
Description copied from class:DistributedWorkQueueWorkAssigner
Remove the given work from the internal state- Specified by:
removeQueuedWork
in classDistributedWorkQueueWorkAssigner
-
-