Class DynamicThrottlePolicy
- java.lang.Object
-
- com.yahoo.messagebus.StaticThrottlePolicy
-
- com.yahoo.messagebus.DynamicThrottlePolicy
-
- All Implemented Interfaces:
ThrottlePolicy
public class DynamicThrottlePolicy extends StaticThrottlePolicy
This is an implementation of theThrottlePolicy
that offers dynamic limits to the number of pending messages aSourceSession
is allowed to have. Pending means the number of sent messages that have not been replied to yet.The algorithm works by increasing the number of messages allowed to be pending, the window size, until this no longer increases throughput. At this point, the algorithm is driven by synthetic attraction towards latencies which satisfy
log10(1 / latency) % 1 = e
, for some constant0 < e < 1
. Weird? Most certainly!The effect is that the window size the algorithm produces, for a saturated ideal server, has a level for each power of ten with an attractor the window size tends towards while on this level, determined by the
e
above. ThesetEfficiencyThreshold(double)
determines thee
constant. Whene
is set so the attractor is close to the start of the interval, this has an inhibiting effect on the algorithm, and it is basically reduced to "increase window size until this no longer increases throughput enough that it defeats the random noise". As the attractor moves towards the end of the intervals, the algorithm becomes increasingly eager in increasing its window size past what it measures as effective — if moved to the very end of the interval, the algorithm explodes. The default setting has the attractor atlog10(2)
of the way from start to end of these levels.Because the algorithm stops increasing the window size when increases in throughput drown in random variations, the
setWindowSizeIncrement(double)
directly influences the efficient work domain of the algorithm. With the default setting of20
, it seems to struggle to increase past window sizes of a couple thousand. Using a static increment (and a backoff factor) seems to be necessary to effectively balance the load different, competing policies are allowed to produce.When there are multiple policies that compete against each other, they will increase load until saturating the server. If keeping all other policies but one fixed, this one policy would still see an increase in throughput with increased window size, even with a saturated server, as it would be given a greater share of the server's resources. However, since all algorithms adjust their windows concurrently, they will all reduce the throughput of the other algorithms. The result is that the commonly see the server as saturated, and commonly reach the behaviour where some increases in window size lead to measurable throughput gains, while others don't.
Now the weighting (
setWeight(double)
comes into play: with equals weights, two algorithms would have a break-even between being governed by the attractors (above), which eventually limits window size, and increases due to positive measurements, at the same point along the window size axis. With smaller weights, i.e., smaller increases to window size, this break-even occurs where the curve is steeper, i.e., where the client has a smaller share of the server. Thus, competing algorithms with different weights end up with a resource distribution roughly proportional to weight.- Author:
- Simon Thoresen Hult, jonmv
-
-
Constructor Summary
Constructors Constructor Description DynamicThrottlePolicy()
Constructs a new instance of this policy and sets the appropriate default values of member data.DynamicThrottlePolicy(com.yahoo.concurrent.Timer timer)
Constructs a new instance of this class using the given clock to calculate efficiency.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description boolean
canSend(Message message, int pendingCount)
Returns whether or not the given message can be sent according to the current state of this policy.int
getMaxPendingCount()
Returns the maximum number of pending messages allowed.double
getMaxWindowSize()
Get the maximum number of pending operations allowed at any time.double
getMinWindowSize()
Get the minimum number of pending operations allowed at any time.double
getWindowSizeBackOff()
double
getWindowSizeIncrement()
void
processMessage(Message message)
This method is called once for every message that was accepted byThrottlePolicy.canSend(Message, int)
and sent.void
processReply(Reply reply)
This method is called once for every reply that is received.DynamicThrottlePolicy
setEfficiencyThreshold(double efficiencyThreshold)
Determines where on each latency level the attractor sits.DynamicThrottlePolicy
setMaxPendingCount(int maxCount)
Sets the maximum number of pending messages allowed.DynamicThrottlePolicy
setMaxThroughput(double maxThroughput)
DynamicThrottlePolicy
setMaxWindowSize(double max)
Sets the maximum number of pending operations allowed at any time, in order to avoid using too much resources.DynamicThrottlePolicy
setMinWindowSize(double min)
Sets the minimum number of pending operations allowed at any time, in order to keep a level of performance.DynamicThrottlePolicy
setResizeRate(double resizeRate)
Sets the rate at which the window size is updated.DynamicThrottlePolicy
setWeight(double weight)
Sets the weight for this client.DynamicThrottlePolicy
setWindowSizeBackOff(double windowSizeBackOff)
Sets the factor of window size to back off to when the algorithm determines that efficiency is not increasing.DynamicThrottlePolicy
setWindowSizeDecrementFactor(double decrementFactor)
Sets the relative step size when decreasing window size.DynamicThrottlePolicy
setWindowSizeIncrement(double windowSizeIncrement)
Sets the step size used when increasing window size.-
Methods inherited from class com.yahoo.messagebus.StaticThrottlePolicy
getMaxPendingSize, getPendingSize, setMaxPendingSize
-
-
-
-
Constructor Detail
-
DynamicThrottlePolicy
public DynamicThrottlePolicy()
Constructs a new instance of this policy and sets the appropriate default values of member data.
-
DynamicThrottlePolicy
public DynamicThrottlePolicy(com.yahoo.concurrent.Timer timer)
Constructs a new instance of this class using the given clock to calculate efficiency.- Parameters:
timer
- the timer to use
-
-
Method Detail
-
getWindowSizeIncrement
public double getWindowSizeIncrement()
-
getWindowSizeBackOff
public double getWindowSizeBackOff()
-
setMaxThroughput
public DynamicThrottlePolicy setMaxThroughput(double maxThroughput)
-
canSend
public boolean canSend(Message message, int pendingCount)
Description copied from interface:ThrottlePolicy
Returns whether or not the given message can be sent according to the current state of this policy.- Specified by:
canSend
in interfaceThrottlePolicy
- Overrides:
canSend
in classStaticThrottlePolicy
- Parameters:
message
- the message to evaluatependingCount
- the current number of pending messages- Returns:
- true to send the message
-
processMessage
public void processMessage(Message message)
Description copied from interface:ThrottlePolicy
This method is called once for every message that was accepted byThrottlePolicy.canSend(Message, int)
and sent.- Specified by:
processMessage
in interfaceThrottlePolicy
- Overrides:
processMessage
in classStaticThrottlePolicy
- Parameters:
message
- the message being sent
-
processReply
public void processReply(Reply reply)
Description copied from interface:ThrottlePolicy
This method is called once for every reply that is received.- Specified by:
processReply
in interfaceThrottlePolicy
- Overrides:
processReply
in classStaticThrottlePolicy
- Parameters:
reply
- the reply received
-
setEfficiencyThreshold
public DynamicThrottlePolicy setEfficiencyThreshold(double efficiencyThreshold)
Determines where on each latency level the attractor sits. 2 is at the very end, and makes this to *boom*. 0.2 is at the very start, and makes the algorithm more conservative. Probably fine to stay away from this.
-
setWindowSizeIncrement
public DynamicThrottlePolicy setWindowSizeIncrement(double windowSizeIncrement)
Sets the step size used when increasing window size.- Parameters:
windowSizeIncrement
- the step size to set- Returns:
- this, to allow chaining
-
setWindowSizeDecrementFactor
public DynamicThrottlePolicy setWindowSizeDecrementFactor(double decrementFactor)
Sets the relative step size when decreasing window size.- Parameters:
decrementFactor
- the step size to set- Returns:
- this, to allow chaining
-
setWindowSizeBackOff
public DynamicThrottlePolicy setWindowSizeBackOff(double windowSizeBackOff)
Sets the factor of window size to back off to when the algorithm determines that efficiency is not increasing. Capped to [0, 1]- Parameters:
windowSizeBackOff
- the back off to set- Returns:
- this, to allow chaining
-
setResizeRate
public DynamicThrottlePolicy setResizeRate(double resizeRate)
Sets the rate at which the window size is updated. The larger the value, the less responsive the resizing becomes. However, the smaller the value, the less accurate the measurements become. Capped to [2, )- Parameters:
resizeRate
- the rate to set- Returns:
- this, to allow chaining
-
setWeight
public DynamicThrottlePolicy setWeight(double weight)
Sets the weight for this client. The larger the value, the more resources will be allocated to this clients. Resources are shared between clients roughly proportionally to the set weights. Must be a positive number.- Parameters:
weight
- the weight to set- Returns:
- this, to allow chaining
-
setMaxWindowSize
public DynamicThrottlePolicy setMaxWindowSize(double max)
Sets the maximum number of pending operations allowed at any time, in order to avoid using too much resources.- Parameters:
max
- the max to set- Returns:
- this, to allow chaining
-
getMaxWindowSize
public double getMaxWindowSize()
Get the maximum number of pending operations allowed at any time.- Returns:
- the maximum number of operations
-
setMinWindowSize
public DynamicThrottlePolicy setMinWindowSize(double min)
Sets the minimum number of pending operations allowed at any time, in order to keep a level of performance.- Parameters:
min
- the min to set- Returns:
- this, to allow chaining
-
getMinWindowSize
public double getMinWindowSize()
Get the minimum number of pending operations allowed at any time.- Returns:
- the minimum number of operations
-
setMaxPendingCount
public DynamicThrottlePolicy setMaxPendingCount(int maxCount)
Description copied from class:StaticThrottlePolicy
Sets the maximum number of pending messages allowed.- Overrides:
setMaxPendingCount
in classStaticThrottlePolicy
- Parameters:
maxCount
- The max count- Returns:
- this, to allow chaining
-
getMaxPendingCount
public int getMaxPendingCount()
Returns the maximum number of pending messages allowed.- Overrides:
getMaxPendingCount
in classStaticThrottlePolicy
- Returns:
- the max limit
-
-