ox.resilience

Members list

Type members

Classlikes

Implements "adaptive" retries: every retry costs failureCost tokens from the bucket, and every success causes successReward tokens to be added to the bucket. If there are not enough tokens, retry is not attempted.

This way retries don't overload a system that is down due to a systemic failure (such as a bug in the code, excessive load etc.): retries will be attempted only as long as there are enough tokens in the bucket, then the load on the downstream system will be reduced so that it can recover. For transient failures (component failure, infrastructure issues etc.), retries still work as expected, as the bucket has enough tokens to cover the cost of multiple retries.

Instances of this class are thread-safe and are designed to be shared. Typically, a single instance should be used to proxy access to a single constrained resource.

An instance with default parameters can be created using AdaptiveRetry.default.

Inspired by:

AdaptiveRetryStrategy from aws-sdk-java-v2
"Try again: The tools and techniques behind resilient systems" from re:Invent 2024

Value parameters

failureCost: Number of tokens to take from tokenBucket when retrying.
successReward: Number of tokens to add back to tokenBucket after a successful operation.
tokenBucket: Instance of TokenBucket. As a token bucket is thread-safe, it can be shared between different instances of AdaptiveRetry, e.g. with a different failureCost.

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: AdaptiveRetry.type

Circuit Breaker. Operations can be dropped, when the breaker is open or if it doesn't take more operation in halfOpen state. The Circuit Breaker might calculate different metrics based on SlidingWindow provided in config. See SlidingWindow for more details.

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: class Object

trait Matchable

class Any
Self type: CircuitBreaker.type

Value parameters

failureRateThreshold: threshold, as percentage of operations that ended in failure
halfOpenTimeoutDuration: time out after which, if not enough calls where registered in half open state, breaker will go back to open state.
minimumNumberOfCalls: minimum number of results that must be registered before metrics are calculated.
numberOfCallsInHalfOpenState: number of results that must be registered to calculate metrics and decide if breaker should go back to open state or close. This is also maximum number of operations that can be started in half open state.
slidingWindow: configures how thresholds will be calculated. See SlidingWindow for more details.
slowCallDurationThreshold: time after which operation is considered slow.
slowCallThreshold: threshold, as percentage of operations that spanned more then slowCallDurationThreshold.
waitDurationOpenState: how much time will pass before breaker will switch from open to half open state.

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: CircuitBreakerConfig.type

Algorithms, which take into account the entire duration of the operation.

There is no leakyBucket algorithm implemented, which is present in StartTimeRateLimiterAlgorithm, because effectively it would result in "max number of operations currently running", which can be achieved with single semaphore.

Attributes

Supertypes: class Object

trait Matchable

class Any
Self type: DurationRateLimiterAlgorithm.type

Attributes

Supertypes: class Object

trait Matchable

class Any
Self type: PercentageThreshold.type

Rate limiter with a customizable algorithm. Operations can be blocked or dropped, when the rate limit is reached. The rate limiter might take into account the start time of the operation, or its entire duration.

Attributes

Companion: object
Supertypes: class Object

trait Matchable

class Any

Attributes

Companion: class
Supertypes: class Object

trait Matchable

class Any
Self type: RateLimiter.type

Determines the algorithm to use for the rate limiter

Attributes

Supertypes: class Object

trait Matchable

class Any
Known subtypes: class FixedWindow

class SlidingWindow

class FixedWindow

class LeakyBucket

class SlidingWindow

A policy that allows to customize when a non-erroneous result is considered successful and when an error is worth retrying (which allows for failing fast on certain errors).

Type parameters

E: The error type of the operation. For operations returning a T or a Try[T], this is fixed to Throwable. For operations returning an Either[E, T], this can be any E.
T: The successful result type for the operation.

Value parameters

isSuccess: A function that determines whether a non-erroneous result is considered successful. By default, every non-erroneous result is considered successful.
isWorthRetrying: A function that determines whether an error is worth retrying. By default, all errors are retried.

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: ResultPolicy.type

A config that defines how to retry a failed operation.

It is a special case of ScheduledConfig with ScheduledConfig.sleepMode always set to SleepMode.Delay

Type parameters

E: The error type of the operation. For operations returning a T or a Try[T], this is fixed to Throwable. For operations returning an Either[E, T], this can be any E.
T: The successful result type for the operation.

Value parameters

onRetry: A function that is invoked after each retry attempt. The callback receives the number of the current retry attempt (starting from 1) and the result of the operation that was attempted. The result is either a successful value or an error. The callback can be used to log information about the retry attempts, or to perform other side effects. By default, the callback does nothing.
resultPolicy: A policy that allows to customize when a non-erroneous result is considered successful and when an error is worth retrying (which allows for failing fast on certain errors). See ResultPolicy for more details.
schedule: The retry schedule which determines the maximum number of retries and the delay between subsequent attempts to execute the operation. See Schedule for more details.

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: RetryConfig.type

Allows to configure how Metrics will be calculated

Attributes

Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Algorithms, which take into account the start time of the operation only.

Attributes

Supertypes: class Object

trait Matchable

class Any
Self type: StartTimeRateLimiterAlgorithm.type

Used by the leaky bucket rate limiter & AdaptiveRetry, to limit the rate of operations.

Attributes

Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Types

Type representing percentage threshold between 0 and 100

Attributes

Value members

Concrete methods

Retries an operation returning a direct result until it succeeds or the config decides to stop.

retry is a special case of scheduled with a given set of defaults. See RetryConfig.

Value parameters

config: The retry config - see RetryConfig.
operation: The operation to retry.

Attributes

Returns: The result of the function if it eventually succeeds.
Throws: anything
The exception thrown by the last attempt if the config decides to stop.
See also: scheduled

Retries an operation returning an scala.util.Either until it succeeds or the config decides to stop. Note that any exceptions thrown by the operation aren't caught and don't cause a retry to happen.

retryEither is a special case of scheduledEither with a given set of defaults. See RetryConfig for more details.

Value parameters

config: The retry config - see RetryConfig.
operation: The operation to retry.

Attributes

Returns: A scala.util.Right if the function eventually succeeds, or, otherwise, a scala.util.Left with the error from the last attempt.
See also: scheduledEither

Retries an operation using the given error mode until it succeeds or the config decides to stop. Note that any exceptions thrown by the operation aren't caught (unless the operation catches them as part of its implementation) and don't cause a retry to happen.

retryWithErrorMode is a special case of scheduledWithErrorMode with a given set of defaults. See RetryConfig for more details.

Value parameters

config: The retry config - see RetryConfig.
em: The error mode to use, which specifies when a result value is considered success, and when a failure.
operation: The operation to retry.

Attributes

Returns

Either:

the result of the function if it eventually succeeds, in the context of F, as dictated by the error mode.
the error E in context F as returned by the last attempt if the config decides to stop.

See also

scheduledWithErrorMode

Extensions

extension (c: PercentageThreshold)

In this article

Generated with