Apache Pekko
1.0.2
API
Generated with
Apache Pekko/org.apache.pekko.cluster.ddata

org.apache.pekko.cluster.ddata

package org.apache.pekko.cluster.ddata

Members list

Packages

package org.apache.pekko.cluster.ddata.protobuf

Type members

Classlikes

abstract class AbstractDeltaReplicatedData[A <: AbstractDeltaReplicatedData[A, B], B <: ReplicatedDelta] extends AbstractReplicatedData[A], DeltaReplicatedData

Java API: Interface for implementing a DeltaReplicatedData in Java.

Java API: Interface for implementing a DeltaReplicatedData in Java.

The type parameter A is a self-recursive type to be defined by the concrete implementation. E.g. class TwoPhaseSet extends AbstractDeltaReplicatedData&lt;TwoPhaseSet, TwoPhaseSet&gt;

Attributes

Source
ReplicatedData.scala
Supertypes
class Object
trait Matchable
class Any
Show all
abstract class AbstractReplicatedData[A <: AbstractReplicatedData[A]] extends ReplicatedData

Java API: Interface for implementing a ReplicatedData in Java.

Java API: Interface for implementing a ReplicatedData in Java.

The type parameter A is a self-recursive type to be defined by the concrete implementation. E.g. class TwoPhaseSet extends AbstractReplicatedData&lt;TwoPhaseSet&gt;

Attributes

Source
ReplicatedData.scala
Supertypes
class Object
trait Matchable
class Any
Known subtypes
trait DeltaReplicatedData extends ReplicatedData

ReplicatedData with additional support for delta-CRDT replication. delta-CRDT is a way to reduce the need for sending the full state for updates. For example adding element 'c' and 'd' to set {'a', 'b'} would result in sending the delta {'c', 'd'} and merge that with the state on the receiving side, resulting in set {'a', 'b', 'c', 'd'}.

ReplicatedData with additional support for delta-CRDT replication. delta-CRDT is a way to reduce the need for sending the full state for updates. For example adding element 'c' and 'd' to set {'a', 'b'} would result in sending the delta {'c', 'd'} and merge that with the state on the receiving side, resulting in set {'a', 'b', 'c', 'd'}.

Learn more about this in the paper Delta State Replicated Data Types.

Attributes

Source
ReplicatedData.scala
Supertypes
class Object
trait Matchable
class Any
Known subtypes
class GCounter
class GSet[A]
class LWWMap[A, B]
class ORMap[A, B]
class ORMultiMap[A, B]
class ORSet[A]
class PNCounter
class PNCounterMap[A]
Show all
object DistributedData extends ExtensionId[DistributedData], ExtensionIdProvider

Attributes

Companion
class
Source
DistributedData.scala
Supertypes
class Object
trait Matchable
class Any
Self type
class DistributedData(system: ExtendedActorSystem) extends Extension

Pekko extension for convenient configuration and use of the Replicator. Configuration settings are defined in the pekko.cluster.ddata section, see reference.conf.

Pekko extension for convenient configuration and use of the Replicator. Configuration settings are defined in the pekko.cluster.ddata section, see reference.conf.

Attributes

Companion
object
Source
DistributedData.scala
Supertypes
trait Extension
class Object
trait Matchable
class Any
object DurableStore

An actor implementing the durable store for the Distributed Data Replicator has to implement the protocol with the messages defined here.

An actor implementing the durable store for the Distributed Data Replicator has to implement the protocol with the messages defined here.

At startup the Replicator creates the durable store actor and sends the Load message to it. It must then reply with 0 or more LoadData messages followed by one LoadAllCompleted message to the sender (the Replicator).

If the LoadAll fails it can throw LoadFailed and the Replicator supervisor will stop itself and the durable store.

When the Replicator needs to store a value it sends a Store message to the durable store actor, which must then reply with the successMsg or failureMsg to the replyTo.

Attributes

Source
DurableStore.scala
Supertypes
class Object
trait Matchable
class Any
Self type
object Flag

Attributes

Companion
class
Source
Flag.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
Flag.type
final case class Flag(enabled: Boolean) extends ReplicatedData, ReplicatedDataSerialization

Implements a boolean flag CRDT that is initialized to false and can be switched to true. true wins over false in merge.

Implements a boolean flag CRDT that is initialized to false and can be switched to true. true wins over false in merge.

This class is immutable, i.e. "modifying" methods return a new instance.

Attributes

Companion
object
Source
Flag.scala
Supertypes
trait Product
trait Equals
trait Serializable
class Object
trait Matchable
class Any
Show all
object FlagKey

Attributes

Companion
class
Source
Flag.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
FlagKey.type
final case class FlagKey(_id: String) extends Key[Flag], ReplicatedDataSerialization

Attributes

Companion
object
Source
Flag.scala
Supertypes
trait Product
trait Equals
class Key[Flag]
trait Serializable
class Object
trait Matchable
class Any
Show all
object GCounter

Attributes

Companion
class
Source
GCounter.scala
Supertypes
class Object
trait Matchable
class Any
Self type
GCounter.type
final class GCounter extends DeltaReplicatedData, ReplicatedDelta, ReplicatedDataSerialization, RemovedNodePruning

Implements a 'Growing Counter' CRDT, also called a 'G-Counter'.

Implements a 'Growing Counter' CRDT, also called a 'G-Counter'.

It is described in the paper A comprehensive study of Convergent and Commutative Replicated Data Types.

A G-Counter is a increment-only counter (inspired by vector clocks) in which only increment and merge are possible. Incrementing the counter adds 1 to the count for the current node. Divergent histories are resolved by taking the maximum count for each node (like a vector clock merge). The value of the counter is the sum of all node counts.

This class is immutable, i.e. "modifying" methods return a new instance.

Attributes

Companion
object
Source
GCounter.scala
Supertypes
trait Serializable
class Object
trait Matchable
class Any
Show all
object GCounterKey

Attributes

Companion
class
Source
GCounter.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final case class GCounterKey(_id: String) extends Key[GCounter], ReplicatedDataSerialization

Attributes

Companion
object
Source
GCounter.scala
Supertypes
trait Product
trait Equals
class Key[GCounter]
trait Serializable
class Object
trait Matchable
class Any
Show all
object GSet

Attributes

Companion
class
Source
GSet.scala
Supertypes
class Object
trait Matchable
class Any
Self type
GSet.type
final case class GSet[A] extends DeltaReplicatedData, ReplicatedDelta, ReplicatedDataSerialization

Implements a 'Add Set' CRDT, also called a 'G-Set'. You can't remove elements of a G-Set.

Implements a 'Add Set' CRDT, also called a 'G-Set'. You can't remove elements of a G-Set.

It is described in the paper A comprehensive study of Convergent and Commutative Replicated Data Types.

A G-Set doesn't accumulate any garbage apart from the elements themselves.

This class is immutable, i.e. "modifying" methods return a new instance.

Attributes

Companion
object
Source
GSet.scala
Supertypes
trait Product
trait Equals
trait Serializable
class Object
trait Matchable
class Any
Show all
object GSetKey

Attributes

Companion
class
Source
GSet.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
GSetKey.type
final case class GSetKey[A](_id: String) extends Key[GSet[A]], ReplicatedDataSerialization

Attributes

Companion
object
Source
GSet.scala
Supertypes
trait Product
trait Equals
class Key[GSet[A]]
trait Serializable
class Object
trait Matchable
class Any
Show all
object Key

Attributes

Companion
class
Source
Key.scala
Supertypes
class Object
trait Matchable
class Any
Self type
Key.type
abstract class Key[+T <: ReplicatedData](val id: KeyId) extends Serializable

Key for the key-value data in Replicator. The type of the data value is defined in the key. Keys are compared equal if the id strings are equal, i.e. use unique identifiers.

Key for the key-value data in Replicator. The type of the data value is defined in the key. Keys are compared equal if the id strings are equal, i.e. use unique identifiers.

Specific classes are provided for the built in data types, e.g. ORSetKey, and you can create your own keys.

Attributes

Companion
object
Source
Key.scala
Supertypes
trait Serializable
class Object
trait Matchable
class Any
Known subtypes
class FlagKey
class GCounterKey
class GSetKey[A]
class LWWMapKey[A, B]
class LWWRegisterKey[A]
class ORMapKey[A, B]
class ORMultiMapKey[A, B]
class ORSetKey[A]
class PNCounterKey
class PNCounterMapKey[A]
Show all
object LWWMap

Attributes

Companion
class
Source
LWWMap.scala
Supertypes
class Object
trait Matchable
class Any
Self type
LWWMap.type
final class LWWMap[A, B] extends DeltaReplicatedData, ReplicatedDataSerialization, RemovedNodePruning

Specialized ORMap with LWWRegister values.

Specialized ORMap with LWWRegister values.

LWWRegister relies on synchronized clocks and should only be used when the choice of value is not important for concurrent updates occurring within the clock skew.

Instead of using timestamps based on System.currentTimeMillis() time it is possible to use a timestamp value based on something else, for example an increasing version number from a database record that is used for optimistic concurrency control.

The defaultClock is using max value of System.currentTimeMillis() and currentTimestamp + 1. This means that the timestamp is increased for changes on the same node that occurs within the same millisecond. It also means that it is safe to use the LWWMap without synchronized clocks when there is only one active writer, e.g. a Cluster Singleton. Such a single writer should then first read current value with ReadMajority (or more) before changing and writing the value with WriteMajority (or more).

For first-write-wins semantics you can use the LWWRegister#reverseClock instead of the LWWRegister#defaultClock

This class is immutable, i.e. "modifying" methods return a new instance.

Attributes

Companion
object
Source
LWWMap.scala
Supertypes
trait Serializable
class Object
trait Matchable
class Any
Show all
object LWWMapKey

Attributes

Companion
class
Source
LWWMap.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
LWWMapKey.type
final case class LWWMapKey[A, B](_id: String) extends Key[LWWMap[A, B]], ReplicatedDataSerialization

Attributes

Companion
object
Source
LWWMap.scala
Supertypes
trait Product
trait Equals
class Key[LWWMap[A, B]]
trait Serializable
class Object
trait Matchable
class Any
Show all
object LWWRegister

Attributes

Companion
class
Source
LWWRegister.scala
Supertypes
class Object
trait Matchable
class Any
Self type
final class LWWRegister[A] extends ReplicatedData, ReplicatedDataSerialization

Implements a 'Last Writer Wins Register' CRDT, also called a 'LWW-Register'.

Implements a 'Last Writer Wins Register' CRDT, also called a 'LWW-Register'.

It is described in the paper A comprehensive study of Convergent and Commutative Replicated Data Types.

Merge takes the register with highest timestamp. Note that this relies on synchronized clocks. LWWRegister should only be used when the choice of value is not important for concurrent updates occurring within the clock skew.

Merge takes the register updated by the node with lowest address (UniqueAddress is ordered) if the timestamps are exactly the same.

Instead of using timestamps based on System.currentTimeMillis() time it is possible to use a timestamp value based on something else, for example an increasing version number from a database record that is used for optimistic concurrency control.

The defaultClock is using max value of System.currentTimeMillis() and currentTimestamp + 1. This means that the timestamp is increased for changes on the same node that occurs within the same millisecond. It also means that it is safe to use the LWWRegister without synchronized clocks when there is only one active writer, e.g. a Cluster Singleton. Such a single writer should then first read current value with ReadMajority (or more) before changing and writing the value with WriteMajority (or more).

For first-write-wins semantics you can use the LWWRegister#reverseClock instead of the LWWRegister#defaultClock

This class is immutable, i.e. "modifying" methods return a new instance.

Attributes

Companion
object
Source
LWWRegister.scala
Supertypes
trait Serializable
class Object
trait Matchable
class Any
Show all
object LWWRegisterKey

Attributes

Companion
class
Source
LWWRegister.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final case class LWWRegisterKey[A](_id: String) extends Key[LWWRegister[A]], ReplicatedDataSerialization

Attributes

Companion
object
Source
LWWRegister.scala
Supertypes
trait Product
trait Equals
class Key[LWWRegister[A]]
trait Serializable
class Object
trait Matchable
class Any
Show all
object LmdbDurableStore

Attributes

Companion
class
Source
DurableStore.scala
Supertypes
class Object
trait Matchable
class Any
Self type
final class LmdbDurableStore(config: Config) extends Actor, ActorLogging

Attributes

Companion
object
Source
DurableStore.scala
Supertypes
trait ActorLogging
trait Actor
class Object
trait Matchable
class Any
final case class ManyVersionVector(versions: TreeMap[UniqueAddress, Long]) extends VersionVector

Attributes

Source
VersionVector.scala
Supertypes
trait Product
trait Equals
trait Serializable
class Object
trait Matchable
class Any
Show all
object ORMap

Attributes

Companion
class
Source
ORMap.scala
Supertypes
class Object
trait Matchable
class Any
Self type
ORMap.type
final class ORMap[A, B <: ReplicatedData] extends DeltaReplicatedData, ReplicatedDataSerialization, RemovedNodePruning

Implements a 'Observed Remove Map' CRDT, also called a 'OR-Map'.

Implements a 'Observed Remove Map' CRDT, also called a 'OR-Map'.

It has similar semantics as an ORSet, but in case of concurrent updates the values are merged, and must therefore be ReplicatedData types themselves.

This class is immutable, i.e. "modifying" methods return a new instance.

Attributes

Companion
object
Source
ORMap.scala
Supertypes
trait Serializable
class Object
trait Matchable
class Any
Show all
object ORMapKey

Attributes

Companion
class
Source
ORMap.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
ORMapKey.type
final case class ORMapKey[A, B <: ReplicatedData](_id: String) extends Key[ORMap[A, B]], ReplicatedDataSerialization

Attributes

Companion
object
Source
ORMap.scala
Supertypes
trait Product
trait Equals
class Key[ORMap[A, B]]
trait Serializable
class Object
trait Matchable
class Any
Show all
object ORMultiMap

Attributes

Companion
class
Source
ORMultiMap.scala
Supertypes
class Object
trait Matchable
class Any
Self type
ORMultiMap.type
final class ORMultiMap[A, B] extends DeltaReplicatedData, ReplicatedDataSerialization, RemovedNodePruning

An immutable multi-map implementation. This class wraps an ORMap with an ORSet for the map's value.

An immutable multi-map implementation. This class wraps an ORMap with an ORSet for the map's value.

This class is immutable, i.e. "modifying" methods return a new instance.

Note that on concurrent adds and removals for the same key (on the same set), removals can be lost.

Attributes

Companion
object
Source
ORMultiMap.scala
Supertypes
trait Serializable
class Object
trait Matchable
class Any
Show all
object ORMultiMapKey

Attributes

Companion
class
Source
ORMultiMap.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final case class ORMultiMapKey[A, B](_id: String) extends Key[ORMultiMap[A, B]], ReplicatedDataSerialization

Attributes

Companion
object
Source
ORMultiMap.scala
Supertypes
trait Product
trait Equals
class Key[ORMultiMap[A, B]]
trait Serializable
class Object
trait Matchable
class Any
Show all
object ORSet

Attributes

Companion
class
Source
ORSet.scala
Supertypes
class Object
trait Matchable
class Any
Self type
ORSet.type
final class ORSet[A] extends DeltaReplicatedData, ReplicatedDataSerialization, RemovedNodePruning

Implements a 'Observed Remove Set' CRDT, also called a 'OR-Set'. Elements can be added and removed any number of times. Concurrent add wins over remove.

Implements a 'Observed Remove Set' CRDT, also called a 'OR-Set'. Elements can be added and removed any number of times. Concurrent add wins over remove.

It is not implemented as in the paper A comprehensive study of Convergent and Commutative Replicated Data Types. This is more space efficient and doesn't accumulate garbage for removed elements. It is described in the paper An optimized conflict-free replicated set The implementation is inspired by the Riak DT riak_dt_orswot.

The ORSet has a version vector that is incremented when an element is added to the set. The node -&gt; count pair for that increment is stored against the element as its "birth dot". Every time the element is re-added to the set, its "birth dot" is updated to that of the node -&gt; count version vector entry resulting from the add. When an element is removed, we simply drop it, no tombstones.

When an element exists in replica A and not replica B, is it because A added it and B has not yet seen that, or that B removed it and A has not yet seen that? In this implementation we compare the dot of the present element to the version vector in the Set it is absent from. If the element dot is not "seen" by the Set version vector, that means the other set has yet to see this add, and the item is in the merged Set. If the Set version vector dominates the dot, that means the other Set has removed this element already, and the item is not in the merged Set.

This class is immutable, i.e. "modifying" methods return a new instance.

Attributes

Companion
object
Source
ORSet.scala
Supertypes
trait Serializable
class Object
trait Matchable
class Any
Show all
object ORSetKey

Attributes

Companion
class
Source
ORSet.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
ORSetKey.type
final case class ORSetKey[A](_id: String) extends Key[ORSet[A]], ReplicatedDataSerialization

Attributes

Companion
object
Source
ORSet.scala
Supertypes
trait Product
trait Equals
class Key[ORSet[A]]
trait Serializable
class Object
trait Matchable
class Any
Show all
final case class OneVersionVector extends VersionVector

Attributes

Source
VersionVector.scala
Supertypes
trait Product
trait Equals
trait Serializable
class Object
trait Matchable
class Any
Show all
object PNCounter

Attributes

Companion
class
Source
PNCounter.scala
Supertypes
class Object
trait Matchable
class Any
Self type
PNCounter.type
final class PNCounter extends DeltaReplicatedData, ReplicatedDelta, ReplicatedDataSerialization, RemovedNodePruning

Implements a 'Increment/Decrement Counter' CRDT, also called a 'PN-Counter'.

Implements a 'Increment/Decrement Counter' CRDT, also called a 'PN-Counter'.

It is described in the paper A comprehensive study of Convergent and Commutative Replicated Data Types.

PN-Counters allow the counter to be incremented by tracking the increments (P) separate from the decrements (N). Both P and N are represented as two internal GCounters. Merge is handled by merging the internal P and N counters. The value of the counter is the value of the P counter minus the value of the N counter.

This class is immutable, i.e. "modifying" methods return a new instance.

Attributes

Companion
object
Source
PNCounter.scala
Supertypes
trait Serializable
class Object
trait Matchable
class Any
Show all
object PNCounterKey

Attributes

Companion
class
Source
PNCounter.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final case class PNCounterKey(_id: String) extends Key[PNCounter], ReplicatedDataSerialization

Attributes

Companion
object
Source
PNCounter.scala
Supertypes
trait Product
trait Equals
class Key[PNCounter]
trait Serializable
class Object
trait Matchable
class Any
Show all
object PNCounterMap

Attributes

Companion
class
Source
PNCounterMap.scala
Supertypes
class Object
trait Matchable
class Any
Self type
final class PNCounterMap[A] extends DeltaReplicatedData, ReplicatedDataSerialization, RemovedNodePruning

Map of named counters. Specialized ORMap with PNCounter values.

Map of named counters. Specialized ORMap with PNCounter values.

This class is immutable, i.e. "modifying" methods return a new instance.

Attributes

Companion
object
Source
PNCounterMap.scala
Supertypes
trait Serializable
class Object
trait Matchable
class Any
Show all
object PNCounterMapKey

Attributes

Companion
class
Source
PNCounterMap.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final case class PNCounterMapKey[A](_id: String) extends Key[PNCounterMap[A]], ReplicatedDataSerialization

Attributes

Companion
object
Source
PNCounterMap.scala
Supertypes
trait Product
trait Equals
class Key[PNCounterMap[A]]
trait Serializable
class Object
trait Matchable
class Any
Show all
trait RemovedNodePruning extends ReplicatedData

ReplicatedData that has support for pruning of data belonging to a specific node may implement this interface. When a node is removed from the cluster these methods will be used by the Replicator to collapse data from the removed node into some other node in the cluster.

ReplicatedData that has support for pruning of data belonging to a specific node may implement this interface. When a node is removed from the cluster these methods will be used by the Replicator to collapse data from the removed node into some other node in the cluster.

See process description in the 'CRDT Garbage' section of the Replicator documentation.

Attributes

Source
ReplicatedData.scala
Supertypes
class Object
trait Matchable
class Any
Known subtypes
class GCounter
class LWWMap[A, B]
class ORMap[A, B]
class ORMultiMap[A, B]
class ORSet[A]
class PNCounter
class PNCounterMap[A]
Show all
trait ReplicatedData

Interface for implementing a state based convergent replicated data type (CvRDT).

Interface for implementing a state based convergent replicated data type (CvRDT).

ReplicatedData types must be serializable with an Akka Serializer. It is highly recommended to implement a serializer with Protobuf or similar. The built in data types are marked with ReplicatedDataSerialization and serialized with pekko.cluster.ddata.protobuf.ReplicatedDataSerializer.

Serialization of the data types are used in remote messages and also for creating message digests (SHA-1) to detect changes. Therefore it is important that the serialization produce the same bytes for the same content. For example sets and maps should be sorted deterministically in the serialization.

ReplicatedData types should be immutable, i.e. "modifying" methods should return a new instance.

Implement the additional methods of DeltaReplicatedData if it has support for delta-CRDT replication.

Attributes

Source
ReplicatedData.scala
Supertypes
class Object
trait Matchable
class Any
Known subtypes
class GCounter
class GSet[A]
class LWWMap[A, B]
class ORMap[A, B]
class ORMultiMap[A, B]
class ORSet[A]
class PNCounter
class PNCounterMap[A]
class Flag
class LWWRegister[A]
trait DeltaOp
trait DeltaOp
Show all
trait ReplicatedDataSerialization extends Serializable

Marker trait for ReplicatedData serialized by pekko.cluster.ddata.protobuf.ReplicatedDataSerializer.

Marker trait for ReplicatedData serialized by pekko.cluster.ddata.protobuf.ReplicatedDataSerializer.

Attributes

Source
ReplicatedData.scala
Supertypes
trait Serializable
class Object
trait Matchable
class Any
Known subtypes
class Flag
class FlagKey
class GCounter
class GCounterKey
class GSet[A]
class GSetKey[A]
class LWWMap[A, B]
class LWWMapKey[A, B]
class LWWRegister[A]
class LWWRegisterKey[A]
trait DeltaOp
class ORMap[A, B]
class ORMapKey[A, B]
class ORMultiMap[A, B]
class ORMultiMapKey[A, B]
trait DeltaOp
class ORSet[A]
class ORSetKey[A]
class PNCounter
class PNCounterKey
class PNCounterMap[A]
class PNCounterMapKey[A]
Show all
trait ReplicatedDelta extends ReplicatedData

The delta must implement this type.

The delta must implement this type.

Attributes

Source
ReplicatedData.scala
Supertypes
class Object
trait Matchable
class Any
Known subtypes
class GCounter
class GSet[A]
trait DeltaOp
trait DeltaOp
class PNCounter
Show all
trait ReplicatedDeltaSize

Some complex deltas grow in size for each update and above a configured threshold such deltas are discarded and sent as full state instead. This interface should be implemented by such deltas to define its size. This is number of elements or similar size hint, not size in bytes. The threshold is defined in pekko.cluster.distributed-data.delta-crdt.max-delta-size or corresponding ReplicatorSettings.

Some complex deltas grow in size for each update and above a configured threshold such deltas are discarded and sent as full state instead. This interface should be implemented by such deltas to define its size. This is number of elements or similar size hint, not size in bytes. The threshold is defined in pekko.cluster.distributed-data.delta-crdt.max-delta-size or corresponding ReplicatorSettings.

Attributes

Source
ReplicatedData.scala
Supertypes
class Object
trait Matchable
class Any
object Replicator

Attributes

Companion
class
Source
Replicator.scala
Supertypes
class Object
trait Matchable
class Any
Self type
Replicator.type
final class Replicator(settings: ReplicatorSettings) extends Actor, ActorLogging

A replicated in-memory data store supporting low latency and high availability requirements.

A replicated in-memory data store supporting low latency and high availability requirements.

The Replicator actor takes care of direct replication and gossip based dissemination of Conflict Free Replicated Data Types (CRDTs) to replicas in the the cluster. The data types must be convergent CRDTs and implement ReplicatedData, i.e. they provide a monotonic merge function and the state changes always converge.

You can use your own custom ReplicatedData or DeltaReplicatedData types, and several types are provided by this package, such as:

For good introduction to the CRDT subject watch the Eventually Consistent Data Structures talk by Sean Cribbs and and the talk by Mark Shapiro and read the excellent paper A comprehensive study of Convergent and Commutative Replicated Data Types by Mark Shapiro et. al.

The Replicator actor must be started on each node in the cluster, or group of nodes tagged with a specific role. It communicates with other Replicator instances with the same path (without address) that are running on other nodes . For convenience it can be used with the DistributedData extension but it can also be started as an ordinary actor using the Replicator.props. If it is started as an ordinary actor it is important that it is given the same name, started on same path, on all nodes.

Delta State Replicated Data Types are supported. delta-CRDT is a way to reduce the need for sending the full state for updates. For example adding element 'c' and 'd' to set {'a', 'b'} would result in sending the delta {'c', 'd'} and merge that with the state on the receiving side, resulting in set {'a', 'b', 'c', 'd'}.

The protocol for replicating the deltas supports causal consistency if the data type is marked with RequiresCausalDeliveryOfDeltas. Otherwise it is only eventually consistent. Without causal consistency it means that if elements 'c' and 'd' are added in two separate Update operations these deltas may occasionally be propagated to nodes in different order than the causal order of the updates. For this example it can result in that set {'a', 'b', 'd'} can be seen before element 'c' is seen. Eventually it will be {'a', 'b', 'c', 'd'}.

== Update ==

To modify and replicate a ReplicatedData value you send a Replicator.Update message to the local Replicator. The current data value for the key of the Update is passed as parameter to the modify function of the Update. The function is supposed to return the new value of the data, which will then be replicated according to the given consistency level.

The modify function is called by the Replicator actor and must therefore be a pure function that only uses the data parameter and stable fields from enclosing scope. It must for example not access sender() reference of an enclosing actor.

Update is intended to only be sent from an actor running in same local ActorSystem as the Replicator, because the modify function is typically not serializable.

You supply a write consistency level which has the following meaning:

  • WriteLocal the value will immediately only be written to the local replica, and later disseminated with gossip

  • WriteTo(n) the value will immediately be written to at least n replicas, including the local replica

  • WriteMajority the value will immediately be written to a majority of replicas, i.e. at least N/2 + 1 replicas, where N is the number of nodes in the cluster (or cluster role group)

  • WriteAll the value will immediately be written to all nodes in the cluster (or all nodes in the cluster role group)

As reply of the Update a Replicator.UpdateSuccess is sent to the sender of the Update if the value was successfully replicated according to the supplied consistency level within the supplied timeout. Otherwise a Replicator.UpdateFailure subclass is sent back. Note that a Replicator.UpdateTimeout reply does not mean that the update completely failed or was rolled back. It may still have been replicated to some nodes, and will eventually be replicated to all nodes with the gossip protocol.

You will always see your own writes. For example if you send two Update messages changing the value of the same key, the modify function of the second message will see the change that was performed by the first Update message.

In the Update message you can pass an optional request context, which the Replicator does not care about, but is included in the reply messages. This is a convenient way to pass contextual information (e.g. original sender) without having to use ask or local correlation data structures.

== Get ==

To retrieve the current value of a data you send Replicator.Get message to the Replicator. You supply a consistency level which has the following meaning:

  • ReadLocal the value will only be read from the local replica

  • ReadFrom(n) the value will be read and merged from n replicas, including the local replica

  • ReadMajority the value will be read and merged from a majority of replicas, i.e. at least N/2 + 1 replicas, where N is the number of nodes in the cluster (or cluster role group)

  • ReadAll the value will be read and merged from all nodes in the cluster (or all nodes in the cluster role group)

As reply of the Get a Replicator.GetSuccess is sent to the sender of the Get if the value was successfully retrieved according to the supplied consistency level within the supplied timeout. Otherwise a Replicator.GetFailure is sent. If the key does not exist the reply will be Replicator.NotFound.

You will always read your own writes. For example if you send a Update message followed by a Get of the same key the Get will retrieve the change that was performed by the preceding Update message. However, the order of the reply messages are not defined, i.e. in the previous example you may receive the GetSuccess before the UpdateSuccess.

In the Get message you can pass an optional request context in the same way as for the Update message, described above. For example the original sender can be passed and replied to after receiving and transforming GetSuccess.

== Subscribe ==

You may also register interest in change notifications by sending Replicator.Subscribe message to the Replicator. It will send Replicator.Changed messages to the registered subscriber when the data for the subscribed key is updated. Subscribers will be notified periodically with the configured notify-subscribers-interval, and it is also possible to send an explicit Replicator.FlushChanges message to the Replicator to notify the subscribers immediately.

The subscriber is automatically removed if the subscriber is terminated. A subscriber can also be deregistered with the Replicator.Unsubscribe message.

== Delete ==

A data entry can be deleted by sending a Replicator.Delete message to the local local Replicator. As reply of the Delete a Replicator.DeleteSuccess is sent to the sender of the Delete if the value was successfully deleted according to the supplied consistency level within the supplied timeout. Otherwise a Replicator.ReplicationDeleteFailure is sent. Note that ReplicationDeleteFailure does not mean that the delete completely failed or was rolled back. It may still have been replicated to some nodes, and may eventually be replicated to all nodes.

A deleted key cannot be reused again, but it is still recommended to delete unused data entries because that reduces the replication overhead when new nodes join the cluster. Subsequent Delete, Update and Get requests will be replied with Replicator.DataDeleted, Replicator.UpdateDataDeleted and Replicator.GetDataDeleted respectively. Subscribers will receive Replicator.Deleted.

In the Delete message you can pass an optional request context in the same way as for the Update message, described above. For example the original sender can be passed and replied to after receiving and transforming DeleteSuccess.

== CRDT Garbage ==

One thing that can be problematic with CRDTs is that some data types accumulate history (garbage). For example a GCounter keeps track of one counter per node. If a GCounter has been updated from one node it will associate the identifier of that node forever. That can become a problem for long running systems with many cluster nodes being added and removed. To solve this problem the Replicator performs pruning of data associated with nodes that have been removed from the cluster. Data types that need pruning have to implement RemovedNodePruning. The pruning consists of several steps:

  • When a node is removed from the cluster it is first important that all updates that were done by that node are disseminated to all other nodes. The pruning will not start before the maxPruningDissemination duration has elapsed. The time measurement is stopped when any replica is unreachable, but it's still recommended to configure this with certain margin. It should be in the magnitude of minutes.

  • The nodes are ordered by their address and the node ordered first is called leader. The leader initiates the pruning by adding a PruningInitialized marker in the data envelope. This is gossiped to all other nodes and they mark it as seen when they receive it.

  • When the leader sees that all other nodes have seen the PruningInitialized marker the leader performs the pruning and changes the marker to PruningPerformed so that nobody else will redo the pruning. The data envelope with this pruning state is a CRDT itself. The pruning is typically performed by "moving" the part of the data associated with the removed node to the leader node. For example, a GCounter is a Map with the node as key and the counts done by that node as value. When pruning the value of the removed node is moved to the entry owned by the leader node. See RemovedNodePruning#prune.

  • Thereafter the data is always cleared from parts associated with the removed node so that it does not come back when merging. See RemovedNodePruning#pruningCleanup

  • After another maxPruningDissemination duration after pruning the last entry from the removed node the PruningPerformed markers in the data envelope are collapsed into a single tombstone entry, for efficiency. Clients may continue to use old data and therefore all data are always cleared from parts associated with tombstoned nodes.

Attributes

Companion
object
Source
Replicator.scala
Supertypes
trait ActorLogging
trait Actor
class Object
trait Matchable
class Any
object ReplicatorSettings

Attributes

Companion
class
Source
Replicator.scala
Supertypes
class Object
trait Matchable
class Any
Self type
final class ReplicatorSettings(val roles: Set[String], val gossipInterval: FiniteDuration, val notifySubscribersInterval: FiniteDuration, val maxDeltaElements: Int, val dispatcher: String, val pruningInterval: FiniteDuration, val maxPruningDissemination: FiniteDuration, val durableStoreProps: Either[(String, Config), Props], val durableKeys: Set[KeyId], val pruningMarkerTimeToLive: FiniteDuration, val durablePruningMarkerTimeToLive: FiniteDuration, val deltaCrdtEnabled: Boolean, val maxDeltaSize: Int, val preferOldest: Boolean, val logDataSizeExceeding: Option[Int])

Value parameters

dispatcher

Id of the dispatcher to use for Replicator actors. If not specified ("") the default dispatcher is used.

durableKeys

Keys that are durable. Prefix matching is supported by using * at the end of a key. All entries can be made durable by including "*" in the Set.

durableStoreProps

Props for the durable store actor, the Left alternative is a tuple of fully qualified actor class name and the config constructor parameter of that class, the Right alternative is the Props of the actor.

gossipInterval

How often the Replicator should send out gossip information.

logDataSizeExceeding

Log data size.

maxDeltaElements

Maximum number of entries to transfer in one gossip message when synchronizing the replicas. Next chunk will be transferred in next round of gossip.

maxPruningDissemination

How long time it takes (worst case) to spread the data to all other replica nodes. This is used when initiating and completing the pruning process of data associated with removed cluster nodes. The time measurement is stopped when any replica is unreachable, so it should be configured to worst case in a healthy cluster.

notifySubscribersInterval

How often the subscribers will be notified of changes, if any.

preferOldest

Update and Get operations are sent to oldest nodes first.

pruningInterval

How often the Replicator checks for pruning of data associated with removed cluster nodes.

roles

Replicas are running on members tagged with these roles. The member must have all given roles. All members are used if empty.

Attributes

Companion
object
Source
Replicator.scala
Supertypes
class Object
trait Matchable
class Any
trait RequiresCausalDeliveryOfDeltas extends ReplicatedDelta

Marker that specifies that the deltas must be applied in causal order. There is some overhead of managing the causal delivery so it should only be used for types that need it.

Marker that specifies that the deltas must be applied in causal order. There is some overhead of managing the causal delivery so it should only be used for types that need it.

Note that if the full state type T is different from the delta type D it is the delta D that should be marked with this.

Attributes

Source
ReplicatedData.scala
Supertypes
class Object
trait Matchable
class Any
Known subtypes
trait DeltaOp
trait DeltaOp
final case class SelfUniqueAddress(uniqueAddress: UniqueAddress)

Cluster non-specific (typed vs classic) wrapper for pekko.cluster.UniqueAddress.

Cluster non-specific (typed vs classic) wrapper for pekko.cluster.UniqueAddress.

Attributes

Source
DistributedData.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object VersionVector

VersionVector module with helper classes and methods.

VersionVector module with helper classes and methods.

Attributes

Companion
class
Source
VersionVector.scala
Supertypes
trait Sum
trait Mirror
class Object
trait Matchable
class Any
Self type
sealed abstract class VersionVector extends ReplicatedData, ReplicatedDataSerialization, RemovedNodePruning

Representation of a Vector-based clock (counting clock), inspired by Lamport logical clocks.

Representation of a Vector-based clock (counting clock), inspired by Lamport logical clocks.

Reference:
  1) Leslie Lamport (1978). "Time, clocks, and the ordering of events in a distributed system". Communications of the ACM 21 (7): 558-565.
  2) Friedemann Mattern (1988). "Virtual Time and Global States of Distributed Systems". Workshop on Parallel and Distributed Algorithms: pp. 215-226

Based on code from org.apache.pekko.cluster.VectorClock.

This class is immutable, i.e. "modifying" methods return a new instance.

Attributes

Companion
object
Source
VersionVector.scala
Supertypes
trait Serializable
class Object
trait Matchable
class Any
Show all
Known subtypes
Generated with