org.apache.cassandra.gms

Class Gossiper

java.lang.Object

org.apache.cassandra.gms.Gossiper

All Implemented Interfaces:

GossiperMBean, IFailureDetectionEventListener

public class Gossiper
extends java.lang.Object
implements IFailureDetectionEventListener, GossiperMBean

This module is responsible for Gossiping information for the local endpoint. This abstraction maintains the list of live and dead endpoints. Periodically i.e. every 1 second this module chooses a random node and initiates a round of Gossip with it. A round of Gossip involves 3 rounds of messaging. For instance if node A wants to initiate a round of Gossip with node B it starts off by sending node B a GossipDigestSynMessage. Node B on receipt of this message sends node A a GossipDigestAckMessage. On receipt of this message node A sends node B a GossipDigestAck2Message which completes a round of Gossip. This module as and when it hears one of the three above mentioned messages updates the Failure Detector with the liveness information. Upon hearing a GossipShutdownMessage, this module will instantly mark the remote node as down in the Failure Detector. This class is not threadsafe and any state changes should happen in the gossip stage.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Gossiper.Props

Field Summary

Fields

Modifier and Type	Field and Description
static long	aVeryLongTime
static Gossiper	instance
static int	intervalInMillis
static java.lang.String	MBEAN_NAME
static int	QUARANTINE_DELAY

Constructor Summary

Constructors

Constructor and Description
Gossiper(boolean registerJmx)

Method Summary

Modifier and Type	Method and Description
void	addExpireTimeForEndpoint(InetAddressAndPort endpoint, long expireTime)
void	addLocalApplicationState(ApplicationState applicationState, VersionedValue value)
void	addLocalApplicationStates(java.util.List<Pair<ApplicationState,VersionedValue>> states)
void	addSavedEndpoint(InetAddressAndPort ep) Add an endpoint we knew about previously, but whose state is unknown
void	advertiseRemoving(InetAddressAndPort endpoint, java.util.UUID hostId, java.util.UUID localHostId) This method will begin removing an existing endpoint from the cluster by spoofing its state This should never be called unless this coordinator has had 'removenode' invoked
void	advertiseTokenRemoved(InetAddressAndPort endpoint, java.util.UUID hostId) Handles switching the endpoint's state from REMOVING_TOKEN to REMOVED_TOKEN This should only be called after advertiseRemoving
void	applyStateLocally(java.util.Map<InetAddressAndPort,EndpointState> epStateMap)
void	assassinateEndpoint(java.lang.String address) Do not call this method unless you know what you are doing.
void	clearUnsafe()
int	compareEndpointStartup(InetAddressAndPort addr1, InetAddressAndPort addr2) determine which endpoint started up earlier
static long	computeExpireTime()
void	convict(InetAddressAndPort endpoint, double phi) This method is part of IFailureDetectionEventListener interface.
java.util.Map<InetAddressAndPort,EndpointState>	doShadowRound()
java.util.Map<InetAddressAndPort,EndpointState>	doShadowRound(java.util.Set<InetAddressAndPort> peers) Do a single 'shadow' round of gossip by retrieving endpoint states that will be stored exclusively in the map return value, instead of endpointStateMap.
void	expireUpgradeFromVersion()
void	forceNewerGeneration()
java.lang.String	getApplicationState(InetAddressAndPort endpoint, ApplicationState state) The value for the provided application state for the provided endpoint as currently known by this Gossip instance.
int	getCurrentGenerationNumber(InetAddressAndPort endpoint)
int	getCurrentGenerationNumber(java.lang.String address)
int	getEndpointCount()
long	getEndpointDowntime(InetAddressAndPort ep)
long	getEndpointDowntime(java.lang.String address)
com.google.common.collect.ImmutableSet<InetAddressAndPort>	getEndpoints()
EndpointState	getEndpointStateForEndpoint(InetAddressAndPort ep)
protected long	getExpireTimeForEndpoint(InetAddressAndPort endpoint)
java.util.UUID	getHostId(InetAddressAndPort endpoint)
java.util.UUID	getHostId(InetAddressAndPort endpoint, java.util.Map<InetAddressAndPort,EndpointState> epStates)
InetAddressAndPort	getInternalAddressAndPort(InetAddressAndPort endpoint)
java.util.Set<InetAddressAndPort>	getLiveMembers()
java.util.Set<InetAddressAndPort>	getLiveTokenOwners()
CassandraVersion	getMinVersion(long delay, java.util.concurrent.TimeUnit timeUnit)
CassandraVersion	getReleaseVersion(InetAddressAndPort ep)
java.util.Map<java.lang.String,java.util.List<java.lang.String>>	getReleaseVersionsWithPort() Returns each node's database release version
java.util.UUID	getSchemaVersion(InetAddressAndPort ep)
java.util.List<java.lang.String>	getSeeds() JMX endpoint for getting the list of seeds from the node
java.util.Set<InetAddressAndPort>	getUnreachableMembers()
java.util.Set<InetAddressAndPort>	getUnreachableTokenOwners()
boolean	hasMajorVersion3Nodes() Returns false only if the information about the version of each node in the cluster is available and ALL the nodes are on 4.0+ (regardless of the patch version).
void	initializeNodeUnsafe(InetAddressAndPort addr, java.util.UUID uuid, int generationNbr)
void	initializeNodeUnsafe(InetAddressAndPort addr, java.util.UUID uuid, int netVersion, int generationNbr)
void	initializeUnreachableNodeUnsafe(InetAddressAndPort addr) Creates a new dead EndpointState that is empty.
void	injectApplicationState(InetAddressAndPort endpoint, ApplicationState state, VersionedValue value)
boolean	isAdministrativelyInactiveState(EndpointState epState)
boolean	isAdministrativelyInactiveState(InetAddressAndPort endpoint)
boolean	isAlive(InetAddressAndPort endpoint)
boolean	isDeadState(EndpointState epState)
boolean	isEnabled()
boolean	isGossipOnlyMember(InetAddressAndPort endpoint)
protected boolean	isInShadowRound()
boolean	isKnownEndpoint(InetAddressAndPort endpoint)
boolean	isSafeForStartup(InetAddressAndPort endpoint, java.util.UUID localHostUUID, boolean isBootstrapping, java.util.Map<InetAddressAndPort,EndpointState> epStates) Check if this node can safely be started and join the ring.
boolean	isSilentShutdownState(EndpointState epState)
boolean	isUpgradingFromVersionLowerThan(CassandraVersion referenceVersion) Returns true if there are nodes on version lower than the provided version
protected void	markAsShutdown(InetAddressAndPort endpoint) This method is used to mark a node as shutdown; that is it gracefully exited on its own and told us about it
void	markDead(InetAddressAndPort addr, EndpointState localState)
protected void	maybeFinishShadowRound(InetAddressAndPort respondent, boolean isInShadowRound, java.util.Map<InetAddressAndPort,EndpointState> epStateMap)
void	maybeInitializeLocalState(int generationNbr)
void	realMarkAlive(InetAddressAndPort addr, EndpointState localState)
void	register(IEndpointStateChangeSubscriber subscriber) Register for interesting state changes.
java.util.List<java.lang.String>	reloadSeeds() JMX interface for triggering an update of the seed node list.
void	removeEndpoint(InetAddressAndPort endpoint) Removes the endpoint from Gossip but retains endpoint state
void	replacedEndpoint(InetAddressAndPort endpoint) Remove the Endpoint and evict immediately, to avoid gossiping about this node.
void	replacementQuarantine(InetAddressAndPort endpoint) Quarantine endpoint specifically for replacement purposes.
static void	runInGossipStageBlocking(java.lang.Runnable runnable)
boolean	seenAnySeed()
void	setLastProcessedMessageAt(long timeInMillis)
void	start(int generationNumber)
void	start(int generationNbr, java.util.Map<ApplicationState,VersionedValue> preloadLocalStates) Start the gossiper with the generation number, preloading the map of application states before starting
void	stop()
void	stopShutdownAndWait(long timeout, java.util.concurrent.TimeUnit unit)
boolean	sufficientForStartupSafetyCheck(java.util.Map<InetAddressAndPort,EndpointState> epStateMap)
void	unregister(IEndpointStateChangeSubscriber subscriber) Unregister interest for state changes.
void	unsafeAssassinateEndpoint(java.lang.String address)
boolean	waitForSchemaAgreement(long maxWait, java.util.concurrent.TimeUnit unit, java.util.function.BooleanSupplier abortCondition) Blockingly wait for all live nodes to agree on the current schema version.
static void	waitToSettle()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MBEAN_NAME

public static final java.lang.String MBEAN_NAME

See Also:

Constant Field Values

intervalInMillis

public static final int intervalInMillis

See Also:

Constant Field Values

QUARANTINE_DELAY

public static final int QUARANTINE_DELAY

instance

public static final Gossiper instance

aVeryLongTime

public static final long aVeryLongTime

Constructor Detail

Gossiper

public Gossiper(boolean registerJmx)

Method Detail

clearUnsafe

public void clearUnsafe()

expireUpgradeFromVersion

public void expireUpgradeFromVersion()

setLastProcessedMessageAt

public void setLastProcessedMessageAt(long timeInMillis)

seenAnySeed

public boolean seenAnySeed()

register

public void register(IEndpointStateChangeSubscriber subscriber)

Register for interesting state changes.

Parameters:

subscriber - module which implements the IEndpointStateChangeSubscriber

unregister

public void unregister(IEndpointStateChangeSubscriber subscriber)

Unregister interest for state changes.

Parameters:

subscriber - module which implements the IEndpointStateChangeSubscriber

getLiveMembers

public java.util.Set<InetAddressAndPort> getLiveMembers()

Returns:

a list of live gossip participants, including fat clients

getLiveTokenOwners

public java.util.Set<InetAddressAndPort> getLiveTokenOwners()

Returns:

a list of live ring members.

getUnreachableMembers

public java.util.Set<InetAddressAndPort> getUnreachableMembers()

Returns:

a list of unreachable gossip participants, including fat clients

getUnreachableTokenOwners

public java.util.Set<InetAddressAndPort> getUnreachableTokenOwners()

Returns:

a list of unreachable token owners

getEndpointDowntime

public long getEndpointDowntime(InetAddressAndPort ep)

runInGossipStageBlocking

public static void runInGossipStageBlocking(java.lang.Runnable runnable)

convict

public void convict(InetAddressAndPort endpoint,
                    double phi)

This method is part of IFailureDetectionEventListener interface. This is invoked by the Failure Detector when it convicts an end point.

Specified by:

convict in interface IFailureDetectionEventListener

Parameters:

endpoint - end point that is convicted.

phi - the value of phi with with ep was convicted

markAsShutdown

protected void markAsShutdown(InetAddressAndPort endpoint)

This method is used to mark a node as shutdown; that is it gracefully exited on its own and told us about it

Parameters:

endpoint - endpoint that has shut itself down

removeEndpoint

public void removeEndpoint(InetAddressAndPort endpoint)

Removes the endpoint from Gossip but retains endpoint state

replacementQuarantine

public void replacementQuarantine(InetAddressAndPort endpoint)

Quarantine endpoint specifically for replacement purposes.

Parameters:

endpoint -

replacedEndpoint

public void replacedEndpoint(InetAddressAndPort endpoint)

Remove the Endpoint and evict immediately, to avoid gossiping about this node. This should only be called when a token is taken over by a new IP address.

Parameters:

endpoint - The endpoint that has been replaced

advertiseRemoving

public void advertiseRemoving(InetAddressAndPort endpoint,
                              java.util.UUID hostId,
                              java.util.UUID localHostId)

This method will begin removing an existing endpoint from the cluster by spoofing its state This should never be called unless this coordinator has had 'removenode' invoked

Parameters:

endpoint - - the endpoint being removed

hostId - - the ID of the host being removed

localHostId - - my own host ID for replication coordination

advertiseTokenRemoved

public void advertiseTokenRemoved(InetAddressAndPort endpoint,
                                  java.util.UUID hostId)

Handles switching the endpoint's state from REMOVING_TOKEN to REMOVED_TOKEN This should only be called after advertiseRemoving

Parameters:

endpoint -

hostId -

unsafeAssassinateEndpoint

public void unsafeAssassinateEndpoint(java.lang.String address)
                               throws java.net.UnknownHostException

Specified by:

unsafeAssassinateEndpoint in interface GossiperMBean

Throws:

java.net.UnknownHostException

assassinateEndpoint

public void assassinateEndpoint(java.lang.String address)
                         throws java.net.UnknownHostException

Do not call this method unless you know what you are doing. It will try extremely hard to obliterate any endpoint from the ring, even if it does not know about it.

Specified by:

assassinateEndpoint in interface GossiperMBean

Parameters:

address -

Throws:

java.net.UnknownHostException

isKnownEndpoint

public boolean isKnownEndpoint(InetAddressAndPort endpoint)

getCurrentGenerationNumber

public int getCurrentGenerationNumber(InetAddressAndPort endpoint)

isGossipOnlyMember

public boolean isGossipOnlyMember(InetAddressAndPort endpoint)

isSafeForStartup

public boolean isSafeForStartup(InetAddressAndPort endpoint,
                                java.util.UUID localHostUUID,
                                boolean isBootstrapping,
                                java.util.Map<InetAddressAndPort,EndpointState> epStates)

Check if this node can safely be started and join the ring. If the node is bootstrapping, examines gossip state for any previous status to decide whether it's safe to allow this node to start and bootstrap. If not bootstrapping, compares the host ID that the node itself has (obtained by reading from system.local or generated if not present) with the host ID obtained from gossip for the endpoint address (if any). This latter case prevents a non-bootstrapping, new node from being started with the same address of a previously started, but currently down predecessor.

Parameters:

endpoint - - the endpoint to check

localHostUUID - - the host id to check

isBootstrapping - - whether the node intends to bootstrap when joining

epStates - - endpoint states in the cluster

Returns:

true if it is safe to start the node, false otherwise

getExpireTimeForEndpoint

protected long getExpireTimeForEndpoint(InetAddressAndPort endpoint)

getEndpointStateForEndpoint

public EndpointState getEndpointStateForEndpoint(InetAddressAndPort ep)

getEndpoints

public com.google.common.collect.ImmutableSet<InetAddressAndPort> getEndpoints()

getEndpointCount

public int getEndpointCount()

getHostId

public java.util.UUID getHostId(InetAddressAndPort endpoint)

getHostId

public java.util.UUID getHostId(InetAddressAndPort endpoint,
                                java.util.Map<InetAddressAndPort,EndpointState> epStates)

getInternalAddressAndPort

public InetAddressAndPort getInternalAddressAndPort(InetAddressAndPort endpoint)

getApplicationState

public java.lang.String getApplicationState(InetAddressAndPort endpoint,
                                            ApplicationState state)

The value for the provided application state for the provided endpoint as currently known by this Gossip instance.

Parameters:

endpoint - the endpoint from which to get the endpoint state.

state - the endpoint state to get.

Returns:

the value of the application state state for endpoint, or null if either endpoint is not known by Gossip or has no value for state.

compareEndpointStartup

public int compareEndpointStartup(InetAddressAndPort addr1,
                                  InetAddressAndPort addr2)

determine which endpoint started up earlier

realMarkAlive

public void realMarkAlive(InetAddressAndPort addr,
                          EndpointState localState)

markDead

public void markDead(InetAddressAndPort addr,
                     EndpointState localState)

isAlive

public boolean isAlive(InetAddressAndPort endpoint)

isDeadState

public boolean isDeadState(EndpointState epState)

isSilentShutdownState

public boolean isSilentShutdownState(EndpointState epState)

isAdministrativelyInactiveState

public boolean isAdministrativelyInactiveState(EndpointState epState)

isAdministrativelyInactiveState

public boolean isAdministrativelyInactiveState(InetAddressAndPort endpoint)

applyStateLocally

public void applyStateLocally(java.util.Map<InetAddressAndPort,EndpointState> epStateMap)

start

public void start(int generationNumber)

start

public void start(int generationNbr,
                  java.util.Map<ApplicationState,VersionedValue> preloadLocalStates)

Start the gossiper with the generation number, preloading the map of application states before starting

doShadowRound

public java.util.Map<InetAddressAndPort,EndpointState> doShadowRound()

doShadowRound

public java.util.Map<InetAddressAndPort,EndpointState> doShadowRound(java.util.Set<InetAddressAndPort> peers)

Do a single 'shadow' round of gossip by retrieving endpoint states that will be stored exclusively in the map return value, instead of endpointStateMap. Used when preparing to join the ring:

• when replacing a node, to get and assume its tokens
• when joining, to check that the local host id matches any previous id for the endpoint address

Method is synchronized, as we use an in-progress flag to indicate that shadow round must be cleared again by calling maybeFinishShadowRound(InetAddressAndPort, boolean, Map). This will update endpointShadowStateMap with received values, in order to return an immutable copy to the caller of doShadowRound(). Therefor only a single shadow round execution is permitted at the same time.

Parameters:

peers - Additional peers to try gossiping with.

Returns:

endpoint states gathered during shadow round or empty map

reloadSeeds

public java.util.List<java.lang.String> reloadSeeds()

JMX interface for triggering an update of the seed node list.

Specified by:

reloadSeeds in interface GossiperMBean

getSeeds

public java.util.List<java.lang.String> getSeeds()

JMX endpoint for getting the list of seeds from the node

Specified by:

getSeeds in interface GossiperMBean

maybeInitializeLocalState

public void maybeInitializeLocalState(int generationNbr)

forceNewerGeneration

public void forceNewerGeneration()

addSavedEndpoint

public void addSavedEndpoint(InetAddressAndPort ep)

Add an endpoint we knew about previously, but whose state is unknown

addLocalApplicationState

public void addLocalApplicationState(ApplicationState applicationState,
                                     VersionedValue value)

addLocalApplicationStates

public void addLocalApplicationStates(java.util.List<Pair<ApplicationState,VersionedValue>> states)

stop

public void stop()

isEnabled

public boolean isEnabled()

sufficientForStartupSafetyCheck

public boolean sufficientForStartupSafetyCheck(java.util.Map<InetAddressAndPort,EndpointState> epStateMap)

maybeFinishShadowRound

protected void maybeFinishShadowRound(InetAddressAndPort respondent,
                                      boolean isInShadowRound,
                                      java.util.Map<InetAddressAndPort,EndpointState> epStateMap)

isInShadowRound

protected boolean isInShadowRound()

initializeUnreachableNodeUnsafe

public void initializeUnreachableNodeUnsafe(InetAddressAndPort addr)

Creates a new dead EndpointState that is empty. This is used during host replacement for edge cases where the seed notified that the endpoint was empty, so need to add such state into gossip explicitly (as empty endpoints are not gossiped outside of the shadow round). see CASSANDRA-16213

initializeNodeUnsafe

public void initializeNodeUnsafe(InetAddressAndPort addr,
                                 java.util.UUID uuid,
                                 int generationNbr)

initializeNodeUnsafe

public void initializeNodeUnsafe(InetAddressAndPort addr,
                                 java.util.UUID uuid,
                                 int netVersion,
                                 int generationNbr)

injectApplicationState

public void injectApplicationState(InetAddressAndPort endpoint,
                                   ApplicationState state,
                                   VersionedValue value)

getEndpointDowntime

public long getEndpointDowntime(java.lang.String address)
                         throws java.net.UnknownHostException

Specified by:

getEndpointDowntime in interface GossiperMBean

Throws:

java.net.UnknownHostException

getCurrentGenerationNumber

public int getCurrentGenerationNumber(java.lang.String address)
                               throws java.net.UnknownHostException

Specified by:

getCurrentGenerationNumber in interface GossiperMBean

Throws:

java.net.UnknownHostException

addExpireTimeForEndpoint

public void addExpireTimeForEndpoint(InetAddressAndPort endpoint,
                                     long expireTime)

computeExpireTime

public static long computeExpireTime()

getReleaseVersion

@Nullable
public CassandraVersion getReleaseVersion(InetAddressAndPort ep)

getReleaseVersionsWithPort

public java.util.Map<java.lang.String,java.util.List<java.lang.String>> getReleaseVersionsWithPort()

Description copied from interface: GossiperMBean

Returns each node's database release version

Specified by:

getReleaseVersionsWithPort in interface GossiperMBean

getSchemaVersion

@Nullable
public java.util.UUID getSchemaVersion(InetAddressAndPort ep)

waitToSettle

public static void waitToSettle()

waitForSchemaAgreement

public boolean waitForSchemaAgreement(long maxWait,
                                      java.util.concurrent.TimeUnit unit,
                                      java.util.function.BooleanSupplier abortCondition)

Blockingly wait for all live nodes to agree on the current schema version.

Parameters:

maxWait - maximum time to wait for schema agreement

unit - TimeUnit of maxWait

Returns:

true if agreement was reached, false if not

hasMajorVersion3Nodes

public boolean hasMajorVersion3Nodes()

Returns false only if the information about the version of each node in the cluster is available and ALL the nodes are on 4.0+ (regardless of the patch version).

isUpgradingFromVersionLowerThan

public boolean isUpgradingFromVersionLowerThan(CassandraVersion referenceVersion)

Returns true if there are nodes on version lower than the provided version

stopShutdownAndWait

public void stopShutdownAndWait(long timeout,
                                java.util.concurrent.TimeUnit unit)
                         throws java.lang.InterruptedException,
                                java.util.concurrent.TimeoutException

Throws:

java.lang.InterruptedException

java.util.concurrent.TimeoutException

getMinVersion

@Nullable
public CassandraVersion getMinVersion(long delay,
                                                java.util.concurrent.TimeUnit timeUnit)