net.spy.memcached

Class MemcachedConnection

java.lang.Object

java.lang.Thread

net.spy.memcached.compat.SpyThread

net.spy.memcached.MemcachedConnection

All Implemented Interfaces:

java.lang.Runnable, ClusterConfigurationObserver, ConfigurationObserver

public class MemcachedConnection
extends SpyThread
implements ClusterConfigurationObserver

Connection to a cluster of memcached servers. MemcachedConnection also acts as an observer for cluster configuration changes. In the mode ClientMode.Dynamic, the ConfigurationPoller notifies the observers when there is change in cluster configuration.

Nested Class Summary

Nested classes/interfaces inherited from class java.lang.Thread

java.lang.Thread.State, java.lang.Thread.UncaughtExceptionHandler

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.concurrent.ConcurrentLinkedQueue<MemcachedNode>	addedQueue AddedQueue is used to track the QueueAttachments for which operations have recently been queued.
protected java.util.concurrent.locks.ReentrantLock	conditionLock
protected FailureMode	failureMode The configured FailureMode.
protected boolean	isInitialClusterConfigApplied
protected NodeLocator	locator The NodeLocator to use for this connection.
protected java.util.concurrent.locks.ReentrantLock	lockForNodeUpdates
protected MetricCollector	metrics The MetricCollector to accumulate metrics (or dummy).
protected MetricType	metricType The current type of metrics to collect.
protected java.util.List<NodeEndPoint>	newEndPoints
protected java.util.List<NodeEndPoint>	nodesToAdd
protected java.util.List<MemcachedNode>	nodesToDelete
protected java.util.concurrent.ConcurrentLinkedQueue<MemcachedNode>	nodesToShutdown Holds all nodes that are scheduled for shutdown.
protected java.util.concurrent.locks.Condition	nodeUpdateCondition
protected boolean	running True if not shutting down or shut down.
protected java.nio.channels.Selector	selector Holds the current Selector to use.
protected boolean	shutDown If the connection is alread shut down or shutting down.

Fields inherited from class java.lang.Thread

MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY

Constructor Summary

Constructors

Constructor and Description
MemcachedConnection(int bufSize, ConnectionFactory f, java.util.List<java.net.InetSocketAddress> socketAddressList, java.util.Collection<ConnectionObserver> obs, FailureMode fm, OperationFactory opfactory) Construct a MemcachedConnection.

Method Summary

Modifier and Type	Method and Description
boolean	addObserver(ConnectionObserver obs) Add a connection observer.
protected void	addOperation(java.net.InetSocketAddress addr, Operation o)
protected void	addOperation(MemcachedNode node, Operation o) Enqueue an operation on the given node.
protected void	addOperation(java.lang.String key, Operation o) Add an operation to a connection identified by the given key.
void	addOperations(java.util.Map<MemcachedNode,Operation> ops) Enqueue the given list of operations on each handling node.
java.util.concurrent.CountDownLatch	broadcastOperation(BroadcastOpFactory of) Broadcast an operation to all nodes.
java.util.concurrent.CountDownLatch	broadcastOperation(BroadcastOpFactory of, java.util.Collection<MemcachedNode> nodes) Broadcast an operation to a collection of nodes.
protected void	checkState() Check to see if this connection is shutting down.
java.lang.String	connectionsStatus() Construct a String containing information about all nodes and their state.
protected MemcachedNode	createConnection(NodeEndPoint endPoint)
protected java.util.List<MemcachedNode>	createConnections(java.util.Collection<NodeEndPoint> endPoints)
void	enqueueOperation(java.net.InetSocketAddress addr, Operation o)
void	enqueueOperation(java.lang.String key, Operation o) Enqueue the given Operation with the used key.
NodeLocator	getLocator() Returns the NodeLocator in use for this connection.
void	handleIO() Handle all IO that flows through the connection.
protected void	handleRetryInformation(byte[] retryMessage) Optionally handle retry (NOT_MY_VBUKET) responses.
protected void	handleWokenUpSelector() Helper method which gets called if the selector is woken up because of the timeout setting, if has been interrupted or if happens during regular write operation phases.
void	insertOperation(MemcachedNode node, Operation o) Insert an operation on the given node to the beginning of the queue.
boolean	isShutDown() Returns whether the connection is shut down or not.
void	notifyUpdate(ClusterConfiguration clusterConfiguration) The publisher calls all the subscribers through this method.
static void	opSucceeded(Operation op) Reset the timeout counter for the given handling node.
static void	opTimedOut(Operation op) Increase the timeout counter for the given handling node.
protected void	queueReconnect(MemcachedNode node) Enqueue the given MemcachedNode for reconnect.
void	redistributeOperation(Operation op) Redistribute the given operation to (potentially) other nodes.
void	redistributeOperations(java.util.Collection<Operation> ops) Redistribute the given list of operations to (potentially) other nodes.
protected void	registerMetrics() Register Metrics for collection.
boolean	removeObserver(ConnectionObserver obs) Remove a connection observer.
void	retryOperation(Operation op) Add a operation to the retry queue.
void	run() Handle IO as long as the application is running.
void	shutdown() Shut down all connections and do not accept further incoming ops.
java.lang.String	toString()
void	waitForConfigChangeApplied() Wait for the configuration change applied.
void	waitForInitialConfigApplied()

Methods inherited from class net.spy.memcached.compat.SpyThread

getLogger

Methods inherited from class java.lang.Thread

activeCount, checkAccess, clone, countStackFrames, currentThread, destroy, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, start, stop, stop, suspend, yield

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

shutDown

protected volatile boolean shutDown

If the connection is alread shut down or shutting down.

selector

protected java.nio.channels.Selector selector

Holds the current Selector to use.

locator

protected NodeLocator locator

The NodeLocator to use for this connection.

failureMode

protected final FailureMode failureMode

The configured FailureMode.

addedQueue

protected final java.util.concurrent.ConcurrentLinkedQueue<MemcachedNode> addedQueue

AddedQueue is used to track the QueueAttachments for which operations have recently been queued.

running

protected volatile boolean running

True if not shutting down or shut down.

nodesToAdd

protected java.util.List<NodeEndPoint> nodesToAdd

nodesToDelete

protected java.util.List<MemcachedNode> nodesToDelete

lockForNodeUpdates

protected final java.util.concurrent.locks.ReentrantLock lockForNodeUpdates

conditionLock

protected final java.util.concurrent.locks.ReentrantLock conditionLock

nodeUpdateCondition

protected final java.util.concurrent.locks.Condition nodeUpdateCondition

isInitialClusterConfigApplied

protected boolean isInitialClusterConfigApplied

nodesToShutdown

protected final java.util.concurrent.ConcurrentLinkedQueue<MemcachedNode> nodesToShutdown

Holds all nodes that are scheduled for shutdown.

newEndPoints

protected final java.util.List<NodeEndPoint> newEndPoints

metrics

protected final MetricCollector metrics

The MetricCollector to accumulate metrics (or dummy).

metricType

protected final MetricType metricType

The current type of metrics to collect.

Constructor Detail

MemcachedConnection

public MemcachedConnection(int bufSize,
                           ConnectionFactory f,
                           java.util.List<java.net.InetSocketAddress> socketAddressList,
                           java.util.Collection<ConnectionObserver> obs,
                           FailureMode fm,
                           OperationFactory opfactory)
                    throws java.io.IOException

Construct a MemcachedConnection.

Parameters:

bufSize - the size of the buffer used for reading from the server

f - the factory that will provide an operation queue

socketAddressList - the addresses of the servers to connect to

obs - the initial observers to add.

fm - the failure mode to use.

opfactory - the operation factory.

Throws:

java.io.IOException - if a connection attempt fails early

Method Detail

waitForInitialConfigApplied

public void waitForInitialConfigApplied()

waitForConfigChangeApplied

public void waitForConfigChangeApplied()

Description copied from interface: ClusterConfigurationObserver

Wait for the configuration change applied. This is invoked whenever the observers get notified except configuration change during the client initialization.

Specified by:

waitForConfigChangeApplied in interface ClusterConfigurationObserver

notifyUpdate

public void notifyUpdate(ClusterConfiguration clusterConfiguration)

Description copied from interface: ClusterConfigurationObserver

The publisher calls all the subscribers through this method. This is invoked whenever there is change in cluster configuration data.

Specified by:

notifyUpdate in interface ClusterConfigurationObserver

Parameters:

clusterConfiguration - - The parameter contains the latest information about the cluster.

registerMetrics

protected void registerMetrics()

Register Metrics for collection. Note that these Metrics may or may not take effect, depending on the MetricCollector implementation. This can be controlled from the DefaultConnectionFactory.

createConnection

protected MemcachedNode createConnection(NodeEndPoint endPoint)
                                  throws java.io.IOException

Throws:

java.io.IOException

createConnections

protected java.util.List<MemcachedNode> createConnections(java.util.Collection<NodeEndPoint> endPoints)
                                                   throws java.io.IOException

Throws:

java.io.IOException

handleIO

public void handleIO()
              throws java.io.IOException

Handle all IO that flows through the connection. This method is called in an endless loop, listens on NIO selectors and dispatches the underlying read/write calls if needed.

Throws:

java.io.IOException

handleWokenUpSelector

protected void handleWokenUpSelector()

Helper method which gets called if the selector is woken up because of the timeout setting, if has been interrupted or if happens during regular write operation phases.

This method can be overriden by child implementations to handle custom behavior on a manually woken selector, like sending pings through the channels to make sure they are alive.

Note that there is no guarantee that this method is at all or in the regular interval called, so all overriding implementations need to take that into account. Also, it needs to take into account that it may be called very often under heavy workloads, so it should not perform extensive tasks in the same thread.

addObserver

public boolean addObserver(ConnectionObserver obs)

Add a connection observer.

Returns:

whether the observer was successfully added.

removeObserver

public boolean removeObserver(ConnectionObserver obs)

Remove a connection observer.

Returns:

true if the observer existed and now doesn't.

handleRetryInformation

protected void handleRetryInformation(byte[] retryMessage)

Optionally handle retry (NOT_MY_VBUKET) responses. This method can be overridden in subclasses to handle the content of the retry message appropriately.

Parameters:

retryMessage - the body of the retry message.

queueReconnect

protected void queueReconnect(MemcachedNode node)

Enqueue the given MemcachedNode for reconnect.

Parameters:

node - the node to reconnect.

redistributeOperations

public void redistributeOperations(java.util.Collection<Operation> ops)

Redistribute the given list of operations to (potentially) other nodes. Note that operations can only be redistributed if they have not been cancelled already, timed out already or do not have definite targets (a key).

Parameters:

ops - the operations to redistribute.

redistributeOperation

public void redistributeOperation(Operation op)

Redistribute the given operation to (potentially) other nodes. Note that operations can only be redistributed if they have not been cancelled already, timed out already or do not have definite targets (a key).

Parameters:

op - the operation to redistribute.

getLocator

public NodeLocator getLocator()

Returns the NodeLocator in use for this connection.

Returns:

the current NodeLocator.

enqueueOperation

public void enqueueOperation(java.lang.String key,
                             Operation o)

Enqueue the given Operation with the used key.

Parameters:

key - the key to use.

o - the Operation to enqueue.

enqueueOperation

public void enqueueOperation(java.net.InetSocketAddress addr,
                             Operation o)

addOperation

protected void addOperation(java.lang.String key,
                            Operation o)

Add an operation to a connection identified by the given key. If the MemcachedNode is active or the FailureMode is set to retry, the primary node will be used for that key. If the primary node is not available and the FailureMode cancel is used, the operation will be cancelled without further retry. For any other FailureMode mechanisms (Redistribute), another possible node is used (only if its active as well). If no other active node could be identified, the original primary node is used and retried.

Parameters:

key - the key the operation is operating upon.

o - the operation to add.

addOperation

protected void addOperation(java.net.InetSocketAddress addr,
                            Operation o)

insertOperation

public void insertOperation(MemcachedNode node,
                            Operation o)

Insert an operation on the given node to the beginning of the queue.

Parameters:

node - the node where to insert the Operation.

o - the operation to insert.

addOperation

protected void addOperation(MemcachedNode node,
                            Operation o)

Enqueue an operation on the given node.

Parameters:

node - the node where to enqueue the Operation.

o - the operation to add.

addOperations

public void addOperations(java.util.Map<MemcachedNode,Operation> ops)

Enqueue the given list of operations on each handling node.

Parameters:

ops - the operations for each node.

broadcastOperation

public java.util.concurrent.CountDownLatch broadcastOperation(BroadcastOpFactory of)

Broadcast an operation to all nodes.

Returns:

a CountDownLatch that will be counted down when the operations are complete.

broadcastOperation

public java.util.concurrent.CountDownLatch broadcastOperation(BroadcastOpFactory of,
                                                              java.util.Collection<MemcachedNode> nodes)

Broadcast an operation to a collection of nodes.

Returns:

a CountDownLatch that will be counted down when the operations are complete.

shutdown

public void shutdown()
              throws java.io.IOException

Shut down all connections and do not accept further incoming ops.

Throws:

java.io.IOException

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Thread

connectionsStatus

public java.lang.String connectionsStatus()

Construct a String containing information about all nodes and their state.

Returns:

a stringified representation of the connection status.

opTimedOut

public static void opTimedOut(Operation op)

Increase the timeout counter for the given handling node.

Parameters:

op - the operation to grab the node from.

opSucceeded

public static void opSucceeded(Operation op)

Reset the timeout counter for the given handling node.

Parameters:

op - the operation to grab the node from.

checkState

protected void checkState()

Check to see if this connection is shutting down.

Throws:

java.lang.IllegalStateException - when shutting down.

run

public void run()

Handle IO as long as the application is running.

Specified by:

run in interface java.lang.Runnable

Overrides:

run in class java.lang.Thread

isShutDown

public boolean isShutDown()

Returns whether the connection is shut down or not.

Returns:

true if the connection is shut down, false otherwise.

retryOperation

public void retryOperation(Operation op)

Add a operation to the retry queue. If the retry queue size is bounded and the size of the queue is reaching that boundary, the operation is cancelled rather than added to the retry queue.

Parameters:

op - the operation to retry.