io.grpc

Class LoadBalancer

java.lang.Object

io.grpc.LoadBalancer

Direct Known Subclasses:

LoadBalancer2

@ExperimentalApi(value="https://github.com/grpc/grpc-java/issues/1771")
 @NotThreadSafe
public abstract class LoadBalancer
extends Object

A pluggable component that receives resolved addresses from NameResolver and provides the channel a usable subchannel when asked.

Overview

A LoadBalancer typically implements three interfaces:

1. LoadBalancer is the main interface. All methods on it are invoked sequentially from the Channel Executor. It receives the results from the NameResolver, updates of subchannels' connectivity states, and the channel's request for the LoadBalancer to shutdown.
2. SubchannelPicker does the actual load-balancing work. It selects a Subchannel for each new RPC.
3. Factory creates a new LoadBalancer instance.

Helper is implemented by gRPC library and provided to Factory. It provides functionalities that a LoadBalancer implementation would typically need.

Channel Executor

Channel Executor is an internal executor of the channel, which is used to serialize all the callback methods on the LoadBalancer interface, thus the balancer implementation doesn't need to worry about synchronization among them. However, the actual thread of the Channel Executor is typically the network thread, thus the following rules must be followed to prevent blocking or even dead-locking in a network

1. Never block in Channel Executor. The callback methods must return quickly. Examples or work that must be avoided: CPU-intensive calculation, waiting on synchronization primitives, blocking I/O, blocking RPCs, etc.
2. Avoid calling into other components with lock held. Channel Executor may run callbacks under a lock, e.g., the transport lock of OkHttp. If your LoadBalancer has a lock, holds the lock in a callback method (e.g., handleSubchannelState()) while calling into another class that may involve locks, be cautious of deadlock. Generally you wouldn't need any locking in the LoadBalancer.

Helper.runSerialized() allows you to schedule a task to be run in the Channel Executor.

The canonical implementation pattern

A LoadBalancer keeps states like the latest addresses from NameResolver, the Subchannel(s) and their latest connectivity states. These states are mutated within the Channel Executor.

A typical SubchannelPicker holds a snapshot of these states. It may have its own states, e.g., a picker from a round-robin load-balancer may keep a pointer to the next Subchannel, which are typically mutated by multiple threads. The picker should only mutate its own state, and should not mutate or re-acquire the states of the LoadBalancer. This way the picker only needs to synchronize its own states, which is typically trivial to implement.

When the LoadBalancer states changes, e.g., Subchannels has become or stopped being READY, and we want subsequent RPCs to use the latest list of READY Subchannels, LoadBalancer would create a new picker, which holds a snapshot of the latest Subchannel list. Refer to the javadoc of handleSubchannelState() how to do this properly.

No synchronization should be necessary between LoadBalancer and its pickers if you follow the pattern above. It may be possible to implement in a different way, but that would usually result in more complicated threading.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	LoadBalancer.Factory
static class	LoadBalancer.Helper Provides essentials for LoadBalancer implementations.
static class	LoadBalancer.PickResult A balancing decision made by SubchannelPicker for an RPC.
static class	LoadBalancer.PickSubchannelArgs Provides arguments for a LoadBalancer.SubchannelPicker.pickSubchannel( LoadBalancer.PickSubchannelArgs).
static class	LoadBalancer.Subchannel A logical connection to a server, or a group of equivalent servers represented by an EquivalentAddressGroup.
static class	LoadBalancer.SubchannelPicker The main balancing logic.

Constructor Summary

Constructors

Constructor and Description
LoadBalancer()

Method Summary

Modifier and Type	Method and Description
abstract void	handleNameResolutionError(Status error) Handles an error from the name resolution system.
abstract void	handleResolvedAddresses(List<ResolvedServerInfoGroup> servers, Attributes attributes) Handles newly resolved server groups and metadata attributes from name resolution system.
abstract void	handleSubchannelState(LoadBalancer.Subchannel subchannel, ConnectivityStateInfo stateInfo) Handles a state change on a Subchannel.
abstract void	shutdown() The channel asks the load-balancer to shutdown.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LoadBalancer

public LoadBalancer()

Method Detail

handleResolvedAddresses

public abstract void handleResolvedAddresses(List<ResolvedServerInfoGroup> servers,
                                             Attributes attributes)

Handles newly resolved server groups and metadata attributes from name resolution system. servers contained in ResolvedServerInfoGroup should be considered equivalent but may be flattened into a single list if needed.

Implementations should not modify the given servers.

Parameters:

servers - the resolved server addresses, never empty.

attributes - extra metadata from naming system.

handleNameResolutionError

public abstract void handleNameResolutionError(Status error)

Handles an error from the name resolution system.

Parameters:

error - a non-OK status

handleSubchannelState

public abstract void handleSubchannelState(LoadBalancer.Subchannel subchannel,
                                           ConnectivityStateInfo stateInfo)

Handles a state change on a Subchannel.

The initial state of a Subchannel is IDLE. You won't get a notification for the initial IDLE state.

If the new state is not SHUTDOWN, this method should create a new picker and call Helper.updatePicker(). Failing to do so may result in unnecessary delays of RPCs. Please refer to PickResult.withSubchannel()'s javadoc for more information.

SHUTDOWN can only happen in two cases. One is that LoadBalancer called LoadBalancer.Subchannel.shutdown() earlier, thus it should have already discarded this Subchannel. The other is that Channel is doing a forced shutdown or has already terminated, thus there won't be further requests to LoadBalancer. Therefore, SHUTDOWN can be safely ignored.

Parameters:

subchannel - the involved Subchannel

stateInfo - the new state

shutdown

public abstract void shutdown()

The channel asks the load-balancer to shutdown. No more callbacks will be called after this method. The implementation should shutdown all Subchannels and OOB channels, and do any other cleanup as necessary.