public abstract class AbstractMessageHandler extends io.netty.channel.ChannelInboundHandlerAdapter implements FrameDecoder.FrameProcessor
FrameDecoder).
Handles small and large messages, corruption, flow control, dispatch of message processing to a suitable
consumer.
# Interaction with FrameDecoder
An AbstractMessageHandler implementation sits on top of a FrameDecoder in the Netty pipeline,
and is tightly coupled with it.
FrameDecoder decodes inbound frames and relies on a supplied FrameDecoder.FrameProcessor to act on them.
AbstractMessageHandler provides two implementations of that interface:
- #process(Frame) is the default, primary processor, and is expected to be implemented by subclasses
- UpToOneMessageFrameProcessor, supplied to the decoder when the handler is reactivated after being
put in waiting mode due to lack of acquirable reserve memory capacity permits
Return value of FrameProcessor#process(Frame) determines whether the decoder should keep processing
frames (if true is returned) or stop until explicitly reactivated (if false is). To reactivate
the decoder (once notified of available resource permits), FrameDecoder.reactivate() is invoked.
# Frames
AbstractMessageHandler operates on frames of messages, and there are several kinds of them:
1. FrameDecoder.IntactFrame that are contained. As names suggest, these contain one or multiple fully contained
messages believed to be uncorrupted. Guaranteed to not contain an part of an incomplete message.
See #processFrameOfContainedMessages(ShareableBytes, Limit, Limit).
2. FrameDecoder.IntactFrame that are NOT contained. These are uncorrupted parts of a large message split over multiple
parts due to their size. Can represent first or subsequent frame of a large message.
See #processFirstFrameOfLargeMessage(IntactFrame, Limit, Limit) and
#processSubsequentFrameOfLargeMessage(Frame).
3. FrameDecoder.CorruptFrame with corrupt header. These are unrecoverable, and force a connection to be dropped.
4. FrameDecoder.CorruptFrame with a valid header, but corrupt payload. These can be either contained or uncontained.
- contained frames with corrupt payload can be gracefully dropped without dropping the connection
- uncontained frames with corrupt payload can be gracefully dropped unless they represent the first
frame of a new large message, as in that case we don't know how many bytes to skip
See #processCorruptFrame(CorruptFrame).
Fundamental frame invariants:
1. A contained frame can only have fully-encapsulated messages - 1 to n, that don't cross frame boundaries
2. An uncontained frame can hold a part of one message only. It can NOT, say, contain end of one large message
and a beginning of another one. All the bytes in an uncontained frame always belong to a single message.
# Small vs large messages
A single handler is equipped to process both small and large messages, potentially interleaved, but the logic
differs depending on size. Small messages are deserialized in place, and then handed off to an appropriate
thread pool for processing. Large messages accumulate frames until completion of a message, then hand off
the untouched frames to the correct thread pool for the verb to be deserialized there and immediately processed.
See AbstractMessageHandler.LargeMessage and subclasses for concrete AbstractMessageHandler implementations for details
of the large-message accumulating state-machine, and ProcessMessage and its inheritors for the differences
in execution.
# Flow control (backpressure)
To prevent message producers from overwhelming and bringing nodes down with more inbound messages that
can be processed in a timely manner, AbstractMessageHandler provides support for implementations to
provide their own flow control policy.
Before we attempt to process a message fully, we first infer its size from the stream. This inference is
delegated to implementations as the encoding of the message size is protocol specific. Having assertained
the size of the incoming message, we then attempt to acquire the corresponding number of memory permits.
If we succeed, then we move on actually process the message. If we fail, the frame decoder deactivates
until sufficient permits are released for the message to be processed and the handler is activated again.
Permits are released back once the message has been fully processed - the definition of which is again
delegated to the concrete implementations.
Every connection has an exclusive number of permits allocated to it. In addition to it, there is a per-endpoint
reserve capacity and a global reserve capacity ResourceLimits.Limit, shared between all connections from the same host
and all connections, respectively. So long as long as the handler stays within its exclusive limit, it doesn't
need to tap into reserve capacity.
If tapping into reserve capacity is necessary, but the handler fails to acquire capacity from either
endpoint of global reserve (and it needs to acquire from both), the handler and its frame decoder become
inactive and register with a AbstractMessageHandler.WaitQueue of the appropriate type, depending on which of the reserves
couldn't be tapped into. Once enough messages have finished processing and had their permits released back
to the reserves, AbstractMessageHandler.WaitQueue will reactivate the sleeping handlers and they'll resume processing frames.
The reason we 'split' reserve capacity into two limits - endpoing and global - is to guarantee liveness, and
prevent single endpoint's connections from taking over the whole reserve, starving other connections.
One permit per byte of serialized message gets acquired. When inflated on-heap, each message will occupy more
than that, necessarily, but despite wide variance, it's a good enough proxy that correlates with on-heap footprint.| Modifier and Type | Class and Description |
|---|---|
protected class |
AbstractMessageHandler.LargeMessage<H> |
static interface |
AbstractMessageHandler.OnHandlerClosed |
static class |
AbstractMessageHandler.WaitQueue
A special-purpose wait queue to park inbound message handlers that failed to allocate
reserve capacity for a message in.
|
| Modifier and Type | Field and Description |
|---|---|
protected io.netty.channel.Channel |
channel |
protected long |
corruptFramesRecovered |
protected long |
corruptFramesUnrecovered |
protected FrameDecoder |
decoder |
protected ResourceLimits.Limit |
endpointReserveCapacity |
protected AbstractMessageHandler.WaitQueue |
endpointWaitQueue |
protected ResourceLimits.Limit |
globalReserveCapacity |
protected AbstractMessageHandler.WaitQueue |
globalWaitQueue |
protected AbstractMessageHandler.LargeMessage<?> |
largeMessage |
protected int |
largeThreshold |
protected AbstractMessageHandler.OnHandlerClosed |
onClosed |
protected long |
queueCapacity |
protected long |
receivedBytes |
protected long |
receivedCount |
protected long |
throttledCount |
protected long |
throttledNanos |
| Constructor and Description |
|---|
AbstractMessageHandler(FrameDecoder decoder,
io.netty.channel.Channel channel,
int largeThreshold,
long queueCapacity,
ResourceLimits.Limit endpointReserveCapacity,
ResourceLimits.Limit globalReserveCapacity,
AbstractMessageHandler.WaitQueue endpointWaitQueue,
AbstractMessageHandler.WaitQueue globalWaitQueue,
AbstractMessageHandler.OnHandlerClosed onClosed) |
| Modifier and Type | Method and Description |
|---|---|
protected ResourceLimits.Outcome |
acquireCapacity(ResourceLimits.Limit endpointReserve,
ResourceLimits.Limit globalReserve,
int bytes) |
protected boolean |
acquireCapacity(ResourceLimits.Limit endpointReserve,
ResourceLimits.Limit globalReserve,
int bytes,
long currentTimeNanos,
long expiresAtNanos)
Try to acquire permits for the inbound message.
|
void |
channelInactive(io.netty.channel.ChannelHandlerContext ctx) |
void |
channelRead(io.netty.channel.ChannelHandlerContext ctx,
java.lang.Object msg) |
protected abstract void |
fatalExceptionCaught(java.lang.Throwable t) |
void |
handlerAdded(io.netty.channel.ChannelHandlerContext ctx) |
protected abstract java.lang.String |
id() |
boolean |
process(FrameDecoder.Frame frame)
Frame processor that the frames should be handed off to.
|
protected abstract void |
processCorruptFrame(FrameDecoder.CorruptFrame frame) |
protected abstract boolean |
processFirstFrameOfLargeMessage(FrameDecoder.IntactFrame frame,
ResourceLimits.Limit endpointReserve,
ResourceLimits.Limit globalReserve) |
protected abstract boolean |
processOneContainedMessage(ShareableBytes bytes,
ResourceLimits.Limit endpointReserve,
ResourceLimits.Limit globalReserve) |
protected boolean |
processSubsequentFrameOfLargeMessage(FrameDecoder.Frame frame) |
void |
releaseCapacity(int bytes) |
protected void |
releaseProcessedCapacity(int size,
Message.Header header)
Invoked to release capacity for a message that has been fully, successfully processed.
|
channelActive, channelReadComplete, channelRegistered, channelUnregistered, channelWritabilityChanged, exceptionCaught, userEventTriggeredensureNotSharable, handlerRemoved, isSharableprotected final FrameDecoder decoder
protected final io.netty.channel.Channel channel
protected final int largeThreshold
protected AbstractMessageHandler.LargeMessage<?> largeMessage
protected final long queueCapacity
protected final ResourceLimits.Limit endpointReserveCapacity
protected final AbstractMessageHandler.WaitQueue endpointWaitQueue
protected final ResourceLimits.Limit globalReserveCapacity
protected final AbstractMessageHandler.WaitQueue globalWaitQueue
protected final AbstractMessageHandler.OnHandlerClosed onClosed
protected long corruptFramesRecovered
protected long corruptFramesUnrecovered
protected long receivedCount
protected long receivedBytes
protected long throttledCount
protected long throttledNanos
public AbstractMessageHandler(FrameDecoder decoder, io.netty.channel.Channel channel, int largeThreshold, long queueCapacity, ResourceLimits.Limit endpointReserveCapacity, ResourceLimits.Limit globalReserveCapacity, AbstractMessageHandler.WaitQueue endpointWaitQueue, AbstractMessageHandler.WaitQueue globalWaitQueue, AbstractMessageHandler.OnHandlerClosed onClosed)
public void channelRead(io.netty.channel.ChannelHandlerContext ctx,
java.lang.Object msg)
channelRead in interface io.netty.channel.ChannelInboundHandlerchannelRead in class io.netty.channel.ChannelInboundHandlerAdapterpublic void handlerAdded(io.netty.channel.ChannelHandlerContext ctx)
handlerAdded in interface io.netty.channel.ChannelHandlerhandlerAdded in class io.netty.channel.ChannelHandlerAdapterpublic boolean process(FrameDecoder.Frame frame) throws java.io.IOException
FrameDecoder.FrameProcessorprocess in interface FrameDecoder.FrameProcessorjava.io.IOExceptionprotected abstract boolean processOneContainedMessage(ShareableBytes bytes, ResourceLimits.Limit endpointReserve, ResourceLimits.Limit globalReserve) throws java.io.IOException
java.io.IOExceptionprotected abstract boolean processFirstFrameOfLargeMessage(FrameDecoder.IntactFrame frame, ResourceLimits.Limit endpointReserve, ResourceLimits.Limit globalReserve) throws java.io.IOException
java.io.IOExceptionprotected boolean processSubsequentFrameOfLargeMessage(FrameDecoder.Frame frame)
protected abstract void processCorruptFrame(FrameDecoder.CorruptFrame frame) throws Crc.InvalidCrc
Crc.InvalidCrcprotected abstract void fatalExceptionCaught(java.lang.Throwable t)
protected boolean acquireCapacity(ResourceLimits.Limit endpointReserve, ResourceLimits.Limit globalReserve, int bytes, long currentTimeNanos, long expiresAtNanos)
protected ResourceLimits.Outcome acquireCapacity(ResourceLimits.Limit endpointReserve, ResourceLimits.Limit globalReserve, int bytes)
public void releaseCapacity(int bytes)
protected void releaseProcessedCapacity(int size,
Message.Header header)
releaseCapacity(int), but is necessary for the verifier
to be able to delay capacity release for backpressure testing.public void channelInactive(io.netty.channel.ChannelHandlerContext ctx)
channelInactive in interface io.netty.channel.ChannelInboundHandlerchannelInactive in class io.netty.channel.ChannelInboundHandlerAdapterprotected abstract java.lang.String id()
Copyright © 2009-2021 The Apache Software Foundation