org.apache.kafka.clients.producer.internals
Class BufferPool

java.lang.Object
  org.apache.kafka.clients.producer.internals.BufferPool

public final class BufferPool
extends java.lang.Object

A pool of ByteBuffers kept under a given memory limit. This class is fairly specific to the needs of the producer. In particular it has the following properties:

1. There is a special "poolable size" and buffers of this size are kept in a free list and recycled
2. It is fair. That is all memory is given to the longest waiting thread until it has sufficient memory. This prevents starvation or deadlock when a thread asks for a large chunk of memory and needs to block until multiple buffers are deallocated.

Constructor Summary

BufferPool(long memory, int poolableSize, boolean blockOnExhaustion, Metrics metrics, Time time)
Create a new buffer pool

Method Summary

java.nio.ByteBuffer	allocate(int size) Allocate a buffer of the given size.
long	availableMemory() the total free memory both unallocated and in the free list
void	deallocate(java.nio.ByteBuffer buffer)
void	deallocate(java.nio.ByteBuffer buffer, int size) Return buffers to the pool.
int	poolableSize() The buffer size that will be retained in the free list after use
int	queued() The number of threads blocked waiting on memory
long	totalMemory() The total memory managed by this pool
long	unallocatedMemory() Get the unallocated memory (not in the free list or in use)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BufferPool

public BufferPool(long memory,
                  int poolableSize,
                  boolean blockOnExhaustion,
                  Metrics metrics,
                  Time time)

Create a new buffer pool

Parameters:

memory - The maximum amount of memory that this buffer pool can allocate

poolableSize - The buffer size to cache in the free list rather than deallocating

blockOnExhaustion - This controls the behavior when the buffer pool is out of memory. If true the allocate(int) call will block and wait for memory to be returned to the pool. If false allocate(int) will throw an exception if the buffer is out of memory.

Method Detail

allocate

public java.nio.ByteBuffer allocate(int size)
                             throws java.lang.InterruptedException

Allocate a buffer of the given size. This method blocks if there is not enough memory and the buffer pool is configured with blocking mode.

Parameters:

size - The buffer size to allocate in bytes

Returns:

The buffer

Throws:

java.lang.InterruptedException - If the thread is interrupted while blocked

java.lang.IllegalArgumentException - if size is larger than the total memory controlled by the pool (and hence we would block forever)

BufferExhaustedException - if the pool is in non-blocking mode and size exceeds the free memory in the pool

deallocate

public void deallocate(java.nio.ByteBuffer buffer,
                       int size)

Return buffers to the pool. If they are of the poolable size add them to the free list, otherwise just mark the memory as free.

Parameters:

buffer - The buffer to return

size - The size of the buffer to mark as deallocated, note that this maybe smaller than buffer.capacity since the buffer may re-allocate itself during in-place compression

deallocate

public void deallocate(java.nio.ByteBuffer buffer)

availableMemory

public long availableMemory()

the total free memory both unallocated and in the free list

unallocatedMemory

public long unallocatedMemory()

Get the unallocated memory (not in the free list or in use)

queued

public int queued()

The number of threads blocked waiting on memory

poolableSize

public int poolableSize()

The buffer size that will be retained in the free list after use

totalMemory

public long totalMemory()

The total memory managed by this pool