Package com.cedarsoftware.util.cache

Class ThreadedLRUCacheStrategy<K,V>

java.lang.Object
com.cedarsoftware.util.cache.ThreadedLRUCacheStrategy<K,V>
Type Parameters:
K - the type of keys maintained by this cache
V - the type of mapped values
All Implemented Interfaces:
Closeable, AutoCloseable, Map<K,V>
public class ThreadedLRUCacheStrategy<K,V> extends Object implements Map<K,V>, Closeable
This class provides a thread-safe Least Recently Used (LRU) cache API that evicts the least recently used items once a threshold is met. It implements the Map interface for convenience.

Algorithm: This implementation uses pure delegation for all mutating operations (put(), putIfAbsent(), computeIfAbsent()) — they delegate directly to the underlying ConcurrentHashMap with zero eviction overhead, providing raw CHM speed for writes.

Background Eviction ("Elves"): A shared daemon thread wakes every 500ms and services all registered caches. For each cache over capacity, the elves work within a time budget (10ms per cache per cycle), performing sample-10 evictions until the cache is back at capacity or the budget is exhausted.

  • Self-limiting CPU: Max ~2% of one core (10ms per 500ms cycle per cache).
  • Adapts to cache size: Large caches with expensive iteration do fewer evictions per cycle; small caches do more.
  • No unbounded work: The elves never spend more than 10ms on a single cache per cycle.

Trade-off: The cache may temporarily exceed its capacity during burst inserts. The elves will drain it back to capacity asynchronously. Users choosing the THREADED strategy accept this approximate capacity behavior in exchange for zero-overhead writes.

Sample-10 Eviction: Instead of sorting all entries (O(n log n)), we sample 10 entries and evict the oldest one. This provides ~95-99% accuracy compared to true LRU (based on Redis research) with O(1) cost.

Probabilistic Timestamp Updates: To minimize overhead, timestamps are updated probabilistically (~12.5% of accesses). This dramatically reduces the cost of volatile writes and System.nanoTime() calls while maintaining approximate LRU behavior. Frequently accessed entries will still have their timestamps updated regularly.

The Threaded strategy allows for O(1) access for get(), put(), and remove() without blocking. It uses ConcurrentHashMapNullSafe internally for null key/value support.

LRUCache supports null for both key and value.

Architecture: All ThreadedLRUCacheStrategy instances share a single cleanup thread that runs every 500ms. Each cache registers itself via a WeakReference, allowing garbage collection of unused caches.

Author:
John DeRegnaucourt ([email protected])
Copyright (c) Cedar Software LLC

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

License

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

  • Constructor Details

    • ThreadedLRUCacheStrategy

      public ThreadedLRUCacheStrategy(int capacity)
      Create a ThreadedLRUCacheStrategy with the specified capacity.

      All mutating operations (put(), putIfAbsent(), computeIfAbsent()) delegate directly to the underlying ConcurrentHashMap with zero eviction overhead. A background cleanup thread ("elves") runs every 500ms to drain surplus entries using time-budgeted sample-10 eviction.

      Parameters:
      capacity - int maximum size for the LRU cache.
      Throws:
      IllegalArgumentException - if capacity is less than 1

    • ThreadedLRUCacheStrategy

      @Deprecated public ThreadedLRUCacheStrategy(int capacity, int cleanupDelayMillis)
      Deprecated.
      Use ThreadedLRUCacheStrategy(int) instead.
      Create a ThreadedLRUCacheStrategy with the specified capacity.

      Note: The cleanupDelayMillis parameter is deprecated and ignored.

      Parameters:
      capacity - int maximum size for the LRU cache.
      cleanupDelayMillis - ignored (formerly: milliseconds before scheduling cleanup)

  • Method Details

    • shutdown

      public void shutdown()
      Shuts down this cache, removing it from the shared cleanup task.

    • forceCleanup

      public void forceCleanup()
      Forces an immediate cleanup of this cache (for testing).

    • shutdownScheduler

      public static boolean shutdownScheduler()
      Shuts down the shared cleanup scheduler used by all ThreadedLRUCacheStrategy instances.
      Returns:
      true if the scheduler terminated cleanly, false if it timed out or was interrupted

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable

    • getCapacity

      public int getCapacity()
      Returns:
      the maximum number of entries in the cache.

    • get

      public V get(Object key)
      Specified by:
      get in interface Map<K,V>

    • put

      public V put(K key, V value)
      Pure delegation to ConcurrentHashMap — zero eviction overhead. The background "elves" handle all eviction asynchronously.
      Specified by:
      put in interface Map<K,V>

    • putAll

      public void putAll(Map<? extends K,? extends V> m)
      Specified by:
      putAll in interface Map<K,V>

    • isEmpty

      public boolean isEmpty()
      Specified by:
      isEmpty in interface Map<K,V>

    • remove

      public V remove(Object key)
      Specified by:
      remove in interface Map<K,V>

    • computeIfAbsent

      public V computeIfAbsent(K key, Function<? super K,? extends V> mappingFunction)
      Pure delegation to ConcurrentHashMap — zero eviction overhead. The background "elves" handle all eviction asynchronously.
      Specified by:
      computeIfAbsent in interface Map<K,V>

    • putIfAbsent

      public V putIfAbsent(K key, V value)
      Pure delegation to ConcurrentHashMap — zero eviction overhead. The background "elves" handle all eviction asynchronously.
      Specified by:
      putIfAbsent in interface Map<K,V>

    • clear

      public void clear()
      Specified by:
      clear in interface Map<K,V>

    • size

      public int size()
      Specified by:
      size in interface Map<K,V>

    • containsKey

      public boolean containsKey(Object key)
      Specified by:
      containsKey in interface Map<K,V>

    • containsValue

      public boolean containsValue(Object value)
      Specified by:
      containsValue in interface Map<K,V>

    • entrySet

      public Set<Map.Entry<K,V>> entrySet()
      Specified by:
      entrySet in interface Map<K,V>

    • keySet

      public Set<K> keySet()
      Specified by:
      keySet in interface Map<K,V>

    • values

      public Collection<V> values()
      Specified by:
      values in interface Map<K,V>

    • equals

      public boolean equals(Object o)
      Specified by:
      equals in interface Map<K,V>
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Specified by:
      hashCode in interface Map<K,V>
      Overrides:
      hashCode in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object