An abstract asynchronous function to enrich a data stream with non-stream data.
An abstract asynchronous function to enrich a data stream with non-stream data. This
class relies on guava's CacheBuilder to load and cache the enrichment data. The
default cache loader assumes the cache key is a uri of a json api endpoint which it
loads asynchronously and converts to the cache value type (CV). You can provide your
own cache loader to load enrichment data in some other way. If you use the default
loader, note you must implicitly provide a circe EntityDecoder to convert the
json api result body to the cache value type.
The behavior of the function can be parameterized with flink config values for
$configPrefix.num.threads (size of thread pool, Int)
$configPrefix.cache.concurrency.level (Int)
$configPrefix.cache.max.size (max number of records in cache, Int)
$configPrefix.cache.expire.after (Duration)
The cache always uses weak keys, allowing for aggressive garbage collection of
unused values.
Subclasses must implement the getCacheKey() and enrichEvent() methods.
An abstract asynchronous function to enrich a data stream with non-stream data. This class relies on guava's CacheBuilder to load and cache the enrichment data. The default cache loader assumes the cache key is a uri of a json api endpoint which it loads asynchronously and converts to the cache value type (CV). You can provide your own cache loader to load enrichment data in some other way. If you use the default loader, note you must implicitly provide a circe EntityDecoder to convert the json api result body to the cache value type.
The behavior of the function can be parameterized with flink config values for
The cache always uses weak keys, allowing for aggressive garbage collection of unused values.
Subclasses must implement the getCacheKey() and enrichEvent() methods.
the input stream element type
the enriched stream output element type
the cache value type