kotlin-spark-api_3.0.2_2.12/org.jetbrains.kotlinx.spark.api/withCached

withCached

inline fun <T, R> <Error class: unknown class><T>.withCached(blockingUnpersist: Boolean = false, executeOnCached: <Error class: unknown class><T>.() -> R): R

This function creates block, where one can call any further computations on already cached dataset Data will be unpersisted automatically at the end of computation

it may be useful in many situations, for example, when one needs to write data to several targets

ds.withCached {
  write()
     .also { it.orc("First destination") }
     .also { it.avro("Second destination") }
}

Return

result of block execution for further usage. It may be anything including source or new dataset

Parameters

blockingUnpersist

if execution should be blocked until everything persisted will be deleted

executeOnCached

Block which should be executed on cached dataset.