dynamic

Type Members

1. implicit final class DynamicIoSCollection[T] extends AnyVal

   

   Enhanced version of SCollection with dynamic destinations methods.

2. case class FileDestinations(path: String, default: String = "default", windowedWrites: Boolean = false, numShards: Int = 0) extends Product with Serializable

   

   Dynamic file destinations settings.

   Dynamic file destinations settings. Output path is <path>/<destination>/part-<shard><suffix> for regular writes and <path>/<destination>/part-<window>-<shard><suffix> for windowed writes, where <destination> is computed with destinationFn.

   path

   path

   default

   default <destination> for empty collections

   windowedWrites

   preserves windowing of input elements and writes them to files based on the element's window

   numShards

   the number of shards to use, or 0 to let the system decide, must be > 0 when windowedWrites is true