fm.lazyseq.SortedLazySeqBuilder
ReadSortedRecords
Related Doc:
package SortedLazySeqBuilder
Run this function on each item after the foreach function
Run this function on each item after the foreach function
This is basically a foreach but without forcing evaluation of this LazySeq.
Same as after() but takes a Resource (i.e.
Same as after() but takes a Resource (i.e. can use it for something like logging)
Assert that this reader is in sorted order
Assert that this reader is in sorted order
Assert that this reader is in sorted order AND unique
Assert that this reader is in sorted order AND unique
Run this function on each item before the foreach function
Run this function on each item before the foreach function
This is basically a foreach but without forcing evaluation of this LazySeq.
Same as before() but takes a Resource (i.e.
Same as before() but takes a Resource (i.e. can use it for something like logging)
Split the LazySeq into num buckets of equal size using a round-robin algorithm
Split the LazySeq into num buckets of equal size using a round-robin algorithm
Creates an asynchronous buffer that spins up a producer thread which feeds data into a BlockingQueue that is read using the resulting LazySeq.
Creates an asynchronous buffer that spins up a producer thread which feeds data into a BlockingQueue that is read using the resulting LazySeq. This overload lets you pass in your own BlockingQueue implementation instead of having one created for you.
Creates an asynchronous buffer that spins up a producer thread which feeds data into a BlockingQueue that is read using the resulting LazySeq.
Creates an asynchronous buffer that spins up a producer thread which feeds data into a BlockingQueue that is read using the resulting LazySeq. The created BlockingQueue will depend on the size passed in. If the size is <= 0 then a SynchronousQueue otherwise an ArrayBlockingQueue is used.
This is the method that sub-classes must implement
This is the method that sub-classes must implement
Mostly standard group by implementation that uses tmp files to store the values of the HashMap
Mostly standard group by implementation that uses tmp files to store the values of the HashMap
A cross between grouped and groupBy that allows you to specify a key to be used (like in groupBy) instead of a fixed count (like in grouped).
A cross between grouped and groupBy that allows you to specify a key to be used (like in groupBy) instead of a fixed count (like in grouped). All elements next to each other with the same key get returned in each group.
e.g. LazySeq.wrap(Seq(1,1,1,2,2,1)).groupedBy{ a => a }.toIndexedSeq => Vector((1,Vector(1, 1, 1)), (2,Vector(2, 2)), (1,Vector(1)))
Merge corresponding records from this sorted read with that sorted reader given a method to get a common key that can be compared.
Merge corresponding records from this sorted read with that sorted reader given a method to get a common key that can be compared.
Execute the method on the first element of the LazySeq whenever it is evaluated.
Execute the method on the first element of the LazySeq whenever it is evaluated.
Note: The first element is still call via foreach
Same as onFirst except for the last element whenever it is evaluated
Same as onFirst except for the last element whenever it is evaluated
Performs a parallel flat map maintaining ordered output
Performs a parallel flat map maintaining ordered output
Performs a parallel flat map maintaining ordered output
Performs a parallel flat map maintaining ordered output
A Parallel foreach
A Parallel foreach
A Parallel foreach
A Parallel foreach
Performs a parallel map maintaining ordered output
Performs a parallel map maintaining ordered output
Performs a parallel map maintaining ordered output
Performs a parallel map maintaining ordered output
Standard partition implementation using LazySeqs
Standard partition implementation using LazySeqs
Collapse elements with the same key by applying a binary operator.
Collapse elements with the same key by applying a binary operator.
Should be similar to doing something like: reader.groupBy(key).values.flatMap{ _.reduce(op) }
A reusable reader that performs a merge sort on the sorted files
Note: We use RandomAccessFile to hold onto the FileDescriptor of the temp file and open FileInputStreams based on the FileDescriptor. This allows us to delete the unlink the temp file from the file system while retaining a reference to it here so we can still read it. Then when the FileDescriptor gets garbage collected OR the JVM shuts down the file will be automatically deleted.