GroupedIterator

Companion object GroupedIterator

class GroupedIterator extends Iterator[(InternalRow, Iterator[InternalRow])]

Iterates over a presorted set of rows, chunking it up by the grouping expression. Each call to next will return a pair containing the current group and an iterator that will return all the elements of that group. Iterators for each group are lazily constructed by extracting rows from the input iterator. As such, full groups are never materialized by this class.

Example input:

Input: [a, 1], [b, 2], [b, 3]
Grouping: x#1
InputSchema: x#1, y#2

Result:

First call to next():  ([a], Iterator([a, 1])
Second call to next(): ([b], Iterator([b, 2], [b, 3])

Note, the class does not handle the case of an empty input for simplicity of implementation. Use the factory to construct a new instance.

Linear Supertypes

Iterator[(InternalRow, Iterator[InternalRow])], IterableOnceOps[(InternalRow, Iterator[InternalRow]), Iterator, Iterator[(InternalRow, Iterator[InternalRow])]], IterableOnce[(InternalRow, Iterator[InternalRow])], AnyRef, Any

Ordering

Alphabetic
By Inheritance

Inherited

GroupedIterator
Iterator
IterableOnceOps
IterableOnce
AnyRef
Any

Hide All
Show All

Visibility

Public
Protected

Type Members

class GroupedIterator[B >: A] extends AbstractIterator[Seq[B]]
Definition Classes
Iterator

Value Members

final def !=(arg0: Any): Boolean
Definition Classes
AnyRef → Any
final def ##: Int
Definition Classes
AnyRef → Any
final def ++[B >: (InternalRow, Iterator[InternalRow])](xs: => IterableOnce[B]): Iterator[B]
Definition Classes
Iterator
Annotations
@inline()
final def ==(arg0: Any): Boolean
Definition Classes
AnyRef → Any
final def addString(b: StringBuilder): StringBuilder
Definition Classes
IterableOnceOps
Annotations
@inline()
final def addString(b: StringBuilder, sep: String): StringBuilder
Definition Classes
IterableOnceOps
Annotations
@inline()
def addString(b: StringBuilder, start: String, sep: String, end: String): StringBuilder
Definition Classes
IterableOnceOps
final def asInstanceOf[T0]: T0
Definition Classes
Any
def buffered: BufferedIterator[(InternalRow, Iterator[InternalRow])]
Definition Classes
Iterator
def clone(): AnyRef
Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws(classOf[java.lang.CloneNotSupportedException]) @native()
def collect[B](pf: PartialFunction[(InternalRow, Iterator[InternalRow]), B]): Iterator[B]
Definition Classes
Iterator → IterableOnceOps
def collectFirst[B](pf: PartialFunction[(InternalRow, Iterator[InternalRow]), B]): Option[B]
Definition Classes
IterableOnceOps
def concat[B >: (InternalRow, Iterator[InternalRow])](xs: => IterableOnce[B]): Iterator[B]
Definition Classes
Iterator
def contains(elem: Any): Boolean
Definition Classes
Iterator
def copyToArray[B >: (InternalRow, Iterator[InternalRow])](xs: Array[B], start: Int, len: Int): Int
Definition Classes
IterableOnceOps
def copyToArray[B >: (InternalRow, Iterator[InternalRow])](xs: Array[B], start: Int): Int
Definition Classes
IterableOnceOps
Annotations
@deprecatedOverriding()
def copyToArray[B >: (InternalRow, Iterator[InternalRow])](xs: Array[B]): Int
Definition Classes
IterableOnceOps
Annotations
@deprecatedOverriding()
def corresponds[B](that: IterableOnce[B])(p: ((InternalRow, Iterator[InternalRow]), B) => Boolean): Boolean
Definition Classes
IterableOnceOps
def count(p: ((InternalRow, Iterator[InternalRow])) => Boolean): Int
Definition Classes
IterableOnceOps
var currentGroup: InternalRow
Holds a copy of an input row that is in the current group.
var currentIterator: Iterator[InternalRow]
var currentRow: InternalRow
Holds null or the row that will be returned on next call to next() in the inner iterator.
def distinct: Iterator[(InternalRow, Iterator[InternalRow])]
Definition Classes
Iterator
def distinctBy[B](f: ((InternalRow, Iterator[InternalRow])) => B): Iterator[(InternalRow, Iterator[InternalRow])]
Definition Classes
Iterator
def drop(n: Int): Iterator[(InternalRow, Iterator[InternalRow])]
Definition Classes
Iterator → IterableOnceOps
def dropWhile(p: ((InternalRow, Iterator[InternalRow])) => Boolean): Iterator[(InternalRow, Iterator[InternalRow])]
Definition Classes
Iterator → IterableOnceOps
def duplicate: (Iterator[(InternalRow, Iterator[InternalRow])], Iterator[(InternalRow, Iterator[InternalRow])])
Definition Classes
Iterator
final def eq(arg0: AnyRef): Boolean
Definition Classes
AnyRef
def equals(arg0: AnyRef): Boolean
Definition Classes
AnyRef → Any
def exists(p: ((InternalRow, Iterator[InternalRow])) => Boolean): Boolean
Definition Classes
IterableOnceOps
def filter(p: ((InternalRow, Iterator[InternalRow])) => Boolean): Iterator[(InternalRow, Iterator[InternalRow])]
Definition Classes
Iterator → IterableOnceOps
def filterNot(p: ((InternalRow, Iterator[InternalRow])) => Boolean): Iterator[(InternalRow, Iterator[InternalRow])]
Definition Classes
Iterator → IterableOnceOps
def finalize(): Unit
Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws(classOf[java.lang.Throwable])
def find(p: ((InternalRow, Iterator[InternalRow])) => Boolean): Option[(InternalRow, Iterator[InternalRow])]
Definition Classes
IterableOnceOps
def flatMap[B](f: ((InternalRow, Iterator[InternalRow])) => IterableOnce[B]): Iterator[B]
Definition Classes
Iterator → IterableOnceOps
def flatten[B](implicit ev: ((InternalRow, Iterator[InternalRow])) => IterableOnce[B]): Iterator[B]
Definition Classes
Iterator → IterableOnceOps
def fold[A1 >: (InternalRow, Iterator[InternalRow])](z: A1)(op: (A1, A1) => A1): A1
Definition Classes
IterableOnceOps
def foldLeft[B](z: B)(op: (B, (InternalRow, Iterator[InternalRow])) => B): B
Definition Classes
IterableOnceOps
def foldRight[B](z: B)(op: ((InternalRow, Iterator[InternalRow]), B) => B): B
Definition Classes
IterableOnceOps
def forall(p: ((InternalRow, Iterator[InternalRow])) => Boolean): Boolean
Definition Classes
IterableOnceOps
def foreach[U](f: ((InternalRow, Iterator[InternalRow])) => U): Unit
Definition Classes
IterableOnceOps
final def getClass(): Class[_ <: AnyRef]
Definition Classes
AnyRef → Any
Annotations
@native()
def grouped[B >: (InternalRow, Iterator[InternalRow])](size: Int): GroupedIterator[B]
Definition Classes
Iterator
def hasNext: Boolean
Return true if we already have the next iterator or fetching a new iterator is successful.
Return true if we already have the next iterator or fetching a new iterator is successful.
Note that, if we get the iterator by next, we should consume it before call hasNext, because we will consume the input data to skip to next group while fetching a new iterator, thus make the previous iterator empty.
Definition Classes
GroupedIterator → Iterator
def hashCode(): Int
Definition Classes
AnyRef → Any
Annotations
@native()
def indexOf[B >: (InternalRow, Iterator[InternalRow])](elem: B, from: Int): Int
Definition Classes
Iterator
def indexOf[B >: (InternalRow, Iterator[InternalRow])](elem: B): Int
Definition Classes
Iterator
def indexWhere(p: ((InternalRow, Iterator[InternalRow])) => Boolean, from: Int): Int
Definition Classes
Iterator
def isEmpty: Boolean
Definition Classes
Iterator → IterableOnceOps
Annotations
@deprecatedOverriding()
final def isInstanceOf[T0]: Boolean
Definition Classes
Any
def isTraversableAgain: Boolean
Definition Classes
IterableOnceOps
final def iterator: Iterator[(InternalRow, Iterator[InternalRow])]
Definition Classes
Iterator → IterableOnce
Annotations
@inline()
val keyOrdering: BaseOrdering
val keyProjection: UnsafeProjection
Creates a row containing only the key for a given input row.
def knownSize: Int
Definition Classes
IterableOnce
final def length: Int
Definition Classes
Iterator
Annotations
@inline()
def map[B](f: ((InternalRow, Iterator[InternalRow])) => B): Iterator[B]
Definition Classes
Iterator → IterableOnceOps
def max[B >: (InternalRow, Iterator[InternalRow])](implicit ord: Ordering[B]): (InternalRow, Iterator[InternalRow])
Definition Classes
IterableOnceOps
def maxBy[B](f: ((InternalRow, Iterator[InternalRow])) => B)(implicit cmp: Ordering[B]): (InternalRow, Iterator[InternalRow])
Definition Classes
IterableOnceOps
def maxByOption[B](f: ((InternalRow, Iterator[InternalRow])) => B)(implicit cmp: Ordering[B]): Option[(InternalRow, Iterator[InternalRow])]
Definition Classes
IterableOnceOps
def maxOption[B >: (InternalRow, Iterator[InternalRow])](implicit ord: Ordering[B]): Option[(InternalRow, Iterator[InternalRow])]
Definition Classes
IterableOnceOps
def min[B >: (InternalRow, Iterator[InternalRow])](implicit ord: Ordering[B]): (InternalRow, Iterator[InternalRow])
Definition Classes
IterableOnceOps
def minBy[B](f: ((InternalRow, Iterator[InternalRow])) => B)(implicit cmp: Ordering[B]): (InternalRow, Iterator[InternalRow])
Definition Classes
IterableOnceOps
def minByOption[B](f: ((InternalRow, Iterator[InternalRow])) => B)(implicit cmp: Ordering[B]): Option[(InternalRow, Iterator[InternalRow])]
Definition Classes
IterableOnceOps
def minOption[B >: (InternalRow, Iterator[InternalRow])](implicit ord: Ordering[B]): Option[(InternalRow, Iterator[InternalRow])]
Definition Classes
IterableOnceOps
final def mkString: String
Definition Classes
IterableOnceOps
Annotations
@inline()
final def mkString(sep: String): String
Definition Classes
IterableOnceOps
Annotations
@inline()
final def mkString(start: String, sep: String, end: String): String
Definition Classes
IterableOnceOps
final def ne(arg0: AnyRef): Boolean
Definition Classes
AnyRef
def next(): (InternalRow, Iterator[InternalRow])
Definition Classes
GroupedIterator → Iterator
def nextOption(): Option[(InternalRow, Iterator[InternalRow])]
Definition Classes
Iterator
def nonEmpty: Boolean
Definition Classes
IterableOnceOps
Annotations
@deprecatedOverriding()
final def notify(): Unit
Definition Classes
AnyRef
Annotations
@native()
final def notifyAll(): Unit
Definition Classes
AnyRef
Annotations
@native()
def padTo[B >: (InternalRow, Iterator[InternalRow])](len: Int, elem: B): Iterator[B]
Definition Classes
Iterator
def partition(p: ((InternalRow, Iterator[InternalRow])) => Boolean): (Iterator[(InternalRow, Iterator[InternalRow])], Iterator[(InternalRow, Iterator[InternalRow])])
Definition Classes
Iterator
def patch[B >: (InternalRow, Iterator[InternalRow])](from: Int, patchElems: Iterator[B], replaced: Int): Iterator[B]
Definition Classes
Iterator
def product[B >: (InternalRow, Iterator[InternalRow])](implicit num: Numeric[B]): B
Definition Classes
IterableOnceOps
def reduce[B >: (InternalRow, Iterator[InternalRow])](op: (B, B) => B): B
Definition Classes
IterableOnceOps
def reduceLeft[B >: (InternalRow, Iterator[InternalRow])](op: (B, (InternalRow, Iterator[InternalRow])) => B): B
Definition Classes
IterableOnceOps
def reduceLeftOption[B >: (InternalRow, Iterator[InternalRow])](op: (B, (InternalRow, Iterator[InternalRow])) => B): Option[B]
Definition Classes
IterableOnceOps
def reduceOption[B >: (InternalRow, Iterator[InternalRow])](op: (B, B) => B): Option[B]
Definition Classes
IterableOnceOps
def reduceRight[B >: (InternalRow, Iterator[InternalRow])](op: ((InternalRow, Iterator[InternalRow]), B) => B): B
Definition Classes
IterableOnceOps
def reduceRightOption[B >: (InternalRow, Iterator[InternalRow])](op: ((InternalRow, Iterator[InternalRow]), B) => B): Option[B]
Definition Classes
IterableOnceOps
def reversed: Iterable[(InternalRow, Iterator[InternalRow])]
Attributes
protected
Definition Classes
IterableOnceOps
def sameElements[B >: (InternalRow, Iterator[InternalRow])](that: IterableOnce[B]): Boolean
Definition Classes
Iterator
def scanLeft[B](z: B)(op: (B, (InternalRow, Iterator[InternalRow])) => B): Iterator[B]
Definition Classes
Iterator → IterableOnceOps
def size: Int
Definition Classes
IterableOnceOps
def slice(from: Int, until: Int): Iterator[(InternalRow, Iterator[InternalRow])]
Definition Classes
Iterator → IterableOnceOps
def sliceIterator(from: Int, until: Int): Iterator[(InternalRow, Iterator[InternalRow])]
Attributes
protected
Definition Classes
Iterator
def sliding[B >: (InternalRow, Iterator[InternalRow])](size: Int, step: Int): GroupedIterator[B]
Definition Classes
Iterator
val sortOrder: Seq[SortOrder]
Compares two input rows and returns 0 if they are in the same group.
def span(p: ((InternalRow, Iterator[InternalRow])) => Boolean): (Iterator[(InternalRow, Iterator[InternalRow])], Iterator[(InternalRow, Iterator[InternalRow])])
Definition Classes
Iterator → IterableOnceOps
def splitAt(n: Int): (Iterator[(InternalRow, Iterator[InternalRow])], Iterator[(InternalRow, Iterator[InternalRow])])
Definition Classes
IterableOnceOps
def stepper[S <: Stepper[_]](implicit shape: StepperShape[(InternalRow, Iterator[InternalRow]), S]): S
Definition Classes
IterableOnce
def sum[B >: (InternalRow, Iterator[InternalRow])](implicit num: Numeric[B]): B
Definition Classes
IterableOnceOps
final def synchronized[T0](arg0: => T0): T0
Definition Classes
AnyRef
def take(n: Int): Iterator[(InternalRow, Iterator[InternalRow])]
Definition Classes
Iterator → IterableOnceOps
def takeWhile(p: ((InternalRow, Iterator[InternalRow])) => Boolean): Iterator[(InternalRow, Iterator[InternalRow])]
Definition Classes
Iterator → IterableOnceOps
def tapEach[U](f: ((InternalRow, Iterator[InternalRow])) => U): Iterator[(InternalRow, Iterator[InternalRow])]
Definition Classes
Iterator → IterableOnceOps
def to[C1](factory: Factory[(InternalRow, Iterator[InternalRow]), C1]): C1
Definition Classes
IterableOnceOps
def toArray[B >: (InternalRow, Iterator[InternalRow])](implicit arg0: ClassTag[B]): Array[B]
Definition Classes
IterableOnceOps
final def toBuffer[B >: (InternalRow, Iterator[InternalRow])]: Buffer[B]
Definition Classes
IterableOnceOps
Annotations
@inline()
def toIndexedSeq: IndexedSeq[(InternalRow, Iterator[InternalRow])]
Definition Classes
IterableOnceOps
def toList: List[(InternalRow, Iterator[InternalRow])]
Definition Classes
IterableOnceOps
def toMap[K, V](implicit ev: <:<[(InternalRow, Iterator[InternalRow]), (K, V)]): Map[K, V]
Definition Classes
IterableOnceOps
def toSeq: Seq[(InternalRow, Iterator[InternalRow])]
Definition Classes
IterableOnceOps
def toSet[B >: (InternalRow, Iterator[InternalRow])]: Set[B]
Definition Classes
IterableOnceOps
def toString(): String
Definition Classes
Iterator → AnyRef → Any
def toVector: Vector[(InternalRow, Iterator[InternalRow])]
Definition Classes
IterableOnceOps
final def wait(): Unit
Definition Classes
AnyRef
Annotations
@throws(classOf[java.lang.InterruptedException])
final def wait(arg0: Long, arg1: Int): Unit
Definition Classes
AnyRef
Annotations
@throws(classOf[java.lang.InterruptedException])
final def wait(arg0: Long): Unit
Definition Classes
AnyRef
Annotations
@throws(classOf[java.lang.InterruptedException]) @native()
def withFilter(p: ((InternalRow, Iterator[InternalRow])) => Boolean): Iterator[(InternalRow, Iterator[InternalRow])]
Definition Classes
Iterator
def zip[B](that: IterableOnce[B]): Iterator[((InternalRow, Iterator[InternalRow]), B)]
Definition Classes
Iterator
def zipAll[A1 >: (InternalRow, Iterator[InternalRow]), B](that: IterableOnce[B], thisElem: A1, thatElem: B): Iterator[(A1, B)]
Definition Classes
Iterator
def zipWithIndex: Iterator[((InternalRow, Iterator[InternalRow]), Int)]
Definition Classes
Iterator → IterableOnceOps

Deprecated Value Members

final def /:[B](z: B)(op: (B, (InternalRow, Iterator[InternalRow])) => B): B
Definition Classes
IterableOnceOps
Annotations
@deprecated @inline()
Deprecated
(Since version 2.13.0) Use foldLeft instead of /:
final def :\[B](z: B)(op: ((InternalRow, Iterator[InternalRow]), B) => B): B
Definition Classes
IterableOnceOps
Annotations
@deprecated @inline()
Deprecated
(Since version 2.13.0) Use foldRight instead of :\
def aggregate[B](z: => B)(seqop: (B, (InternalRow, Iterator[InternalRow])) => B, combop: (B, B) => B): B
Definition Classes
IterableOnceOps
Annotations
@deprecated
Deprecated
(Since version 2.13.0) aggregate is not relevant for sequential collections. Use foldLeft(z)(seqop) instead.
final def copyToBuffer[B >: (InternalRow, Iterator[InternalRow])](dest: Buffer[B]): Unit
Definition Classes
IterableOnceOps
Annotations
@deprecated @inline()
Deprecated
(Since version 2.13.0) Use dest ++= coll instead
final def hasDefiniteSize: Boolean
Definition Classes
Iterator → IterableOnceOps
Annotations
@deprecated @inline()
Deprecated
(Since version 2.13.0) hasDefiniteSize on Iterator is the same as isEmpty
def scanRight[B](z: B)(op: ((InternalRow, Iterator[InternalRow]), B) => B): Iterator[B]
Definition Classes
Iterator
Annotations
@deprecated
Deprecated
(Since version 2.13.0) Call scanRight on an Iterable instead.
def seq: execution.GroupedIterator.this.type
Definition Classes
Iterator
Annotations
@deprecated
Deprecated
(Since version 2.13.0) Iterator.seq always returns the iterator itself
final def toIterator: Iterator[(InternalRow, Iterator[InternalRow])]
Definition Classes
IterableOnceOps
Annotations
@deprecated @inline()
Deprecated
(Since version 2.13.0) Use .iterator instead of .toIterator
final def toStream: Stream[(InternalRow, Iterator[InternalRow])]
Definition Classes
IterableOnceOps
Annotations
@deprecated @inline()
Deprecated
(Since version 2.13.0) Use .to(LazyList) instead of .toStream

Packages

GroupedIterator

Companion object GroupedIterator

class GroupedIterator extends Iterator[(InternalRow, Iterator[InternalRow])]

Type Members

Value Members

Deprecated Value Members

Inherited from Iterator[(InternalRow, Iterator[InternalRow])]

Inherited from IterableOnceOps[(InternalRow, Iterator[InternalRow]), Iterator, Iterator[(InternalRow, Iterator[InternalRow])]]

Inherited from IterableOnce[(InternalRow, Iterator[InternalRow])]

Inherited from AnyRef

Inherited from Any

Ungrouped

Packages

GroupedIterator

Companion object GroupedIterator

class GroupedIterator extends Iterator[(InternalRow, Iterator[InternalRow])]

Type Members

Value Members

Deprecated Value Members

Inherited from Iterator[(InternalRow, Iterator[InternalRow])]

Inherited from IterableOnceOps[(InternalRow, Iterator[InternalRow]), Iterator, Iterator[(InternalRow, Iterator[InternalRow])]]

Inherited from IterableOnce[(InternalRow, Iterator[InternalRow])]

Inherited from AnyRef

Inherited from Any

Ungrouped

GroupedIterator