com.ibm.spark.netezza

NetezzaDataReader

class NetezzaDataReader extends Iterator[NetezzaRow]

Creates reader for the given partitions. Reader fetches the data for the Netezza source table by creating external table to tranfer from Netezza host system to a remote spark client using named pipes.

Executes the external table statement in separate thread to allow reading of data in parallel from the named pipe by the RDD iterator.

Linear Supertypes
Iterator[NetezzaRow], TraversableOnce[NetezzaRow], GenTraversableOnce[NetezzaRow], AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. NetezzaDataReader
  2. Iterator
  3. TraversableOnce
  4. GenTraversableOnce
  5. AnyRef
  6. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new NetezzaDataReader(conn: Connection, table: String, columns: Array[String], filters: Array[Filter], partition: NetezzaPartition, schema: StructType)

Type Members

  1. class GroupedIterator[B >: A] extends AbstractIterator[Seq[B]] with Iterator[Seq[B]]

    Definition Classes
    Iterator

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. def ++[B >: NetezzaRow](that: ⇒ GenTraversableOnce[B]): Iterator[B]

    Definition Classes
    Iterator
  5. def /:[B](z: B)(op: (B, NetezzaRow) ⇒ B): B

    Definition Classes
    TraversableOnce → GenTraversableOnce
  6. def :\[B](z: B)(op: (NetezzaRow, B) ⇒ B): B

    Definition Classes
    TraversableOnce → GenTraversableOnce
  7. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  8. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  9. def addString(b: StringBuilder): StringBuilder

    Definition Classes
    TraversableOnce
  10. def addString(b: StringBuilder, sep: String): StringBuilder

    Definition Classes
    TraversableOnce
  11. def addString(b: StringBuilder, start: String, sep: String, end: String): StringBuilder

    Definition Classes
    TraversableOnce
  12. def aggregate[B](z: B)(seqop: (B, NetezzaRow) ⇒ B, combop: (B, B) ⇒ B): B

    Definition Classes
    TraversableOnce → GenTraversableOnce
  13. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  14. def buffered: BufferedIterator[NetezzaRow]

    Definition Classes
    Iterator
  15. def buildExternalTableQuery(pipeId: String): String

    Build externa table query for the specified options.

    Build externa table query for the specified options.

    pipeId

    id of the names pipe the Netezza system should write the data.

    returns

    External table query to unload the data.

  16. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  17. def close(): Unit

  18. var closed: Boolean

  19. def collect[B](pf: PartialFunction[NetezzaRow, B]): Iterator[B]

    Definition Classes
    Iterator
    Annotations
    @migration
    Migration

    (Changed in version 2.8.0) collect has changed. The previous behavior can be reproduced with toSeq.

  20. def collectFirst[B](pf: PartialFunction[NetezzaRow, B]): Option[B]

    Definition Classes
    TraversableOnce
  21. def contains(elem: Any): Boolean

    Definition Classes
    Iterator
  22. def copyToArray[B >: NetezzaRow](xs: Array[B], start: Int, len: Int): Unit

    Definition Classes
    Iterator → TraversableOnce → GenTraversableOnce
  23. def copyToArray[B >: NetezzaRow](xs: Array[B]): Unit

    Definition Classes
    TraversableOnce → GenTraversableOnce
  24. def copyToArray[B >: NetezzaRow](xs: Array[B], start: Int): Unit

    Definition Classes
    TraversableOnce → GenTraversableOnce
  25. def copyToBuffer[B >: NetezzaRow](dest: Buffer[B]): Unit

    Definition Classes
    TraversableOnce
  26. def corresponds[B](that: GenTraversableOnce[B])(p: (NetezzaRow, B) ⇒ Boolean): Boolean

    Definition Classes
    Iterator
  27. def count(p: (NetezzaRow) ⇒ Boolean): Int

    Definition Classes
    TraversableOnce → GenTraversableOnce
  28. val delimiter: Char

  29. def drop(n: Int): Iterator[NetezzaRow]

    Definition Classes
    Iterator
  30. def dropWhile(p: (NetezzaRow) ⇒ Boolean): Iterator[NetezzaRow]

    Definition Classes
    Iterator
  31. def duplicate: (Iterator[NetezzaRow], Iterator[NetezzaRow])

    Definition Classes
    Iterator
  32. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  33. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  34. val escapeChar: Char

  35. var execThread: StatementExecutorThread

  36. def exists(p: (NetezzaRow) ⇒ Boolean): Boolean

    Definition Classes
    Iterator → TraversableOnce → GenTraversableOnce
  37. def filter(p: (NetezzaRow) ⇒ Boolean): Iterator[NetezzaRow]

    Definition Classes
    Iterator
  38. def filterNot(p: (NetezzaRow) ⇒ Boolean): Iterator[NetezzaRow]

    Definition Classes
    Iterator
  39. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  40. def find(p: (NetezzaRow) ⇒ Boolean): Option[NetezzaRow]

    Definition Classes
    Iterator → TraversableOnce → GenTraversableOnce
  41. var firstCall: Boolean

  42. var fis: FileInputStream

  43. def flatMap[B](f: (NetezzaRow) ⇒ GenTraversableOnce[B]): Iterator[B]

    Definition Classes
    Iterator
  44. def fold[A1 >: NetezzaRow](z: A1)(op: (A1, A1) ⇒ A1): A1

    Definition Classes
    TraversableOnce → GenTraversableOnce
  45. def foldLeft[B](z: B)(op: (B, NetezzaRow) ⇒ B): B

    Definition Classes
    TraversableOnce → GenTraversableOnce
  46. def foldRight[B](z: B)(op: (NetezzaRow, B) ⇒ B): B

    Definition Classes
    TraversableOnce → GenTraversableOnce
  47. def forall(p: (NetezzaRow) ⇒ Boolean): Boolean

    Definition Classes
    Iterator → TraversableOnce → GenTraversableOnce
  48. def foreach[U](f: (NetezzaRow) ⇒ U): Unit

    Definition Classes
    Iterator → TraversableOnce → GenTraversableOnce
  49. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  50. def grouped[B >: NetezzaRow](size: Int): GroupedIterator[B]

    Definition Classes
    Iterator
  51. def hasDefiniteSize: Boolean

    Definition Classes
    Iterator → TraversableOnce → GenTraversableOnce
  52. def hasNext: Boolean

    Returns true if there are record in the pipe, otherwise false.

    Returns true if there are record in the pipe, otherwise false.

    Definition Classes
    NetezzaDataReader → Iterator
  53. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  54. def indexOf[B >: NetezzaRow](elem: B): Int

    Definition Classes
    Iterator
  55. def indexWhere(p: (NetezzaRow) ⇒ Boolean): Int

    Definition Classes
    Iterator
  56. var input: BufferedReader

  57. def isEmpty: Boolean

    Definition Classes
    Iterator → TraversableOnce → GenTraversableOnce
  58. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  59. def isTraversableAgain: Boolean

    Definition Classes
    Iterator → GenTraversableOnce
  60. var isr: InputStreamReader

  61. def length: Int

    Definition Classes
    Iterator
  62. def map[B](f: (NetezzaRow) ⇒ B): Iterator[B]

    Definition Classes
    Iterator
  63. def max[B >: NetezzaRow](implicit cmp: Ordering[B]): NetezzaRow

    Definition Classes
    TraversableOnce → GenTraversableOnce
  64. def maxBy[B](f: (NetezzaRow) ⇒ B)(implicit cmp: Ordering[B]): NetezzaRow

    Definition Classes
    TraversableOnce → GenTraversableOnce
  65. def min[B >: NetezzaRow](implicit cmp: Ordering[B]): NetezzaRow

    Definition Classes
    TraversableOnce → GenTraversableOnce
  66. def minBy[B](f: (NetezzaRow) ⇒ B)(implicit cmp: Ordering[B]): NetezzaRow

    Definition Classes
    TraversableOnce → GenTraversableOnce
  67. def mkString: String

    Definition Classes
    TraversableOnce → GenTraversableOnce
  68. def mkString(sep: String): String

    Definition Classes
    TraversableOnce → GenTraversableOnce
  69. def mkString(start: String, sep: String, end: String): String

    Definition Classes
    TraversableOnce → GenTraversableOnce
  70. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  71. def next(): NetezzaRow

    Definition Classes
    NetezzaDataReader → Iterator
  72. var nextLine: String

  73. var nextRecord: NetezzaRow

  74. def nonEmpty: Boolean

    Definition Classes
    TraversableOnce → GenTraversableOnce
  75. final def notify(): Unit

    Definition Classes
    AnyRef
  76. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  77. def padTo[A1 >: NetezzaRow](len: Int, elem: A1): Iterator[A1]

    Definition Classes
    Iterator
  78. def partition(p: (NetezzaRow) ⇒ Boolean): (Iterator[NetezzaRow], Iterator[NetezzaRow])

    Definition Classes
    Iterator
  79. def patch[B >: NetezzaRow](from: Int, patchElems: Iterator[B], replaced: Int): Iterator[B]

    Definition Classes
    Iterator
  80. var pipe: File

  81. def product[B >: NetezzaRow](implicit num: Numeric[B]): B

    Definition Classes
    TraversableOnce → GenTraversableOnce
  82. val recordParser: NetezzaRecordParser

  83. def reduce[A1 >: NetezzaRow](op: (A1, A1) ⇒ A1): A1

    Definition Classes
    TraversableOnce → GenTraversableOnce
  84. def reduceLeft[B >: NetezzaRow](op: (B, NetezzaRow) ⇒ B): B

    Definition Classes
    TraversableOnce
  85. def reduceLeftOption[B >: NetezzaRow](op: (B, NetezzaRow) ⇒ B): Option[B]

    Definition Classes
    TraversableOnce → GenTraversableOnce
  86. def reduceOption[A1 >: NetezzaRow](op: (A1, A1) ⇒ A1): Option[A1]

    Definition Classes
    TraversableOnce → GenTraversableOnce
  87. def reduceRight[B >: NetezzaRow](op: (NetezzaRow, B) ⇒ B): B

    Definition Classes
    TraversableOnce → GenTraversableOnce
  88. def reduceRightOption[B >: NetezzaRow](op: (NetezzaRow, B) ⇒ B): Option[B]

    Definition Classes
    TraversableOnce → GenTraversableOnce
  89. def reversed: List[NetezzaRow]

    Attributes
    protected[this]
    Definition Classes
    TraversableOnce
  90. def sameElements(that: Iterator[_]): Boolean

    Definition Classes
    Iterator
  91. def scanLeft[B](z: B)(op: (B, NetezzaRow) ⇒ B): Iterator[B]

    Definition Classes
    Iterator
  92. def scanRight[B](z: B)(op: (NetezzaRow, B) ⇒ B): Iterator[B]

    Definition Classes
    Iterator
  93. def seq: Iterator[NetezzaRow]

    Definition Classes
    Iterator → TraversableOnce → GenTraversableOnce
  94. def size: Int

    Definition Classes
    TraversableOnce → GenTraversableOnce
  95. def slice(from: Int, until: Int): Iterator[NetezzaRow]

    Definition Classes
    Iterator
  96. def sliding[B >: NetezzaRow](size: Int, step: Int): GroupedIterator[B]

    Definition Classes
    Iterator
  97. def span(p: (NetezzaRow) ⇒ Boolean): (Iterator[NetezzaRow], Iterator[NetezzaRow])

    Definition Classes
    Iterator
  98. def startExternalTableDataUnload(): Unit

    Start the external table executor that unloads the data.

    Start the external table executor that unloads the data. It is necessary to do the setup in separate method instead of class constructor to allow caller to execute close() call in error cases, otherwise job can hang forever.

  99. var stmt: PreparedStatement

  100. def sum[B >: NetezzaRow](implicit num: Numeric[B]): B

    Definition Classes
    TraversableOnce → GenTraversableOnce
  101. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  102. def take(n: Int): Iterator[NetezzaRow]

    Definition Classes
    Iterator
  103. def takeWhile(p: (NetezzaRow) ⇒ Boolean): Iterator[NetezzaRow]

    Definition Classes
    Iterator
  104. def to[Col[_]](implicit cbf: CanBuildFrom[Nothing, NetezzaRow, Col[NetezzaRow]]): Col[NetezzaRow]

    Definition Classes
    TraversableOnce → GenTraversableOnce
  105. def toArray[B >: NetezzaRow](implicit arg0: ClassTag[B]): Array[B]

    Definition Classes
    TraversableOnce → GenTraversableOnce
  106. def toBuffer[B >: NetezzaRow]: Buffer[B]

    Definition Classes
    TraversableOnce → GenTraversableOnce
  107. def toIndexedSeq: IndexedSeq[NetezzaRow]

    Definition Classes
    TraversableOnce → GenTraversableOnce
  108. def toIterable: Iterable[NetezzaRow]

    Definition Classes
    TraversableOnce → GenTraversableOnce
  109. def toIterator: Iterator[NetezzaRow]

    Definition Classes
    Iterator → GenTraversableOnce
  110. def toList: List[NetezzaRow]

    Definition Classes
    TraversableOnce → GenTraversableOnce
  111. def toMap[T, U](implicit ev: <:<[NetezzaRow, (T, U)]): Map[T, U]

    Definition Classes
    TraversableOnce → GenTraversableOnce
  112. def toSeq: Seq[NetezzaRow]

    Definition Classes
    TraversableOnce → GenTraversableOnce
  113. def toSet[B >: NetezzaRow]: Set[B]

    Definition Classes
    TraversableOnce → GenTraversableOnce
  114. def toStream: Stream[NetezzaRow]

    Definition Classes
    Iterator → GenTraversableOnce
  115. def toString(): String

    Definition Classes
    Iterator → AnyRef → Any
  116. def toTraversable: Traversable[NetezzaRow]

    Definition Classes
    Iterator → TraversableOnce → GenTraversableOnce
  117. def toVector: Vector[NetezzaRow]

    Definition Classes
    TraversableOnce → GenTraversableOnce
  118. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  119. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  120. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  121. def withFilter(p: (NetezzaRow) ⇒ Boolean): Iterator[NetezzaRow]

    Definition Classes
    Iterator
  122. def zip[B](that: Iterator[B]): Iterator[(NetezzaRow, B)]

    Definition Classes
    Iterator
  123. def zipAll[B, A1 >: NetezzaRow, B1 >: B](that: Iterator[B], thisElem: A1, thatElem: B1): Iterator[(A1, B1)]

    Definition Classes
    Iterator
  124. def zipWithIndex: Iterator[(NetezzaRow, Int)]

    Definition Classes
    Iterator

Deprecated Value Members

  1. def /:\[A1 >: NetezzaRow](z: A1)(op: (A1, A1) ⇒ A1): A1

    Definition Classes
    GenTraversableOnce
    Annotations
    @deprecated
    Deprecated

    (Since version 2.10.0) use fold instead

Inherited from Iterator[NetezzaRow]

Inherited from TraversableOnce[NetezzaRow]

Inherited from GenTraversableOnce[NetezzaRow]

Inherited from AnyRef

Inherited from Any

Ungrouped