Packages

  • package root
    Definition Classes
    root
  • package io
    Definition Classes
    root
  • package tarantool
    Definition Classes
    io
  • package spark
    Definition Classes
    tarantool
  • package connector

    Tarantool connector for Apache Spark.

    Tarantool connector for Apache Spark.

    Call tarantoolSpace method on the SparkContext object to create a TarantoolRDD exposing Tarantool space as a Spark RDD.

    Example:

    Execute the following on a Cartridge router node (the tarantool/crud module must be installed):

    local crud = require('crud')

crud.insert('test_space', {1, nil, 'a1', 'Don Quixote', 'Miguel de Cervantes', 1605})
crud.insert('test_space', {2, nil, 'a2', 'The Great Gatsby', 'F. Scott Fitzgerald', 1925})
crud.insert('test_space', {3, nil, 'a3', 'War and Peace', 'Leo Tolstoy', 1869})

    Write the following in your Java client code:

    import io.tarantool.spark.connector._

val sparkMasterHost = "127.0.0.1"
val tarantoolRouterAddress = "127.0.0.1:3301"
val space = "test_space"

// Populate the Spark config with the address of a Cartridge router node and credentials:
val conf = new SparkConf(true)
conf.set ("tarantool.username", "admin")
conf.set ("tarantool.password", "testapp-cluster-cookie")
conf.set ("tarantool.hosts", tarantoolRouterAddress)

// Connect to the Spark cluster:
val sc = new SparkContext("spark://" + sparkMasterHost + ":7077", "example", conf)

// Read the space and print its contents:
val rdd = sc.tarantoolSpace(space)
rdd.toArray().foreach(println)

sc.stop()
    Definition Classes
    spark
  • package rdd
    Definition Classes
    connector
  • package converter
    Definition Classes
    rdd
  • TarantoolBaseRDD
  • TarantoolJavaRDD
  • TarantoolReadRDD
  • TarantoolWriteRDD
c

io.tarantool.spark.connector.rdd

TarantoolJavaRDD

class TarantoolJavaRDD[R] extends JavaRDD[R]

Bridge from Scala TarantoolReadRDD to Java.

Instances of this class may be instantiated using methods of SparkContextJavaFunctions.

Linear Supertypes
JavaRDD[R], AbstractJavaRDDLike[R, JavaRDD[R]], JavaRDDLike[R, JavaRDD[R]], Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. TarantoolJavaRDD
  2. JavaRDD
  3. AbstractJavaRDDLike
  4. JavaRDDLike
  5. Serializable
  6. Serializable
  7. AnyRef
  8. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new TarantoolJavaRDD(rdd: TarantoolReadRDD[R], classTag: ClassTag[R])
  2. new TarantoolJavaRDD(rdd: TarantoolReadRDD[R], clazz: Class[R])

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. def aggregate[U](zeroValue: U)(seqOp: Function2[U, R, U], combOp: Function2[U, U, U]): U
    Definition Classes
    JavaRDDLike
  5. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  6. def cache(): JavaRDD[R]
    Definition Classes
    JavaRDD
  7. def cartesian[U](other: JavaRDDLike[U, _]): JavaPairRDD[R, U]
    Definition Classes
    JavaRDDLike
  8. def checkpoint(): Unit
    Definition Classes
    JavaRDDLike
  9. implicit val classTag: ClassTag[R]
    Definition Classes
    JavaRDD → JavaRDDLike
  10. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  11. def coalesce(numPartitions: Int, shuffle: Boolean): JavaRDD[R]
    Definition Classes
    JavaRDD
  12. def coalesce(numPartitions: Int): JavaRDD[R]
    Definition Classes
    JavaRDD
  13. def collect(): List[R]
    Definition Classes
    JavaRDDLike
  14. def collectAsync(): JavaFutureAction[List[R]]
    Definition Classes
    JavaRDDLike
  15. def collectPartitions(partitionIds: Array[Int]): Array[List[R]]
    Definition Classes
    JavaRDDLike
  16. def context: SparkContext
    Definition Classes
    JavaRDDLike
  17. def count(): Long
    Definition Classes
    JavaRDDLike
  18. def countApprox(timeout: Long): PartialResult[BoundedDouble]
    Definition Classes
    JavaRDDLike
  19. def countApprox(timeout: Long, confidence: Double): PartialResult[BoundedDouble]
    Definition Classes
    JavaRDDLike
  20. def countApproxDistinct(relativeSD: Double): Long
    Definition Classes
    JavaRDDLike
  21. def countAsync(): JavaFutureAction[Long]
    Definition Classes
    JavaRDDLike
  22. def countByValue(): Map[R, Long]
    Definition Classes
    JavaRDDLike
  23. def countByValueApprox(timeout: Long): PartialResult[Map[R, BoundedDouble]]
    Definition Classes
    JavaRDDLike
  24. def countByValueApprox(timeout: Long, confidence: Double): PartialResult[Map[R, BoundedDouble]]
    Definition Classes
    JavaRDDLike
  25. def distinct(numPartitions: Int): JavaRDD[R]
    Definition Classes
    JavaRDD
  26. def distinct(): JavaRDD[R]
    Definition Classes
    JavaRDD
  27. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  28. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  29. def filter(f: Function[R, Boolean]): JavaRDD[R]
    Definition Classes
    JavaRDD
  30. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  31. def first(): R
    Definition Classes
    JavaRDDLike
  32. def flatMap[U](f: FlatMapFunction[R, U]): JavaRDD[U]
    Definition Classes
    JavaRDDLike
  33. def flatMapToDouble(f: DoubleFlatMapFunction[R]): JavaDoubleRDD
    Definition Classes
    JavaRDDLike
  34. def flatMapToPair[K2, V2](f: PairFlatMapFunction[R, K2, V2]): JavaPairRDD[K2, V2]
    Definition Classes
    JavaRDDLike
  35. def fold(zeroValue: R)(f: Function2[R, R, R]): R
    Definition Classes
    JavaRDDLike
  36. def foreach(f: VoidFunction[R]): Unit
    Definition Classes
    JavaRDDLike
  37. def foreachAsync(f: VoidFunction[R]): JavaFutureAction[Void]
    Definition Classes
    JavaRDDLike
  38. def foreachPartition(f: VoidFunction[Iterator[R]]): Unit
    Definition Classes
    JavaRDDLike
  39. def foreachPartitionAsync(f: VoidFunction[Iterator[R]]): JavaFutureAction[Void]
    Definition Classes
    JavaRDDLike
  40. def getCheckpointFile(): Optional[String]
    Definition Classes
    JavaRDDLike
  41. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  42. def getNumPartitions: Int
    Definition Classes
    JavaRDDLike
    Annotations
    @Since( "1.6.0" )
  43. def getStorageLevel: StorageLevel
    Definition Classes
    JavaRDDLike
  44. def glom(): JavaRDD[List[R]]
    Definition Classes
    JavaRDDLike
  45. def groupBy[U](f: Function[R, U], numPartitions: Int): JavaPairRDD[U, Iterable[R]]
    Definition Classes
    JavaRDDLike
  46. def groupBy[U](f: Function[R, U]): JavaPairRDD[U, Iterable[R]]
    Definition Classes
    JavaRDDLike
  47. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  48. def id: Int
    Definition Classes
    JavaRDDLike
  49. def intersection(other: JavaRDD[R]): JavaRDD[R]
    Definition Classes
    JavaRDD
  50. def isCheckpointed: Boolean
    Definition Classes
    JavaRDDLike
  51. def isEmpty(): Boolean
    Definition Classes
    JavaRDDLike
  52. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  53. def iterator(split: Partition, taskContext: TaskContext): Iterator[R]
    Definition Classes
    JavaRDDLike
  54. def keyBy[U](f: Function[R, U]): JavaPairRDD[U, R]
    Definition Classes
    JavaRDDLike
  55. def map[R](f: Function[R, R]): JavaRDD[R]
    Definition Classes
    JavaRDDLike
  56. def mapPartitions[U](f: FlatMapFunction[Iterator[R], U], preservesPartitioning: Boolean): JavaRDD[U]
    Definition Classes
    JavaRDDLike
  57. def mapPartitions[U](f: FlatMapFunction[Iterator[R], U]): JavaRDD[U]
    Definition Classes
    JavaRDDLike
  58. def mapPartitionsToDouble(f: DoubleFlatMapFunction[Iterator[R]], preservesPartitioning: Boolean): JavaDoubleRDD
    Definition Classes
    JavaRDDLike
  59. def mapPartitionsToDouble(f: DoubleFlatMapFunction[Iterator[R]]): JavaDoubleRDD
    Definition Classes
    JavaRDDLike
  60. def mapPartitionsToPair[K2, V2](f: PairFlatMapFunction[Iterator[R], K2, V2], preservesPartitioning: Boolean): JavaPairRDD[K2, V2]
    Definition Classes
    JavaRDDLike
  61. def mapPartitionsToPair[K2, V2](f: PairFlatMapFunction[Iterator[R], K2, V2]): JavaPairRDD[K2, V2]
    Definition Classes
    JavaRDDLike
  62. def mapPartitionsWithIndex[R](f: Function2[Integer, Iterator[R], Iterator[R]], preservesPartitioning: Boolean): JavaRDD[R]
    Definition Classes
    JavaRDDLike
  63. def mapToDouble[R](f: DoubleFunction[R]): JavaDoubleRDD
    Definition Classes
    JavaRDDLike
  64. def mapToPair[K2, V2](f: PairFunction[R, K2, V2]): JavaPairRDD[K2, V2]
    Definition Classes
    JavaRDDLike
  65. def max(comp: Comparator[R]): R
    Definition Classes
    JavaRDDLike
  66. def min(comp: Comparator[R]): R
    Definition Classes
    JavaRDDLike
  67. def name(): String
    Definition Classes
    JavaRDDLike
  68. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  69. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  70. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  71. def partitioner: Optional[Partitioner]
    Definition Classes
    JavaRDDLike
  72. def partitions: List[Partition]
    Definition Classes
    JavaRDDLike
  73. def persist(newLevel: StorageLevel): JavaRDD[R]
    Definition Classes
    JavaRDD
  74. def pipe(command: List[String], env: Map[String, String], separateWorkingDir: Boolean, bufferSize: Int, encoding: String): JavaRDD[String]
    Definition Classes
    JavaRDDLike
  75. def pipe(command: List[String], env: Map[String, String], separateWorkingDir: Boolean, bufferSize: Int): JavaRDD[String]
    Definition Classes
    JavaRDDLike
  76. def pipe(command: List[String], env: Map[String, String]): JavaRDD[String]
    Definition Classes
    JavaRDDLike
  77. def pipe(command: List[String]): JavaRDD[String]
    Definition Classes
    JavaRDDLike
  78. def pipe(command: String): JavaRDD[String]
    Definition Classes
    JavaRDDLike
  79. def randomSplit(weights: Array[Double], seed: Long): Array[JavaRDD[R]]
    Definition Classes
    JavaRDD
  80. def randomSplit(weights: Array[Double]): Array[JavaRDD[R]]
    Definition Classes
    JavaRDD
  81. def rdd(): TarantoolReadRDD[R]
    Definition Classes
    TarantoolJavaRDD → JavaRDD → JavaRDDLike
    Annotations
    @Override()
  82. def reduce(f: Function2[R, R, R]): R
    Definition Classes
    JavaRDDLike
  83. def repartition(numPartitions: Int): JavaRDD[R]
    Definition Classes
    JavaRDD
  84. def sample(withReplacement: Boolean, fraction: Double, seed: Long): JavaRDD[R]
    Definition Classes
    JavaRDD
  85. def sample(withReplacement: Boolean, fraction: Double): JavaRDD[R]
    Definition Classes
    JavaRDD
  86. def saveAsObjectFile(path: String): Unit
    Definition Classes
    JavaRDDLike
  87. def saveAsTextFile(path: String, codec: Class[_ <: CompressionCodec]): Unit
    Definition Classes
    JavaRDDLike
  88. def saveAsTextFile(path: String): Unit
    Definition Classes
    JavaRDDLike
  89. def setName(name: String): JavaRDD[R]
    Definition Classes
    JavaRDD
  90. def sortBy[S](f: Function[R, S], ascending: Boolean, numPartitions: Int): JavaRDD[R]
    Definition Classes
    JavaRDD
  91. def subtract(other: JavaRDD[R], p: Partitioner): JavaRDD[R]
    Definition Classes
    JavaRDD
  92. def subtract(other: JavaRDD[R], numPartitions: Int): JavaRDD[R]
    Definition Classes
    JavaRDD
  93. def subtract(other: JavaRDD[R]): JavaRDD[R]
    Definition Classes
    JavaRDD
  94. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  95. def take(num: Int): List[R]
    Definition Classes
    JavaRDDLike
  96. def takeAsync(num: Int): JavaFutureAction[List[R]]
    Definition Classes
    JavaRDDLike
  97. def takeOrdered(num: Int): List[R]
    Definition Classes
    JavaRDDLike
  98. def takeOrdered(num: Int, comp: Comparator[R]): List[R]
    Definition Classes
    JavaRDDLike
  99. def takeSample(withReplacement: Boolean, num: Int, seed: Long): List[R]
    Definition Classes
    JavaRDDLike
  100. def takeSample(withReplacement: Boolean, num: Int): List[R]
    Definition Classes
    JavaRDDLike
  101. def toDebugString(): String
    Definition Classes
    JavaRDDLike
  102. def toLocalIterator(): Iterator[R]
    Definition Classes
    JavaRDDLike
  103. def toString(): String
    Definition Classes
    JavaRDD → AnyRef → Any
  104. def top(num: Int): List[R]
    Definition Classes
    JavaRDDLike
  105. def top(num: Int, comp: Comparator[R]): List[R]
    Definition Classes
    JavaRDDLike
  106. def treeAggregate[U](zeroValue: U, seqOp: Function2[U, R, U], combOp: Function2[U, U, U]): U
    Definition Classes
    JavaRDDLike
  107. def treeAggregate[U](zeroValue: U, seqOp: Function2[U, R, U], combOp: Function2[U, U, U], depth: Int): U
    Definition Classes
    JavaRDDLike
  108. def treeReduce(f: Function2[R, R, R]): R
    Definition Classes
    JavaRDDLike
  109. def treeReduce(f: Function2[R, R, R], depth: Int): R
    Definition Classes
    JavaRDDLike
  110. def union(other: JavaRDD[R]): JavaRDD[R]
    Definition Classes
    JavaRDD
  111. def unpersist(blocking: Boolean): JavaRDD[R]
    Definition Classes
    JavaRDD
  112. def unpersist(): JavaRDD[R]
    Definition Classes
    JavaRDD
  113. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  114. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  115. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  116. def wrapRDD(rdd: RDD[R]): JavaRDD[R]
    Definition Classes
    JavaRDD → JavaRDDLike
  117. def zip[U](other: JavaRDDLike[U, _]): JavaPairRDD[R, U]
    Definition Classes
    JavaRDDLike
  118. def zipPartitions[U, V](other: JavaRDDLike[U, _], f: FlatMapFunction2[Iterator[R], Iterator[U], V]): JavaRDD[V]
    Definition Classes
    JavaRDDLike
  119. def zipWithIndex(): JavaPairRDD[R, Long]
    Definition Classes
    JavaRDDLike
  120. def zipWithUniqueId(): JavaPairRDD[R, Long]
    Definition Classes
    JavaRDDLike

Inherited from JavaRDD[R]

Inherited from AbstractJavaRDDLike[R, JavaRDD[R]]

Inherited from JavaRDDLike[R, JavaRDD[R]]

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped