Packages

  • package root
    Definition Classes
    root
  • package io
    Definition Classes
    root
  • package tarantool
    Definition Classes
    io
  • package spark
    Definition Classes
    tarantool
  • package connector

    Tarantool connector for Apache Spark.

    Tarantool connector for Apache Spark.

    Call tarantoolSpace method on the SparkContext object to create a TarantoolRDD exposing Tarantool space as a Spark RDD.

    Example:

    Execute the following on a Cartridge router node (the tarantool/crud module must be installed):

    local crud = require('crud')

crud.insert('test_space', {1, nil, 'a1', 'Don Quixote', 'Miguel de Cervantes', 1605})
crud.insert('test_space', {2, nil, 'a2', 'The Great Gatsby', 'F. Scott Fitzgerald', 1925})
crud.insert('test_space', {3, nil, 'a3', 'War and Peace', 'Leo Tolstoy', 1869})

    Write the following in your Java client code:

    import io.tarantool.spark.connector._

val sparkMasterHost = "127.0.0.1"
val tarantoolRouterAddress = "127.0.0.1:3301"
val space = "test_space"

// Populate the Spark config with the address of a Cartridge router node and credentials:
val conf = new SparkConf(true)
conf.set ("tarantool.username", "admin")
conf.set ("tarantool.password", "testapp-cluster-cookie")
conf.set ("tarantool.hosts", tarantoolRouterAddress)

// Connect to the Spark cluster:
val sc = new SparkContext("spark://" + sparkMasterHost + ":7077", "example", conf)

// Read the space and print its contents:
val rdd = sc.tarantoolSpace(space)
rdd.toArray().foreach(println)

sc.stop()
    Definition Classes
    spark
  • package config
    Definition Classes
    connector
  • package connection
    Definition Classes
    connector
  • package partition
    Definition Classes
    connector
  • package rdd
    Definition Classes
    connector
  • package util
    Definition Classes
    connector
  • Logging
  • SparkContextFunctions
  • SparkContextJavaFunctions
  • TarantoolSpark
  • TarantoolSparkException
c

io.tarantool.spark.connector

SparkContextJavaFunctions

class SparkContextJavaFunctions extends AnyRef

Java API for bridging SparkContextFunctions functionality into Java code

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SparkContextJavaFunctions
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new SparkContextJavaFunctions(sparkContext: SparkContext)

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable])
  9. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  12. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  13. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  14. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  15. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  16. def tarantoolSpace[R](spaceName: String, conditions: Conditions, tupleConverter: SerializableFunction[TarantoolTuple, R], targetClass: Class[R]): TarantoolJavaRDD[R]

    Load data from Tarantool space.

    Load data from Tarantool space.

    The resulting tuples are converted into instances of the specified type using the provided converter.

    spaceName

    Tarantool space name

    conditions

    filtering conditions for space

    tupleConverter

    custom converter from Tarantool tuples to a target entity class

    targetClass

    target entity class

    returns

    instance of TarantoolJavaRDD

  17. def tarantoolSpace[R](spaceName: String, conditions: Conditions, tupleConverterFactory: TupleConverterFactory[R]): TarantoolJavaRDD[R]

    Load data from a Tarantool space, filtering them with the specified conditions.

    Load data from a Tarantool space, filtering them with the specified conditions.

    Tarantool tuples are converted into the target entity type using a converter provided by the specified tuple converter factory.

    spaceName

    Tarantool space name

    conditions

    space filtering conditions

    tupleConverterFactory

    provides the converter of TarantoolTuple into the target entity type R

    returns

    instance of TarantoolReadRDD

  18. def tarantoolSpace(spaceName: String, conditions: Conditions): TarantoolJavaRDD[TarantoolTuple]

    Load data from a Tarantool space, filtering them with the specified conditions.

    Load data from a Tarantool space, filtering them with the specified conditions. The resulting RDD is filled with TarantoolTuple.

    Example: 

    local crud = require('crud')

crud.insert('test_space', {1, nil, 'a1', 'Don Quixote', 'Miguel de Cervantes', 1605})
crud.insert('test_space', {2, nil, 'a2', 'The Great Gatsby', 'F. Scott Fitzgerald', 1925})
crud.insert('test_space', {3, nil, 'a3', 'War and Peace', 'Leo Tolstoy', 1869})
...

TarantoolJavaRDD rdd = TarantoolSpark.contextFunctions(jsc)
     .tarantoolSpace("test_space", Conditions.indexGreaterThan("id", Collections.singletonList(1)));
rdd.first().getInteger("id"); // 1
rdd.first().getString("author"); // "Miguel de Cervantes"

    spaceName

    Tarantool space name

    conditions

    space filtering conditions

    returns

    instance of TarantoolReadRDD

  19. def toJavaRDD[T](rdd: TarantoolReadRDD[T], targetClass: Class[T]): TarantoolJavaRDD[T]

    Converts TarantoolReadRDD into TarantoolJavaRDD

    Converts TarantoolReadRDD into TarantoolJavaRDD

    rdd

    TarantoolReadRDD instance

    targetClass

    target POJO class

    returns

    TarantoolJavaRDD instance

  20. def toString(): String
    Definition Classes
    AnyRef → Any
  21. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  22. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  23. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()

Inherited from AnyRef

Inherited from Any

Ungrouped