SparkSessionAPISparkImpl

Value Members

1. final def !=(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

2. final def ##(): Int

   

   Definition Classes

   AnyRef → Any

3. final def ==(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

4. final def asInstanceOf[T0]: T0

   

   Definition Classes

   Any

5. def broadcast[T](value: T)(implicit arg0: ClassTag[T]): BroadcastAPI[T]

   

   Broadcast a read-only variable to the cluster, returning a broadcast object for reading it in distributed functions.

   Broadcast a read-only variable to the cluster, returning a broadcast object for reading it in distributed functions. The variable will be sent to each cluster only once.

   Definition Classes

   SparkSessionAPISparkImpl → SparkSessionAPI

6. def clone(): AnyRef

   

   Attributes

   protected[java.lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( ... )

7. def collectionAccumulator[T](name: String): CollectionAccumulatorAPI[T]

   

   Create and register a CollectionAccumulator, which starts with empty list and accumulates inputs by adding them into the list.

   Create and register a CollectionAccumulator, which starts with empty list and accumulates inputs by adding them into the list.

   Definition Classes

   SparkSessionAPISparkImpl → SparkSessionAPI

8. def collectionAccumulator[T]: CollectionAccumulatorAPI[T]

   

   Create and register a CollectionAccumulator, which starts with empty list and accumulates inputs by adding them into the list.

   Create and register a CollectionAccumulator, which starts with empty list and accumulates inputs by adding them into the list.

   Definition Classes

   SparkSessionAPISparkImpl → SparkSessionAPI

9. def createDataset[T](data: Seq[T])(implicit arg0: ClassTag[T], enc: Encoder[T]): DataSetAPI[T]

   

   Create new DataSet based on Scala collection

   Create new DataSet based on Scala collection

   Definition Classes

   SparkSessionAPISparkImpl → SparkSessionAPI

10. def createDataset[T](data: RDDAPI[T])(implicit arg0: ClassTag[T], enc: Encoder[T]): DataSetAPI[T]

    

    Create new DataSet based on RDD

    Create new DataSet based on RDD

    Definition Classes

    SparkSessionAPI

11. def createRDD[T](data: Seq[T])(implicit arg0: ClassTag[T]): RDDAPI[T]

    

    Create new RDD based on Scala collection

    Create new RDD based on Scala collection

    Definition Classes

    SparkSessionAPISparkImpl → SparkSessionAPI

12. def doubleAccumulator(name: String): DoubleAccumulatorAPI

    

    Create and register a double accumulator, which starts with 0 and accumulates inputs by add.

    Create and register a double accumulator, which starts with 0 and accumulates inputs by add.

    Definition Classes

    SparkSessionAPISparkImpl → SparkSessionAPI

13. def doubleAccumulator: DoubleAccumulatorAPI

    

    Create and register a double accumulator, which starts with 0 and accumulates inputs by add.

    Create and register a double accumulator, which starts with 0 and accumulates inputs by add.

    Definition Classes

    SparkSessionAPISparkImpl → SparkSessionAPI

14. final def eq(arg0: AnyRef): Boolean

    

    Definition Classes

    AnyRef

15. def equals(arg0: Any): Boolean

    

    Definition Classes

    AnyRef → Any

16. def finalize(): Unit

    

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

17. final def getClass(): Class[_]

    

    Definition Classes

    AnyRef → Any

18. def hashCode(): Int

    

    Definition Classes

    AnyRef → Any

19. object implicitImpl extends SQLImplicits

    

20. val implicits: SQLImplicits

    

21. final def isInstanceOf[T0]: Boolean

    

    Definition Classes

    Any

22. def longAccumulator(name: String): LongAccumulatorAPI

    

    Create and register a long accumulator, which starts with 0 and accumulates inputs by add.

    Create and register a long accumulator, which starts with 0 and accumulates inputs by add.

    Definition Classes

    SparkSessionAPISparkImpl → SparkSessionAPI

23. def longAccumulator: LongAccumulatorAPI

    

    Create and register a long accumulator, which starts with 0 and accumulates inputs by add.

    Create and register a long accumulator, which starts with 0 and accumulates inputs by add.

    Definition Classes

    SparkSessionAPISparkImpl → SparkSessionAPI

24. final def ne(arg0: AnyRef): Boolean

    

    Definition Classes

    AnyRef

25. final def notify(): Unit

    

    Definition Classes

    AnyRef

26. final def notifyAll(): Unit

    

    Definition Classes

    AnyRef

27. def read[T]: ReaderExecutor[T]

    

    Returns a ReaderExecutor that can be used to read non-streaming data in as a DataSet

    Returns a ReaderExecutor that can be used to read non-streaming data in as a DataSet

    Definition Classes

    SparkSessionAPISparkImpl → SparkSessionAPI

28. def register(acc: AccumulatorV2API[_, _]): Unit

    

    Register the given accumulator.

    Register the given accumulator.

    Definition Classes

    SparkSessionAPISparkImpl → SparkSessionAPI

29. def register(acc: AccumulatorV2API[_, _], name: String): Unit

    

    Register the given accumulator with given name.

    Register the given accumulator with given name.

    Definition Classes

    SparkSessionAPISparkImpl → SparkSessionAPI

30. final def synchronized[T0](arg0: ⇒ T0): T0

    

    Definition Classes

    AnyRef

31. def textFile(path: String, minPartitions: Int = 2): RDDAPI[String]

    

    Read a text file from HDFS, a local file system (available on all nodes), or any Hadoop-supported file system URI, and return it as an RDD of Strings.

    Read a text file from HDFS, a local file system (available on all nodes), or any Hadoop-supported file system URI, and return it as an RDD of Strings.

    Definition Classes

    SparkSessionAPISparkImpl → SparkSessionAPI

32. def toString(): String

    

    Definition Classes

    AnyRef → Any

33. final def wait(): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

34. final def wait(arg0: Long, arg1: Int): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

35. final def wait(arg0: Long): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )