spark

Value Members

1. val SPARK_BRANCH: String

   

2. val SPARK_BUILD_DATE: String

   

3. val SPARK_BUILD_USER: String

   

4. val SPARK_REPO_URL: String

   

5. val SPARK_REVISION: String

   

6. val SPARK_VERSION: String

   

7. package status

   

8. package streaming

   

   Spark Streaming functionality.

   Spark Streaming functionality. org.apache.spark.streaming.StreamingContext serves as the main entry point to Spark Streaming, while org.apache.spark.streaming.dstream.DStream is the data type representing a continuous sequence of RDDs, representing a continuous stream of data.

   In addition, org.apache.spark.streaming.dstream.PairDStreamFunctions contains operations available only on DStreams of key-value pairs, such as groupByKey and reduceByKey. These operations are automatically available on any DStream of the right type (e.g. DStream[(Int, Int)] through implicit conversions.

   For the Java API of Spark Streaming, take a look at the org.apache.spark.streaming.api.java.JavaStreamingContext which serves as the entry point, and the org.apache.spark.streaming.api.java.JavaDStream and the org.apache.spark.streaming.api.java.JavaPairDStream which have the DStream functionality.