HttpRDD

Value Members

1. final def !=(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

2. final def ##(): Int

   

   Definition Classes

   AnyRef → Any

3. final def ==(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

4. final def asInstanceOf[T0]: T0

   

   Definition Classes

   Any

5. def clone(): AnyRef

   

   Attributes

   protected[java.lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( ... )

6. def create[T](sc: SparkContext, uri: URI, partitioning: Array[URIModifier], mapper: (String) ⇒ T)(implicit arg0: ClassTag[T]): RDD[T]

   

   Ctor with predefined amount of functionality, creates RDD from HTTP endpoint

   Ctor with predefined amount of functionality, creates RDD from HTTP endpoint

   sc

   - spark context

   uri

   - request uri

   partitioning

   - uri partitioner, define separate partitioning uri from the root uri

   mapper

   - transformer from String to T By default it expect to receive record per new line with default http client without any kind of request modifications

   returns

   Spark RDD

7. def create[T](sc: SparkContext, uri: URI, partitioning: Array[URIModifier], requestModifier: (HttpGet) ⇒ HttpGet, httpClientBuilder: () ⇒ CloseableHttpClient, iterable: (InputStream) ⇒ Iterator[T])(implicit arg0: ClassTag[T]): RDD[T]

   

   Base ctor that provide possibility to customize you rdd

   Base ctor that provide possibility to customize you rdd

   sc

   - spark context

   uri

   - request uri

   partitioning

   - uri partitioner, define separate partitioning uri from the root uri

   requestModifier

   - transformation that will be applied to http request before execution

   httpClientBuilder

   - http client builder

   iterable

   - http response content => iterable transformation

   returns

   Spark RDD

8. final def eq(arg0: AnyRef): Boolean

   

   Definition Classes

   AnyRef

9. def equals(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

10. def finalize(): Unit

    

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

11. final def getClass(): Class[_]

    

    Definition Classes

    AnyRef → Any

12. def hashCode(): Int

    

    Definition Classes

    AnyRef → Any

13. final def isInstanceOf[T0]: Boolean

    

    Definition Classes

    Any

14. final def ne(arg0: AnyRef): Boolean

    

    Definition Classes

    AnyRef

15. final def notify(): Unit

    

    Definition Classes

    AnyRef

16. final def notifyAll(): Unit

    

    Definition Classes

    AnyRef

17. final def synchronized[T0](arg0: ⇒ T0): T0

    

    Definition Classes

    AnyRef

18. def toString(): String

    

    Definition Classes

    AnyRef → Any

19. final def wait(): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

20. final def wait(arg0: Long, arg1: Int): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

21. final def wait(arg0: Long): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )