Packages

object EvaluatePython

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. EvaluatePython
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Value Members

  1. def javaToPython(rdd: RDD[Any]): RDD[Array[Byte]]

    Convert an RDD of Java objects to an RDD of serialized Python objects, that is usable by PySpark.

  2. def makeFromJava(dataType: DataType): (Any) => Any

    Make a converter that converts obj to the type specified by the data type, or returns null if the type of obj is unexpected.

    Make a converter that converts obj to the type specified by the data type, or returns null if the type of obj is unexpected. Because Python doesn't enforce the type.

  3. def needConversionInPython(dt: DataType): Boolean
  4. def registerPicklers(): Unit

    This should be called before trying to serialize any above classes un cluster mode, this should be put in the closure

  5. def toJava(obj: Any, dataType: DataType): Any

    Helper for converting from Catalyst type to java type suitable for Pickle.