FlinkMLTools

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
val EXECUTION_ENVIRONMENT_NAME: String
object ModuloKeyPartitioner extends Partitioner[Int]

Distributes the elements by taking the modulo of their keys and assigning it to this channel
final def asInstanceOf[T0]: T0

Definition Classes
Any
def block[T](input: DataSet[T], numBlocks: Int, partitionerOption: Option[Partitioner[Int]] = None)(implicit arg0: TypeInformation[T], arg1: ClassTag[T]): DataSet[Block[T]]

Groups the DataSet input into numBlocks blocks.
Groups the DataSet input into numBlocks blocks.
T
Type of the DataSet's elements
input
the input dataset
numBlocks
Number of Blocks
partitionerOption
Optional partitioner to control the partitioning
returns
The different datasets grouped into blocks
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
def hashCode(): Int

Definition Classes
AnyRef → Any
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
def persist[A, B, C, D, E](ds1: DataSet[A], ds2: DataSet[B], ds3: DataSet[C], ds4: DataSet[D], ds5: DataSet[E], path1: String, path2: String, path3: String, path4: String, path5: String)(implicit arg0: ClassTag[A], arg1: TypeInformation[A], arg2: ClassTag[B], arg3: TypeInformation[B], arg4: ClassTag[C], arg5: TypeInformation[C], arg6: ClassTag[D], arg7: TypeInformation[D], arg8: ClassTag[E], arg9: TypeInformation[E]): (DataSet[A], DataSet[B], DataSet[C], DataSet[D], DataSet[E])

Writes multiple DataSets to the specified paths and returns them as DataSources for subsequent operations.
Writes multiple DataSets to the specified paths and returns them as DataSources for subsequent operations.
A
Type of first DataSet's elements
B
Type of second DataSet's elements
C
Type of third DataSet's elements
D
Type of fourth DataSet's elements
E
Type of fifth DataSet's elements
ds1
First DataSet to write to disk
ds2
Second DataSet to write to disk
ds3
Third DataSet to write to disk
ds4
Fourth DataSet to write to disk
ds5
Fifth DataSet to write to disk
path1
Path for ds1
path2
Path for ds2
path3
Path for ds3
path4
Path for ds4
path5
Path for ds5
returns
Tuple of DataSets reading the just written files
def persist[A, B, C, D](ds1: DataSet[A], ds2: DataSet[B], ds3: DataSet[C], ds4: DataSet[D], path1: String, path2: String, path3: String, path4: String)(implicit arg0: ClassTag[A], arg1: TypeInformation[A], arg2: ClassTag[B], arg3: TypeInformation[B], arg4: ClassTag[C], arg5: TypeInformation[C], arg6: ClassTag[D], arg7: TypeInformation[D]): (DataSet[A], DataSet[B], DataSet[C], DataSet[D])

Writes multiple DataSets to the specified paths and returns them as DataSources for subsequent operations.
Writes multiple DataSets to the specified paths and returns them as DataSources for subsequent operations.
A
Type of first DataSet's elements
B
Type of second DataSet's elements
C
Type of third DataSet's elements
D
Type of fourth DataSet's elements
ds1
First DataSet to write to disk
ds2
Second DataSet to write to disk
ds3
Third DataSet to write to disk
ds4
Fourth DataSet to write to disk
path1
Path for ds1
path2
Path for ds2
path3
Path for ds3
path4
Path for ds4
returns
Tuple of DataSets reading the just written files
def persist[A, B, C](ds1: DataSet[A], ds2: DataSet[B], ds3: DataSet[C], path1: String, path2: String, path3: String)(implicit arg0: ClassTag[A], arg1: TypeInformation[A], arg2: ClassTag[B], arg3: TypeInformation[B], arg4: ClassTag[C], arg5: TypeInformation[C]): (DataSet[A], DataSet[B], DataSet[C])

Writes multiple DataSets to the specified paths and returns them as DataSources for subsequent operations.
Writes multiple DataSets to the specified paths and returns them as DataSources for subsequent operations.
A
Type of first DataSet's elements
B
Type of second DataSet's elements
C
Type of third DataSet's elements
ds1
First DataSet to write to disk
ds2
Second DataSet to write to disk
ds3
Third DataSet to write to disk
path1
Path for ds1
path2
Path for ds2
path3
Path for ds3
returns
Tuple of DataSets reading the just written files
def persist[A, B](ds1: DataSet[A], ds2: DataSet[B], path1: String, path2: String)(implicit arg0: ClassTag[A], arg1: TypeInformation[A], arg2: ClassTag[B], arg3: TypeInformation[B]): (DataSet[A], DataSet[B])

Writes multiple DataSets to the specified paths and returns them as DataSources for subsequent operations.
Writes multiple DataSets to the specified paths and returns them as DataSources for subsequent operations.
A
Type of the first DataSet's elements
B
Type of the second DataSet's elements
ds1
First DataSet to write to disk
ds2
Second DataSet to write to disk
path1
Path for ds1
path2
Path for ds2
returns
Tuple of DataSets reading the just written files
def persist[T](dataset: DataSet[T], path: String)(implicit arg0: ClassTag[T], arg1: TypeInformation[T]): DataSet[T]

Writes a DataSet to the specified path and returns it as a DataSource for subsequent operations.
Writes a DataSet to the specified path and returns it as a DataSource for subsequent operations.
T
Type of the DataSet elements
dataset
DataSet to write to disk
path
File path to write dataset to
returns
DataSet reading the just written file
def registerFlinkMLTypes(env: ExecutionEnvironment): Unit

Registers the different FlinkML related types for Kryo serialization
Registers the different FlinkML related types for Kryo serialization
env
The Flink execution environment where the types need to be registered
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
AnyRef → Any
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

Related Doc: package common

object FlinkMLTools

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

val EXECUTION_ENVIRONMENT_NAME: String

object ModuloKeyPartitioner extends Partitioner[Int]

final def asInstanceOf[T0]: T0

def block[T](input: DataSet[T], numBlocks: Int, partitionerOption: Option[Partitioner[Int]] = None)(implicit arg0: TypeInformation[T], arg1: ClassTag[T]): DataSet[Block[T]]

def clone(): AnyRef

final def eq(arg0: AnyRef): Boolean

def equals(arg0: Any): Boolean

def finalize(): Unit

final def getClass(): Class[_]

def hashCode(): Int

final def isInstanceOf[T0]: Boolean

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

def persist[A, B, C](ds1: DataSet[A], ds2: DataSet[B], ds3: DataSet[C], path1: String, path2: String, path3: String)(implicit arg0: ClassTag[A], arg1: TypeInformation[A], arg2: ClassTag[B], arg3: TypeInformation[B], arg4: ClassTag[C], arg5: TypeInformation[C]): (DataSet[A], DataSet[B], DataSet[C])

def persist[A, B](ds1: DataSet[A], ds2: DataSet[B], path1: String, path2: String)(implicit arg0: ClassTag[A], arg1: TypeInformation[A], arg2: ClassTag[B], arg3: TypeInformation[B]): (DataSet[A], DataSet[B])

def persist[T](dataset: DataSet[T], path: String)(implicit arg0: ClassTag[T], arg1: TypeInformation[T]): DataSet[T]

def registerFlinkMLTypes(env: ExecutionEnvironment): Unit

final def synchronized[T0](arg0: ⇒ T0): T0

def toString(): String

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Inherited from AnyRef

Inherited from Any

Ungrouped