org.apache.flink.python.api.streaming.data

Class PythonStreamer<S extends PythonSender,OUT>

java.lang.Object

org.apache.flink.python.api.streaming.data.PythonStreamer<S,OUT>

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

PythonDualInputStreamer, PythonSingleInputStreamer

public class PythonStreamer<S extends PythonSender,OUT>
extends Object
implements Serializable

This streamer is used by functions to send/receive data to/from an external python process.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected Thread	errorPrinter
protected AbstractRichFunction	function
protected DataInputStream	in
protected static org.slf4j.Logger	LOG
protected AtomicReference<String>	msg
protected DataOutputStream	out
protected Thread	outPrinter
protected int	port
protected PythonReceiver<OUT>	receiver
protected S	sender
protected ServerSocket	server
protected static int	SIGNAL_BUFFER_REQUEST
protected static int	SIGNAL_BUFFER_REQUEST_G0
protected static int	SIGNAL_BUFFER_REQUEST_G1
protected static int	SIGNAL_ERROR
protected static int	SIGNAL_FINISHED
protected static byte	SIGNAL_LAST
protected Socket	socket

Constructor Summary

Constructors

Constructor and Description
PythonStreamer(AbstractRichFunction function, Configuration config, int envID, int setID, boolean usesByteArray, S sender)

Method Summary

Methods

Modifier and Type	Method and Description
void	close() Closes this streamer.
static void	destroyProcess(Process process)
void	open() Starts the python script.
void	sendBroadCastVariables(Configuration config) Sends all broadcast-variables encoded in the configuration to the external process.
protected void	sendReadConfirmation()
protected void	sendWriteNotification(int size, boolean hasNext)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected static final org.slf4j.Logger LOG

SIGNAL_BUFFER_REQUEST

protected static final int SIGNAL_BUFFER_REQUEST

See Also:

Constant Field Values

SIGNAL_BUFFER_REQUEST_G0

protected static final int SIGNAL_BUFFER_REQUEST_G0

See Also:

Constant Field Values

SIGNAL_BUFFER_REQUEST_G1

protected static final int SIGNAL_BUFFER_REQUEST_G1

See Also:

Constant Field Values

SIGNAL_FINISHED

protected static final int SIGNAL_FINISHED

See Also:

Constant Field Values

SIGNAL_ERROR

protected static final int SIGNAL_ERROR

See Also:

Constant Field Values

SIGNAL_LAST

protected static final byte SIGNAL_LAST

See Also:

Constant Field Values

server

protected transient ServerSocket server

socket

protected transient Socket socket

in

protected transient DataInputStream in

out

protected transient DataOutputStream out

port

protected int port

sender

protected S extends PythonSender sender

receiver

protected PythonReceiver<OUT> receiver

msg

protected AtomicReference<String> msg

function

protected final AbstractRichFunction function

outPrinter

protected transient Thread outPrinter

errorPrinter

protected transient Thread errorPrinter

Constructor Detail

PythonStreamer

public PythonStreamer(AbstractRichFunction function,
              Configuration config,
              int envID,
              int setID,
              boolean usesByteArray,
              S sender)

Method Detail

open

public void open()
          throws IOException

Starts the python script.

Throws:

IOException

close

public void close()
           throws IOException

Closes this streamer.

Throws:

IOException

destroyProcess

public static void destroyProcess(Process process)
                           throws IOException

Throws:

IOException

sendWriteNotification

protected void sendWriteNotification(int size,
                         boolean hasNext)
                              throws IOException

Throws:

IOException

sendReadConfirmation

protected void sendReadConfirmation()
                             throws IOException

Throws:

IOException

sendBroadCastVariables

public final void sendBroadCastVariables(Configuration config)
                                  throws IOException

Sends all broadcast-variables encoded in the configuration to the external process.

Parameters:

config - configuration object containing broadcast-variable count and names

Throws:

IOException