org.apache.flink.python.util

Class PythonConfigUtil

java.lang.Object

org.apache.flink.python.util.PythonConfigUtil

public class PythonConfigUtil
extends Object

A Util class to get the StreamExecutionEnvironment configuration and merged configuration with environment settings.

Field Summary

Fields

Modifier and Type	Field and Description
static String	KEYED_STREAM_VALUE_OPERATOR_NAME
static String	STREAM_KEY_BY_MAP_OPERATOR_NAME
static String	STREAM_PARTITION_CUSTOM_MAP_OPERATOR_NAME

Constructor Summary

Constructors

Constructor and Description
PythonConfigUtil()

Method Summary

Modifier and Type	Method and Description
static void	configPythonOperator(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env)
static void	declareManagedMemory(org.apache.flink.api.dag.Transformation<?> transformation, org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env, org.apache.flink.table.api.TableConfig tableConfig) Set Python Operator Use Managed Memory.
static org.apache.flink.streaming.api.graph.StreamGraph	generateStreamGraphWithDependencies(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env, boolean clearTransformations) Generate a StreamGraph for transformations maintained by current StreamExecutionEnvironment, and reset the merged env configurations with dependencies to every OneInputPythonFunctionOperator.
static org.apache.flink.configuration.Configuration	getEnvConfigWithDependencies(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env) A static method to get the StreamExecutionEnvironment configuration merged with python dependency management configurations.
static org.apache.flink.configuration.Configuration	getEnvironmentConfig(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env) Get the private method StreamExecutionEnvironment.getConfiguration() by reflection recursively.
static org.apache.flink.configuration.Configuration	getMergedConfig(org.apache.flink.api.java.ExecutionEnvironment env, org.apache.flink.table.api.TableConfig tableConfig)
static org.apache.flink.configuration.Configuration	getMergedConfig(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env, org.apache.flink.table.api.TableConfig tableConfig)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

KEYED_STREAM_VALUE_OPERATOR_NAME

public static final String KEYED_STREAM_VALUE_OPERATOR_NAME

See Also:

Constant Field Values

STREAM_KEY_BY_MAP_OPERATOR_NAME

public static final String STREAM_KEY_BY_MAP_OPERATOR_NAME

See Also:

Constant Field Values

STREAM_PARTITION_CUSTOM_MAP_OPERATOR_NAME

public static final String STREAM_PARTITION_CUSTOM_MAP_OPERATOR_NAME

See Also:

Constant Field Values

Constructor Detail

PythonConfigUtil

public PythonConfigUtil()

Method Detail

getEnvConfigWithDependencies

public static org.apache.flink.configuration.Configuration getEnvConfigWithDependencies(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env)
                                                                                 throws InvocationTargetException,
                                                                                        IllegalAccessException,
                                                                                        NoSuchMethodException

A static method to get the StreamExecutionEnvironment configuration merged with python dependency management configurations.

Throws:

InvocationTargetException

IllegalAccessException

NoSuchMethodException

getEnvironmentConfig

public static org.apache.flink.configuration.Configuration getEnvironmentConfig(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env)
                                                                         throws InvocationTargetException,
                                                                                IllegalAccessException,
                                                                                NoSuchMethodException

Get the private method StreamExecutionEnvironment.getConfiguration() by reflection recursively. Then access the method to get the configuration of the given StreamExecutionEnvironment.

Throws:

InvocationTargetException

IllegalAccessException

NoSuchMethodException

declareManagedMemory

public static void declareManagedMemory(org.apache.flink.api.dag.Transformation<?> transformation,
                                        org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env,
                                        org.apache.flink.table.api.TableConfig tableConfig)

Set Python Operator Use Managed Memory.

generateStreamGraphWithDependencies

public static org.apache.flink.streaming.api.graph.StreamGraph generateStreamGraphWithDependencies(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env,
                                                                                                   boolean clearTransformations)
                                                                                            throws IllegalAccessException,
                                                                                                   NoSuchMethodException,
                                                                                                   InvocationTargetException,
                                                                                                   NoSuchFieldException

Generate a StreamGraph for transformations maintained by current StreamExecutionEnvironment, and reset the merged env configurations with dependencies to every OneInputPythonFunctionOperator. It is an idempotent operation that can be call multiple times. Remember that only when need to execute the StreamGraph can we set the clearTransformations to be True.

Throws:

IllegalAccessException

NoSuchMethodException

InvocationTargetException

NoSuchFieldException

configPythonOperator

public static void configPythonOperator(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env)
                                 throws IllegalAccessException,
                                        NoSuchMethodException,
                                        InvocationTargetException,
                                        NoSuchFieldException

Throws:

IllegalAccessException

NoSuchMethodException

InvocationTargetException

NoSuchFieldException

getMergedConfig

public static org.apache.flink.configuration.Configuration getMergedConfig(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env,
                                                                           org.apache.flink.table.api.TableConfig tableConfig)

getMergedConfig

public static org.apache.flink.configuration.Configuration getMergedConfig(org.apache.flink.api.java.ExecutionEnvironment env,
                                                                           org.apache.flink.table.api.TableConfig tableConfig)