org.apache.flink.python.util

Class PythonDependencyUtils

java.lang.Object

org.apache.flink.python.util.PythonDependencyUtils

@Internal
public class PythonDependencyUtils
extends Object

Utility class for Python dependency management. The dependencies will be registered at the distributed cache.

Field Summary

Fields

Modifier and Type	Field and Description
static String	CACHE
static String	FILE
static String	FILE_DELIMITER
static String	PARAM_DELIMITER
static org.apache.flink.configuration.ConfigOption<Map<String,String>>	PYTHON_ARCHIVES
static org.apache.flink.configuration.ConfigOption<Map<String,String>>	PYTHON_FILES
static org.apache.flink.configuration.ConfigOption<Map<String,String>>	PYTHON_REQUIREMENTS_FILE

Constructor Summary

Constructors

Constructor and Description
PythonDependencyUtils()

Method Summary

Modifier and Type	Method and Description
static org.apache.flink.configuration.Configuration	configurePythonDependencies(List<org.apache.flink.api.java.tuple.Tuple2<String,org.apache.flink.api.common.cache.DistributedCache.DistributedCacheEntry>> cachedFiles, org.apache.flink.configuration.Configuration config) Adds python dependencies to registered cache file list according to given configuration and returns a new configuration which contains the metadata of the registered python dependencies.
static void	merge(org.apache.flink.configuration.Configuration config, org.apache.flink.configuration.Configuration pythonConfiguration)
static org.apache.flink.configuration.Configuration	parsePythonDependencyConfiguration(org.apache.commons.cli.CommandLine commandLine)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

FILE

public static final String FILE

See Also:

Constant Field Values

CACHE

public static final String CACHE

See Also:

Constant Field Values

FILE_DELIMITER

public static final String FILE_DELIMITER

See Also:

Constant Field Values

PARAM_DELIMITER

public static final String PARAM_DELIMITER

See Also:

Constant Field Values

PYTHON_FILES

public static final org.apache.flink.configuration.ConfigOption<Map<String,String>> PYTHON_FILES

PYTHON_REQUIREMENTS_FILE

public static final org.apache.flink.configuration.ConfigOption<Map<String,String>> PYTHON_REQUIREMENTS_FILE

PYTHON_ARCHIVES

public static final org.apache.flink.configuration.ConfigOption<Map<String,String>> PYTHON_ARCHIVES

Constructor Detail

PythonDependencyUtils

public PythonDependencyUtils()

Method Detail

configurePythonDependencies

public static org.apache.flink.configuration.Configuration configurePythonDependencies(List<org.apache.flink.api.java.tuple.Tuple2<String,org.apache.flink.api.common.cache.DistributedCache.DistributedCacheEntry>> cachedFiles,
                                                                                       org.apache.flink.configuration.Configuration config)

Adds python dependencies to registered cache file list according to given configuration and returns a new configuration which contains the metadata of the registered python dependencies.

Parameters:

cachedFiles - The list used to store registered cached files.

config - The configuration which contains python dependency configuration.

Returns:

A new configuration which contains the metadata of the registered python dependencies.

parsePythonDependencyConfiguration

public static org.apache.flink.configuration.Configuration parsePythonDependencyConfiguration(org.apache.commons.cli.CommandLine commandLine)

merge

public static void merge(org.apache.flink.configuration.Configuration config,
                         org.apache.flink.configuration.Configuration pythonConfiguration)