IN
- Type of the input elements.OUT
- Type of the output elements.UDFIN
- Type of the UDF input type.@Internal public abstract class AbstractPythonScalarFunctionOperator<IN,OUT,UDFIN> extends AbstractStatelessFunctionOperator<IN,OUT,UDFIN>
ScalarFunction
s. It executes the
Python ScalarFunction
s in separate Python execution environment.
The inputs are assumed as the following format: {{{ +------------------+--------------+ | forwarded fields | extra fields | +------------------+--------------+ }}}.
The Python UDFs may take input columns directly from the input row or the execution result of Java UDFs: 1) The input columns from the input row can be referred from the 'forwarded fields'; 2) The Java UDFs will be computed and the execution results can be referred from the 'extra fields'.
The outputs will be as the following format: {{{ +------------------+-------------------------+ | forwarded fields | scalar function results | +------------------+-------------------------+ }}}.
Modifier and Type | Field and Description |
---|---|
protected int[] |
forwardedFields
The offset of the fields which should be forwarded.
|
protected org.apache.flink.table.functions.python.PythonFunctionInfo[] |
scalarFunctions
The Python
ScalarFunction s to be executed. |
bais, baisWrapper, baos, baosWrapper, forwardedInputQueue, inputType, outputType, userDefinedFunctionInputOffsets, userDefinedFunctionInputType, userDefinedFunctionOutputType
elementCount, maxBundleSize, pythonFunctionRunner
Modifier and Type | Method and Description |
---|---|
String |
getFunctionUrn() |
String |
getInputOutputCoderUrn() |
org.apache.flink.table.functions.python.PythonEnv |
getPythonEnv()
Returns the
PythonEnv used to create PythonEnvironmentManager.. |
FlinkFnApi.UserDefinedFunctions |
getUserDefinedFunctionsProto()
Gets the proto representation of the Python user-defined functions to be executed.
|
void |
open() |
bufferInput, createPythonFunctionRunner, getFunctionInput, processElement, processElementInternal
endInput
checkInvokeFinishBundleByCount, close, createPythonEnvironmentManager, dispose, emitResult, emitResults, getConfig, getFlinkMetricContainer, getPythonConfig, invokeFinishBundle, isBundleFinished, prepareSnapshotPreBarrier, processWatermark, setCurrentKey, setPythonConfig
getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, reportOrForwardLatencyMarker, setChainingStrategy, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotState
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
setKeyContextElement
close, dispose, getMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState
notifyCheckpointAborted, notifyCheckpointComplete
protected final org.apache.flink.table.functions.python.PythonFunctionInfo[] scalarFunctions
ScalarFunction
s to be executed.protected final int[] forwardedFields
public void open() throws Exception
public org.apache.flink.table.functions.python.PythonEnv getPythonEnv()
AbstractPythonFunctionOperator
PythonEnv
used to create PythonEnvironmentManager..getPythonEnv
in class AbstractPythonFunctionOperator<OUT>
public FlinkFnApi.UserDefinedFunctions getUserDefinedFunctionsProto()
getUserDefinedFunctionsProto
in class AbstractStatelessFunctionOperator<IN,OUT,UDFIN>
public String getFunctionUrn()
getFunctionUrn
in class AbstractStatelessFunctionOperator<IN,OUT,UDFIN>
public String getInputOutputCoderUrn()
getInputOutputCoderUrn
in class AbstractStatelessFunctionOperator<IN,OUT,UDFIN>
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.