@Internal public abstract class AbstractPythonStreamAggregateOperator extends AbstractOneInputPythonFunctionOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>
AbstractPythonStreamGroupAggregateOperator
and PythonStreamGroupWindowAggregateOperator
.Modifier and Type | Field and Description |
---|---|
protected org.apache.flink.core.memory.ByteArrayInputStreamWithPos |
bais
Reusable InputStream used to holding the execution results to be deserialized.
|
protected org.apache.flink.core.memory.DataInputViewStreamWrapper |
baisWrapper
InputStream Wrapper.
|
protected org.apache.flink.core.memory.ByteArrayOutputStreamWithPos |
baos
Reusable OutputStream used to holding the serialized input elements.
|
protected org.apache.flink.core.memory.DataOutputViewStreamWrapper |
baosWrapper
OutputStream Wrapper.
|
protected static String |
FLINK_AGGREGATE_FUNCTION_SCHEMA_CODER_URN |
protected org.apache.flink.table.types.logical.RowType |
inputType
The input logical type.
|
protected org.apache.flink.table.types.logical.RowType |
outputType
The output logical type.
|
protected StreamRecordRowDataWrappingCollector |
rowDataWrapper
The collector used to collect records.
|
protected org.apache.flink.table.types.logical.RowType |
userDefinedFunctionInputType
The user-defined function input logical type.
|
protected org.apache.flink.table.types.logical.RowType |
userDefinedFunctionOutputType
The user-defined function output logical type.
|
elementCount, maxBundleSize, pythonFunctionRunner
Constructor and Description |
---|
AbstractPythonStreamAggregateOperator(org.apache.flink.configuration.Configuration config,
org.apache.flink.table.types.logical.RowType inputType,
org.apache.flink.table.types.logical.RowType outputType,
org.apache.flink.table.functions.python.PythonAggregateFunctionInfo[] aggregateFunctions,
org.apache.flink.table.planner.typeutils.DataViewUtils.DataViewSpec[][] dataViewSpecs,
int[] grouping,
int indexOfCountStar,
boolean generateUpdateBefore,
String coderUrn,
FlinkFnApi.CoderParam.OutputMode outputMode) |
Modifier and Type | Method and Description |
---|---|
PythonFunctionRunner |
createPythonFunctionRunner()
Creates the
PythonFunctionRunner which is responsible for Python user-defined
function execution. |
Object |
getCurrentKey() |
abstract String |
getFunctionUrn() |
protected org.apache.flink.table.types.logical.RowType |
getKeyType() |
org.apache.flink.table.functions.python.PythonEnv |
getPythonEnv()
Returns the
PythonEnv used to create PythonEnvironmentManager.. |
abstract org.apache.flink.table.types.logical.RowType |
getUserDefinedFunctionInputType() |
abstract org.apache.flink.table.types.logical.RowType |
getUserDefinedFunctionOutputType() |
protected FlinkFnApi.UserDefinedAggregateFunctions |
getUserDefinedFunctionsProto()
Gets the proto representation of the Python user-defined aggregate functions to be executed.
|
void |
open() |
void |
processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element) |
abstract void |
processElementInternal(org.apache.flink.table.data.RowData value) |
void |
setCurrentKey(Object key)
As the beam state gRPC service will access the KeyedStateBackend in parallel with this
operator, we must override this method to prevent changing the current key of the
KeyedStateBackend while the beam service is handling requests.
|
endInput
checkInvokeFinishBundleByCount, close, createPythonEnvironmentManager, dispose, emitResult, emitResults, getConfig, getFlinkMetricContainer, getPythonConfig, invokeFinishBundle, isBundleFinished, prepareSnapshotPreBarrier, processWatermark, setPythonConfig
getChainingStrategy, getContainingTask, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, reportOrForwardLatencyMarker, setChainingStrategy, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotState
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
setKeyContextElement
close, dispose, getMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState
@VisibleForTesting protected static final String FLINK_AGGREGATE_FUNCTION_SCHEMA_CODER_URN
protected final org.apache.flink.table.types.logical.RowType inputType
protected final org.apache.flink.table.types.logical.RowType outputType
protected transient org.apache.flink.table.types.logical.RowType userDefinedFunctionInputType
protected transient org.apache.flink.table.types.logical.RowType userDefinedFunctionOutputType
protected transient org.apache.flink.core.memory.ByteArrayInputStreamWithPos bais
protected transient org.apache.flink.core.memory.DataInputViewStreamWrapper baisWrapper
protected transient org.apache.flink.core.memory.ByteArrayOutputStreamWithPos baos
protected transient org.apache.flink.core.memory.DataOutputViewStreamWrapper baosWrapper
protected transient StreamRecordRowDataWrappingCollector rowDataWrapper
public AbstractPythonStreamAggregateOperator(org.apache.flink.configuration.Configuration config, org.apache.flink.table.types.logical.RowType inputType, org.apache.flink.table.types.logical.RowType outputType, org.apache.flink.table.functions.python.PythonAggregateFunctionInfo[] aggregateFunctions, org.apache.flink.table.planner.typeutils.DataViewUtils.DataViewSpec[][] dataViewSpecs, int[] grouping, int indexOfCountStar, boolean generateUpdateBefore, String coderUrn, FlinkFnApi.CoderParam.OutputMode outputMode)
public void open() throws Exception
open
in interface org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>
open
in class AbstractPythonFunctionOperator<org.apache.flink.table.data.RowData>
Exception
public void processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element) throws Exception
Exception
public PythonFunctionRunner createPythonFunctionRunner() throws Exception
AbstractPythonFunctionOperator
PythonFunctionRunner
which is responsible for Python user-defined
function execution.createPythonFunctionRunner
in class AbstractPythonFunctionOperator<org.apache.flink.table.data.RowData>
Exception
public void setCurrentKey(Object key)
setCurrentKey
in interface org.apache.flink.streaming.api.operators.KeyContext
setCurrentKey
in class AbstractPythonFunctionOperator<org.apache.flink.table.data.RowData>
public Object getCurrentKey()
getCurrentKey
in interface org.apache.flink.streaming.api.operators.KeyContext
getCurrentKey
in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>
public org.apache.flink.table.functions.python.PythonEnv getPythonEnv()
AbstractPythonFunctionOperator
PythonEnv
used to create PythonEnvironmentManager..getPythonEnv
in class AbstractPythonFunctionOperator<org.apache.flink.table.data.RowData>
protected org.apache.flink.table.types.logical.RowType getKeyType()
protected FlinkFnApi.UserDefinedAggregateFunctions getUserDefinedFunctionsProto()
public abstract String getFunctionUrn()
public abstract void processElementInternal(org.apache.flink.table.data.RowData value) throws Exception
Exception
public abstract org.apache.flink.table.types.logical.RowType getUserDefinedFunctionInputType()
public abstract org.apache.flink.table.types.logical.RowType getUserDefinedFunctionOutputType()
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.