@Internal public abstract class HadoopOutputFormatBase<K,V,T> extends HadoopOutputFormatCommonBase<T> implements org.apache.flink.api.common.io.FinalizeOnMaster
Modifier and Type | Field and Description |
---|---|
protected static Object |
CLOSE_MUTEX |
protected org.apache.hadoop.conf.Configuration |
configuration |
protected static Object |
CONFIGURE_MUTEX |
protected org.apache.hadoop.mapreduce.TaskAttemptContext |
context |
protected org.apache.hadoop.mapreduce.OutputFormat<K,V> |
mapreduceOutputFormat |
protected static Object |
OPEN_MUTEX |
protected org.apache.hadoop.mapreduce.OutputCommitter |
outputCommitter |
protected org.apache.hadoop.mapreduce.RecordWriter<K,V> |
recordWriter |
protected int |
taskNumber |
credentials
Constructor and Description |
---|
HadoopOutputFormatBase(org.apache.hadoop.mapreduce.OutputFormat<K,V> mapreduceOutputFormat,
org.apache.hadoop.mapreduce.Job job) |
Modifier and Type | Method and Description |
---|---|
void |
close()
commit the task by moving the output file out from the temporary directory.
|
void |
configure(org.apache.flink.configuration.Configuration parameters) |
void |
finalizeGlobal(int parallelism) |
org.apache.hadoop.conf.Configuration |
getConfiguration() |
void |
open(int taskNumber,
int numTasks)
create the temporary output file for hadoop RecordWriter.
|
read, write
getRuntimeContext, setRuntimeContext
protected static final Object OPEN_MUTEX
protected static final Object CONFIGURE_MUTEX
protected static final Object CLOSE_MUTEX
protected org.apache.hadoop.conf.Configuration configuration
protected transient org.apache.hadoop.mapreduce.OutputCommitter outputCommitter
protected transient org.apache.hadoop.mapreduce.TaskAttemptContext context
protected transient int taskNumber
public org.apache.hadoop.conf.Configuration getConfiguration()
public void configure(org.apache.flink.configuration.Configuration parameters)
configure
in interface org.apache.flink.api.common.io.OutputFormat<T>
public void open(int taskNumber, int numTasks) throws IOException
open
in interface org.apache.flink.api.common.io.OutputFormat<T>
taskNumber
- The number of the parallel instance.numTasks
- The number of parallel tasks.IOException
public void close() throws IOException
close
in interface org.apache.flink.api.common.io.OutputFormat<T>
IOException
public void finalizeGlobal(int parallelism) throws IOException
finalizeGlobal
in interface org.apache.flink.api.common.io.FinalizeOnMaster
IOException
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.