com.google.cloud.hadoop.io.bigquery

Class AbstractExportToCloudStorage

java.lang.Object

com.google.cloud.hadoop.io.bigquery.AbstractExportToCloudStorage

All Implemented Interfaces:

Export

Direct Known Subclasses:

ShardedExportToCloudStorage, UnshardedExportToCloudStorage

public abstract class AbstractExportToCloudStorage
extends java.lang.Object
implements Export

An abstract base class for BigQuery exports to GCS for use by MapReduce job setup

Field Summary

Fields

Modifier and Type	Field and Description
protected BigQueryHelper	bigQueryHelper
protected org.apache.hadoop.conf.Configuration	configuration
static java.lang.String	DESTINATION_FORMAT_KEY
protected com.google.api.services.bigquery.model.JobReference	exportJobReference
protected ExportFileFormat	fileFormat
protected java.lang.String	gcsPath
protected java.lang.String	projectId
protected com.google.api.services.bigquery.model.Table	tableToExport

Constructor Summary

Constructors

Constructor and Description
AbstractExportToCloudStorage(org.apache.hadoop.conf.Configuration configuration, java.lang.String gcsPath, ExportFileFormat fileFormat, BigQueryHelper bigQueryHelper, java.lang.String projectId, com.google.api.services.bigquery.model.Table tableToExport)

Method Summary

Modifier and Type	Method and Description
void	beginExport() Start exporting data
void	cleanupExport() Delete any temp tables or temporary data locations.
void	prepare() Create any temporary directories, tables, etc

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.google.cloud.hadoop.io.bigquery.Export

getExportPaths, getSplits, waitForUsableMapReduceInput

Field Detail

DESTINATION_FORMAT_KEY

public static final java.lang.String DESTINATION_FORMAT_KEY

See Also:

Constant Field Values

configuration

protected final org.apache.hadoop.conf.Configuration configuration

gcsPath

protected final java.lang.String gcsPath

fileFormat

protected final ExportFileFormat fileFormat

bigQueryHelper

protected final BigQueryHelper bigQueryHelper

projectId

protected final java.lang.String projectId

tableToExport

protected final com.google.api.services.bigquery.model.Table tableToExport

exportJobReference

protected com.google.api.services.bigquery.model.JobReference exportJobReference

Constructor Detail

AbstractExportToCloudStorage

public AbstractExportToCloudStorage(org.apache.hadoop.conf.Configuration configuration,
                                    java.lang.String gcsPath,
                                    ExportFileFormat fileFormat,
                                    BigQueryHelper bigQueryHelper,
                                    java.lang.String projectId,
                                    com.google.api.services.bigquery.model.Table tableToExport)

Method Detail

prepare

public void prepare()
             throws java.io.IOException

Description copied from interface: Export

Create any temporary directories, tables, etc

Specified by:

prepare in interface Export

Throws:

java.io.IOException

beginExport

public void beginExport()
                 throws java.io.IOException

Description copied from interface: Export

Start exporting data

Specified by:

beginExport in interface Export

Throws:

java.io.IOException

cleanupExport

public void cleanupExport()
                   throws java.io.IOException

Description copied from interface: Export

Delete any temp tables or temporary data locations.

Specified by:

cleanupExport in interface Export

Throws:

java.io.IOException