org.elasticsearch.spark.sql.streaming

Class SparkSqlStreamingConfigs$

java.lang.Object

org.elasticsearch.spark.sql.streaming.SparkSqlStreamingConfigs$

public class SparkSqlStreamingConfigs$
extends Object

Configurations specifically used for Spark Structured Streaming

Field Summary

Fields

Modifier and Type	Field and Description
static SparkSqlStreamingConfigs$	MODULE$ Static reference to the singleton instance of this Scala object.

Constructor Summary

Constructors

Constructor and Description
SparkSqlStreamingConfigs$()

Method Summary

Modifier and Type	Method and Description
String	constructCommitLogPath(Settings settings) Determines the location of the streaming commit log.
String	ES_INTERNAL_APP_ID()
String	ES_INTERNAL_APP_NAME()
String	ES_INTERNAL_QUERY_NAME()
String	ES_INTERNAL_SESSION_CHECKPOINT_LOCATION()
String	ES_INTERNAL_USER_CHECKPOINT_LOCATION()
long	ES_SINK_LOG_CLEANUP_DELAY_DEFAULT()
String	ES_SINK_LOG_CLEANUP_DELAY()
int	ES_SINK_LOG_COMPACT_INTERVAL_DEFAULT()
String	ES_SINK_LOG_COMPACT_INTERVAL()
boolean	ES_SINK_LOG_DELETION_DEFAULT()
String	ES_SINK_LOG_DELETION()
boolean	ES_SINK_LOG_ENABLE_DEFAULT()
String	ES_SINK_LOG_ENABLE()
String	ES_SINK_LOG_PATH()
scala.Option<String>	getAppId(Settings settings) The ID of the current Spark application, if set
scala.Option<String>	getAppName(Settings settings) The name of the current Spark application, if set
int	getDefaultCompactInterval(Settings settings)
long	getFileCleanupDelayMs(Settings settings) The number of milliseconds to wait before cleaning up compacted log files
boolean	getIsDeletingExpiredLog(Settings settings)
scala.Option<String>	getLogPath(Settings settings) The log path, if set, is the complete log path to be used for the commit log.
scala.Option<String>	getQueryName(Settings settings) The name of the current Spark Streaming Query, if set
scala.Option<String>	getSessionCheckpointLocation(Settings settings) The name of the default session checkpoint location, if set
boolean	getSinkLogEnabled(Settings settings) Determines if we should use the commit log for writes, or if we should go without one.
scala.Option<String>	getUserSpecifiedCheckpointLocation(Settings settings) The name of the user specified checkpoint location for the current Spark Streaming Query, if set

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MODULE$

public static final SparkSqlStreamingConfigs$ MODULE$

Static reference to the singleton instance of this Scala object.

Constructor Detail

SparkSqlStreamingConfigs$

public SparkSqlStreamingConfigs$()

Method Detail

ES_SINK_LOG_ENABLE

public String ES_SINK_LOG_ENABLE()

ES_SINK_LOG_ENABLE_DEFAULT

public boolean ES_SINK_LOG_ENABLE_DEFAULT()

ES_SINK_LOG_PATH

public String ES_SINK_LOG_PATH()

ES_INTERNAL_APP_NAME

public String ES_INTERNAL_APP_NAME()

ES_INTERNAL_APP_ID

public String ES_INTERNAL_APP_ID()

ES_INTERNAL_QUERY_NAME

public String ES_INTERNAL_QUERY_NAME()

ES_INTERNAL_USER_CHECKPOINT_LOCATION

public String ES_INTERNAL_USER_CHECKPOINT_LOCATION()

ES_INTERNAL_SESSION_CHECKPOINT_LOCATION

public String ES_INTERNAL_SESSION_CHECKPOINT_LOCATION()

ES_SINK_LOG_CLEANUP_DELAY

public String ES_SINK_LOG_CLEANUP_DELAY()

ES_SINK_LOG_CLEANUP_DELAY_DEFAULT

public long ES_SINK_LOG_CLEANUP_DELAY_DEFAULT()

ES_SINK_LOG_DELETION

public String ES_SINK_LOG_DELETION()

ES_SINK_LOG_DELETION_DEFAULT

public boolean ES_SINK_LOG_DELETION_DEFAULT()

ES_SINK_LOG_COMPACT_INTERVAL

public String ES_SINK_LOG_COMPACT_INTERVAL()

ES_SINK_LOG_COMPACT_INTERVAL_DEFAULT

public int ES_SINK_LOG_COMPACT_INTERVAL_DEFAULT()

getSinkLogEnabled

public boolean getSinkLogEnabled(Settings settings)

Determines if we should use the commit log for writes, or if we should go without one.

Parameters:

settings - connector settings

Returns:

true if we should use the commit log, false if we should not

constructCommitLogPath

public String constructCommitLogPath(Settings settings)

Determines the location of the streaming commit log.

Parameters:

settings - connector settings

Returns:

the location to use as the commit log

getLogPath

public scala.Option<String> getLogPath(Settings settings)

The log path, if set, is the complete log path to be used for the commit log.

Parameters:

settings - connector settings

Returns:

either Some log path or None

getAppName

public scala.Option<String> getAppName(Settings settings)

The name of the current Spark application, if set

Parameters:

settings - connector settings

Returns:

Some name or None

getAppId

public scala.Option<String> getAppId(Settings settings)

The ID of the current Spark application, if set

Parameters:

settings - connector settings

Returns:

Some id or None

getQueryName

public scala.Option<String> getQueryName(Settings settings)

The name of the current Spark Streaming Query, if set

Parameters:

settings - connector settings

Returns:

Some query name or None

getUserSpecifiedCheckpointLocation

public scala.Option<String> getUserSpecifiedCheckpointLocation(Settings settings)

The name of the user specified checkpoint location for the current Spark Streaming Query, if set

Parameters:

settings - connector settings

Returns:

Some checkpoint location or None

getSessionCheckpointLocation

public scala.Option<String> getSessionCheckpointLocation(Settings settings)

The name of the default session checkpoint location, if set

Parameters:

settings - connector settings

Returns:

Some checkpoint location or None

getFileCleanupDelayMs

public long getFileCleanupDelayMs(Settings settings)

The number of milliseconds to wait before cleaning up compacted log files

Parameters:

settings - connector settings

Returns:

time in millis if set, or the default delay

getIsDeletingExpiredLog

public boolean getIsDeletingExpiredLog(Settings settings)

Parameters:

settings - connector settings

Returns:

getDefaultCompactInterval

public int getDefaultCompactInterval(Settings settings)

Parameters:

settings - connector settings

Returns: