Create direct loghub DStream.
Create direct loghub DStream.
Set cursorStartTime
a valid value when using LogHubCursorPosition.SPECIAL_TIMER_CURSOR mode
at the first time with current mConsumerGroup
StreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
The Aliyun Access Key Id.
The Aliyun Access Key Secret.
The endpoint of loghub.
Zookeeper parameters.
Set user defined cursor type.
Set user defined cursor position (Unix Timestamp).
(Since version 1.8.0)
Create direct loghub DStream.
Create direct loghub DStream.
JavaStreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
The Aliyun Access Key Id.
The Aliyun Access Key Secret.
The endpoint of loghub.
Zookeeper parameters.
Set user defined cursor type.
(Since version 1.8.0)
Create direct loghub DStream.
Create direct loghub DStream.
Set cursorStartTime
a valid value when using LogHubCursorPosition.SPECIAL_TIMER_CURSOR mode
at the first time with current mConsumerGroup
StreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
The Aliyun Access Key Id.
The Aliyun Access Key Secret.
The endpoint of loghub.
Zookeeper parameters.
Set user defined cursor type.
Set user defined cursor position (Unix Timestamp).
(Since version 1.8.0)
Create direct loghub DStream.
Create direct loghub DStream.
StreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
The Aliyun Access Key Id.
The Aliyun Access Key Secret.
The endpoint of loghub.
Zookeeper parameters.
Set user defined cursor type.
(Since version 1.8.0)
(Since version 1.8.0)
Create loghub DStream.
Create loghub DStream.
JavaStreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
The number of receivers.
Storage level to use for storing the received objects.
Set user defined cursor type.
Set user defined cursor position (Unix Timestamp).
Whether to force to set consume position as the
mLoghubCursorStartTime
.
(Since version 1.8.0)
Create loghub DStream.
Create loghub DStream.
JavaStreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
The endpoint of loghub.
The number of receivers.
The Aliyun Access Key Id.
The Aliyun Access Key Secret.
Storage level to use for storing the received objects.
Set user defined cursor type.
Set user defined cursor position (Unix Timestamp).
Whether to force to set consume position as the
mLoghubCursorStartTime
.
(Since version 1.8.0)
Create loghub DStream.
Create loghub DStream.
JavaStreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
Storage level to use for storing the received objects.
Set user defined cursor type.
Set user defined cursor position (Unix Timestamp).
Whether to force to set consume position as the
mLoghubCursorStartTime
.
(Since version 1.8.0)
Create loghub DStream.
Create loghub DStream.
JavaStreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
The endpoint of loghub.
The Aliyun Access Key Id.
The Aliyun Access Key Secret.
Storage level to use for storing the received objects.
Set user defined cursor type.
Set user defined cursor position (Unix Timestamp).
Whether to force to set consume position as the
mLoghubCursorStartTime
.
(Since version 1.8.0)
Create loghub DStream.
Create loghub DStream.
JavaStreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
The number of receivers.
Storage level to use for storing the received objects.
(Since version 1.8.0)
Create loghub DStream.
Create loghub DStream.
JavaStreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
The endpoint of loghub.
The number of receivers.
The Aliyun Access Key Id.
The Aliyun Access Key Secret.
Storage level to use for storing the received objects.
(Since version 1.8.0)
Create loghub DStream.
Create loghub DStream.
JavaStreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
Storage level to use for storing the received objects.
(Since version 1.8.0)
Create loghub DStream.
Create loghub DStream.
JavaStreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
The endpoint of loghub.
The Aliyun Access Key Id.
The Aliyun Access Key Secret.
Storage level to use for storing the received objects.
(Since version 1.8.0)
Create loghub DStream.
Create loghub DStream.
StreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
The number of receivers.
Storage level to use for storing the received objects.
Set user defined cursor type.
Set user defined cursor position (Unix Timestamp).
Whether to force to set consume position as the
mLoghubCursorStartTime
.
(Since version 1.8.0)
Create loghub DStream.
Create loghub DStream.
StreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
The endpoint of loghub.
The number of receivers.
The Aliyun Access Key Id.
The Aliyun Access Key Secret.
Storage level to use for storing the received objects.
Set user defined cursor type.
Set user defined cursor position (Unix Timestamp).
Whether to force to set consume position as the
mLoghubCursorStartTime
.
(Since version 1.8.0)
Create loghub DStream.
Create loghub DStream.
StreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
Storage level to use for storing the received objects.
Set user defined cursor type.
Set user defined cursor position (Unix Timestamp).
Whether to force to set consume position as the
mLoghubCursorStartTime
.
(Since version 1.8.0)
Create loghub DStream.
Create loghub DStream.
StreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
The endpoint of loghub.
The Aliyun Access Key Id.
The Aliyun Access Key Secret.
Storage level to use for storing the received objects.
Set user defined cursor type.
Set user defined cursor position (Unix Timestamp).
Whether to force to set consume position as the
mLoghubCursorStartTime
.
(Since version 1.8.0)
Create loghub DStream.
Create loghub DStream.
val loghubProject = "sample-project" val logStream = "sample-logstore" val loghubGroupName = "sample-group" val numReceivers = 2 val batchInterval = Milliseconds(5 * 1000) val conf = new SparkConf().setAppName("Test Loghub") val ssc = new StreamingContext(conf, batchInterval) val loghubStream = LoghubUtils.createStream( ssc, loghubProject, logStream, loghubGroupName, numReceivers, StorageLevel.MEMORY_AND_DISK)
StreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
The number of receivers.
Storage level to use for storing the received objects.
(Since version 1.8.0)
Create loghub DStream.
Create loghub DStream.
val loghubProject = "sample-project" val logStream = "sample-logstore" val loghubGroupName = "sample-group" val endpoint = "cn-hangzhou-intranet.sls.aliyuncs.com" val accessKeyId = "kj7aY*******UYx6" val accessKeySecret = "AiNMAlxz*************1PxaPaL8t" val numReceivers = 2 val batchInterval = Milliseconds(5 * 1000) val conf = new SparkConf().setAppName("Test Loghub") val ssc = new StreamingContext(conf, batchInterval) val loghubStream = LoghubUtils.createStream( ssc, loghubProject, logStream, loghubGroupName, endpoint, numReceivers, accessKeyId, accessKeySecret, StorageLevel.MEMORY_AND_DISK)
StreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
The endpoint of loghub.
The number of receivers.
The Aliyun Access Key Id.
The Aliyun Access Key Secret.
Storage level to use for storing the received objects.
(Since version 1.8.0)
val loghubProject = "sample-project" val logStream = "sample-logstore" val loghubGroupName = "sample-group" val batchInterval = Milliseconds(5 * 1000) val conf = new SparkConf().setAppName("Test Loghub") val ssc = new StreamingContext(conf, batchInterval) val loghubStream = LoghubUtils.createStream( ssc, loghubProject, logStream, loghubGroupName, StorageLevel.MEMORY_AND_DISK)
StreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
Storage level to use for storing the received objects.
(Since version 1.8.0)
val loghubProject = "sample-project" val logStream = "sample-logstore" val loghubGroupName = "sample-group" val endpoint = "cn-hangzhou-intranet.sls.aliyuncs.com" val accessKeyId = "kj7aY*******UYx6" val accessKeySecret = "AiNMAlxz*************1PxaPaL8t" val batchInterval = Milliseconds(5 * 1000) val conf = new SparkConf().setAppName("Test Loghub") val ssc = new StreamingContext(conf, batchInterval) val loghubStream = LoghubUtils.createStream( ssc, loghubProject, logStream, loghubGroupName, endpoint, accessKeyId, accessKeySecret, StorageLevel.MEMORY_AND_DISK)
StreamingContext.
The name of LogService
project.
The name of logStore.
The group name of loghub consumer. All consumer process which has the same group name will consumer specific logStore together.
The endpoint of loghub.
The Aliyun Access Key Id.
The Aliyun Access Key Secret.
Storage level to use for storing the received objects.
(Since version 1.8.0)
Various utility classes for working with Aliyun LogService.