This is a input stream just for the testsuites.
This is a input stream just for the testsuites. This is equivalent to a checkpointable, replayable, reliable message queue like Kafka. It requires a sequence as input, and returns the i_th element at the i_th batch under manual clock.
Runs the streams set up in ssc
on manual clock for numBatches
batches and
returns the collected output.
Runs the streams set up in ssc
on manual clock for numBatches
batches and
returns the collected output. It will wait until numExpectedOutput
number of
output data has been collected or timeout (set by maxWaitTimeMillis
) is reached.
Returns a sequence of items for each RDD.
Set up required DStreams to test the binary operation using the sequence of input collections.
Set up required DStreams to test the DStream operation using the two sequences of input collections.
Test binary DStream operation with two lists of inputs
Test binary DStream operation with two lists of inputs
First sequence of input collections
Second sequence of input collections
Binary DStream operation to be applied to the 2 inputs
Sequence of expected output collections
Number of batches to run the operation for
Compare the output values with the expected output values as sets (order matters) or as lists (order does not matter)
Test binary DStream operation with two lists of inputs, with number of batches to run same as the number of expected output values
Test unary DStream operation with a list of inputs
Test unary DStream operation with a list of inputs
Sequence of input collections
Binary DStream operation to be applied to the 2 inputs
Sequence of expected output collections
Number of batches to run the operation for
Compare the output values with the expected output values as sets (order matters) or as lists (order does not matter)
Test unary DStream operation with a list of inputs, with number of batches to run same as the number of expected output values
Verify whether the output values after running a DStream operation is same as the expected output values, by comparing the output collections either as lists (order matters) or sets (order does not matter)
Run a block of code with the given StreamingContext and automatically stop the context when the block completes or when an exception is thrown.
Please use the afterAll(ConfigMap) method of trait BeforeAndAfterAllConfigMap instead.
Please use the afterEach(TestData) method of trait BeforeAndAfterEachTestData instead.
Please use the afterEach(TestData) method of trait BeforeAndAfterEachTestData instead.
This method has been deprecated in favor of macro assertion and will be removed in a future version of ScalaTest. If you need this, please copy the source code into your own trait instead.
This method has been deprecated in favor of macro assertion and will be removed in a future version of ScalaTest. If you need this, please copy the source code into your own trait instead.
This method has been deprecated in favor of macro assumption and will be removed in a future version of ScalaTest. If you need this, please copy the source code into your own trait instead.
This method has been deprecated in favor of macro assumption and will be removed in a future version of ScalaTest. If you need this, please copy the source code into your own trait instead.
Please use the beforeAll(ConfigMap) method of trait BeforeAndAfterAllConfigMap instead.
Please use the beforeEach(TestData) method of trait BeforeAndAfterEachTestData instead.
Please use the beforeEach(TestData) method of trait BeforeAndAfterEachTestData instead.
This expect method has been deprecated. Please replace all invocations of expect with an identical invocation of assertResult instead.
This expect method has been deprecated. Please replace all invocations of expect with an identical invocation of assertResult instead.
This expectResult method has been deprecated. Please replace all invocations of expectResult with an identical invocation of assertResult instead.
This expectResult method has been deprecated. Please replace all invocations of expectResult with an identical invocation of assertResult instead.
This is the base trait for Spark Streaming testsuites. This provides basic functionality to run user-defined set of input on user-defined stream operations, and verify the output.