A trait that can be extended when testing a source.
Advance the trigger clock's time manually.
Assert that a body is true
Assert that a condition on the active query is true
Signals that a failure is expected and should not kill the test.
Signals that a failure is expected and should not kill the test.
a function to verify the error.
if this is a fatal error. If so, the error should also be caught by UncaughtExceptionHandler.
A trait that can be extended when testing a source.
Starts the stream, resuming if data has already been processed.
Starts the stream, resuming if data has already been processed. It must not be running.
A trait for actions that can be performed while testing a streaming DataFrame.
A trait to mark actions that require the stream to be actively running.
Performs multiple actions while locking the stream from progressing.
Performs multiple actions while locking the stream from progressing. This is applicable only to MicroBatchExecution, as progress of ContinuousExecution cannot be controlled from the driver.
Adds the given data to the stream.
Adds the given data to the stream. Subsequent check answers will block until this data has been processed.
Checks to make sure that the current data stored in the sink matches the expectedAnswer
.
Checks to make sure that the current data stored in the sink matches the expectedAnswer
.
This operation automatically blocks until all added data has been processed.
Checks to make sure that the current data stored in the sink matches the expectedAnswer
.
Checks to make sure that the current data stored in the sink matches the expectedAnswer
.
This operation automatically blocks until all added data has been processed.
Execute arbitrary code
Adds data to multiple memory streams such that all the data will be made visible in the same batch.
Adds data to multiple memory streams such that all the data will be made visible in the same batch. This is applicable only to MicroBatchExecution, as this coordination cannot be performed at the driver in ContinuousExecutions.
Stops the stream.
Stops the stream. It must currently be running.
Asserts that a given Dataset
will be executed using the given number of cached results.
Asserts that a given Dataset
will be executed using the given number of cached results.
Asserts that a given Dataset
does not have missing inputs in all the analyzed plans.
Asserts that a given Dataset
does not have missing inputs in all the analyzed plans.
Runs the plan and makes sure the answer is within absTol of the expected result.
Runs the plan and makes sure the answer is within absTol of the expected result.
the DataFrame
to be executed
the expected result in a Seq
of Rows
.
the absolute tolerance between actual and expected answers.
Runs the plan and makes sure the answer matches the expected result.
Runs the plan and makes sure the answer matches the expected result.
the DataFrame
to be executed
the expected result in a Seq
of Rows
.
Evaluates a dataset to make sure that the result of calling collect matches the given expected answer.
Evaluates a dataset to make sure that the result of calling collect matches the given expected answer.
Evaluates a dataset to make sure that the result of calling collect matches the given expected answer, after sort.
Evaluates a dataset to make sure that the result of calling collect matches the given expected answer, after sort.
Runs the plan and makes sure the answer contains all of the keywords.
Runs the plan and makes sure the answer contains all of the keywords.
Runs the plan and makes sure the answer does NOT contain any of the keywords.
Runs the plan and makes sure the answer does NOT contain any of the keywords.
Fails the test if the two expressions do not match
Fails the test if the two expressions do not match
Fails the test if the join order in the two plans do not match
Fails the test if the join order in the two plans do not match
Fails the test if the two plans do not match
Fails the test if the two plans do not match
Since attribute references are given globally unique ids during analysis, we must normalize them to check if two different queries are identical.
Since attribute references are given globally unique ids during analysis, we must normalize them to check if two different queries are identical.
Normalizes plans: - Filter the filter conditions that appear in a plan.
Normalizes plans: - Filter the filter conditions that appear in a plan. For instance, ((expr 1 && expr 2) && expr 3), (expr 1 && expr 2 && expr 3), (expr 3 && (expr 1 && expr 2) etc., will all now be equivalent. - Sample the seed will replaced by 0L. - Join conditions will be resorted by hashCode.
Creates a stress test that randomly starts/stops/adds data/checks the result.
Creates a stress test that randomly starts/stops/adds data/checks the result.
a dataframe that executes + 1 on a stream of integers, returning the result
actions need to run before starting the stress test.
an add data action that adds the given numbers to the stream, encoding them as needed
the iteration number
Creates a stress test that randomly starts/stops/adds data/checks the result.
Creates a stress test that randomly starts/stops/adds data/checks the result.
a dataframe that executes + 1 on a stream of integers, returning the result
an add data action that adds the given numbers to the stream, encoding them as needed
the iteration number
How long to wait for an active stream to catch up when checking a result.
Executes the specified actions on the given streaming DataFrame and provides helpful error messages in the case of failures or incorrect answers.
Executes the specified actions on the given streaming DataFrame and provides helpful error messages in the case of failures or incorrect answers.
Note that if the stream is not explicitly started before an action that requires it to be running then it will be automatically started before performing any other actions.
Log the suite name and the test name before and after each test.
Log the suite name and the test name before and after each test.
Subclasses should never override this method. If they wish to run custom code before and after each test, they should mix in the {{org.scalatest.BeforeAndAfter}} trait instead.
Sets all SQL configurations specified in pairs
, calls f
, and then restores all SQL
configurations.
Sets all SQL configurations specified in pairs
, calls f
, and then restores all SQL
configurations.
The parameterless execute method has been deprecated and will be removed in a future version of ScalaTest. Please invoke execute with empty parens instead: execute().
The trap method is no longer needed for demos in the REPL, which now abreviates stack traces, and will be removed in a future version of ScalaTest
A framework for implementing tests for streaming queries and sources.
A test consists of a set of steps (expressed as a
StreamAction
) that are executed in order, blocking as necessary to let the stream catch up. For example, the following adds some data to a stream, blocking until it can verify that the correct values are eventually produced.Note that while we do sleep to allow the other thread to progress without spinning,
StreamAction
checks should not depend on the amount of time spent sleeping. Instead they should check the actual progress of the stream before verifying the required test condition.Currently it is assumed that all streaming queries will eventually complete in 10 seconds to avoid hanging forever in the case of failures. However, individual suites can change this by overriding
streamingTimeout
.