Trait

org.apache.spark.sql.streaming

StreamTest

Related Doc: package streaming

Permalink

trait StreamTest extends QueryTest with TimeLimits with BeforeAndAfterAll

A framework for implementing tests for streaming queries and sources.

A test consists of a set of steps (expressed as a StreamAction) that are executed in order, blocking as necessary to let the stream catch up. For example, the following adds some data to a stream, blocking until it can verify that the correct values are eventually produced.

val inputData = MemoryStream[Int]
val mapped = inputData.toDS().map(_ + 1)

testStream(mapped)(
  AddData(inputData, 1, 2, 3),
  CheckAnswer(2, 3, 4))

Note that while we do sleep to allow the other thread to progress without spinning, StreamAction checks should not depend on the amount of time spent sleeping. Instead they should check the actual progress of the stream before verifying the required test condition.

Currently it is assumed that all streaming queries will eventually complete in 10 seconds to avoid hanging forever in the case of failures. However, individual suites can change this by overriding streamingTimeout.

Linear Supertypes
TimeLimits, QueryTest, PlanTest, PlanTestBase, PredicateHelper, SparkFunSuite, Logging, BeforeAndAfterAll, SuiteMixin, FunSuite, FunSuiteLike, Documenting, Alerting, Notifying, Informing, TestRegistration, TestSuite, Suite, Serializable, Serializable, Assertions, TripleEquals, TripleEqualsSupport, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. StreamTest
  2. TimeLimits
  3. QueryTest
  4. PlanTest
  5. PlanTestBase
  6. PredicateHelper
  7. SparkFunSuite
  8. Logging
  9. BeforeAndAfterAll
  10. SuiteMixin
  11. FunSuite
  12. FunSuiteLike
  13. Documenting
  14. Alerting
  15. Notifying
  16. Informing
  17. TestRegistration
  18. TestSuite
  19. Suite
  20. Serializable
  21. Serializable
  22. Assertions
  23. TripleEquals
  24. TripleEqualsSupport
  25. AnyRef
  26. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. trait AddData extends StreamAction

    Permalink

    A trait that can be extended when testing a source.

  2. case class AddDataMemory[A](source: MemoryStreamBase[A], data: Seq[A]) extends AddData with Product with Serializable

    Permalink
  3. case class AdvanceManualClock(timeToAdd: Long) extends StreamAction with Product with Serializable

    Permalink

    Advance the trigger clock's time manually.

  4. class Assert extends StreamAction

    Permalink

    Assert that a body is true

  5. class AssertOnQuery extends StreamAction

    Permalink

    Assert that a condition on the active query is true

  6. class AssertionsHelper extends AnyRef

    Permalink
    Definition Classes
    Assertions
  7. case class CheckAnswerRows(expectedAnswer: Seq[Row], lastOnly: Boolean, isSorted: Boolean) extends StreamAction with StreamMustBeRunning with Product with Serializable

    Permalink
  8. case class CheckAnswerRowsByFunc(globalCheckFunction: (Seq[Row]) ⇒ Unit, lastOnly: Boolean) extends StreamAction with StreamMustBeRunning with Product with Serializable

    Permalink
  9. case class CheckAnswerRowsContains(expectedAnswer: Seq[Row], lastOnly: Boolean = false) extends StreamAction with StreamMustBeRunning with Product with Serializable

    Permalink
  10. case class CheckNewAnswerRows(expectedAnswer: Seq[Row]) extends StreamAction with StreamMustBeRunning with Product with Serializable

    Permalink
  11. class CheckingEqualizer[L] extends AnyRef

    Permalink
    Definition Classes
    TripleEqualsSupport
  12. class Equalizer[L] extends AnyRef

    Permalink
    Definition Classes
    TripleEqualsSupport
  13. case class ExpectFailure[T <: Throwable](assertFailure: (Throwable) ⇒ Unit = _ => {}, isFatalError: Boolean = false)(implicit evidence$5: ClassTag[T]) extends StreamAction with Product with Serializable

    Permalink

    Signals that a failure is expected and should not kill the test.

    Signals that a failure is expected and should not kill the test.

    assertFailure

    a function to verify the error.

    isFatalError

    if this is a fatal error. If so, the error should also be caught by UncaughtExceptionHandler.

  14. trait ExternalAction extends StreamAction

    Permalink

    A trait that can be extended when testing a source.

  15. trait NoArgTest extends () ⇒ Outcome with TestData

    Permalink
    Attributes
    protected
    Definition Classes
    TestSuite
  16. case class SetManualClock(time: Long) extends StreamAction with Product with Serializable

    Permalink
  17. case class StartStream(trigger: Trigger = defaultTrigger, triggerClock: Clock = new SystemClock, additionalConfs: Map[String, String] = Map.empty, checkpointLocation: String = null) extends StreamAction with Product with Serializable

    Permalink

    Starts the stream, resuming if data has already been processed.

    Starts the stream, resuming if data has already been processed. It must not be running.

  18. trait StreamAction extends AnyRef

    Permalink

    A trait for actions that can be performed while testing a streaming DataFrame.

  19. trait StreamMustBeRunning extends AnyRef

    Permalink

    A trait to mark actions that require the stream to be actively running.

  20. case class StreamProgressLockedActions(actions: Seq[StreamAction], desc: String = null) extends StreamAction with Product with Serializable

    Permalink

    Performs multiple actions while locking the stream from progressing.

    Performs multiple actions while locking the stream from progressing. This is applicable only to MicroBatchExecution, as progress of ContinuousExecution cannot be controlled from the driver.

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. def !==[T](right: Spread[T]): TripleEqualsInvocationOnSpread[T]

    Permalink
    Definition Classes
    TripleEqualsSupport
  3. def !==(right: Null): TripleEqualsInvocation[Null]

    Permalink
    Definition Classes
    TripleEqualsSupport
  4. def !==[T](right: T): TripleEqualsInvocation[T]

    Permalink
    Definition Classes
    TripleEqualsSupport
  5. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  6. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  7. def ===[T](right: Spread[T]): TripleEqualsInvocationOnSpread[T]

    Permalink
    Definition Classes
    TripleEqualsSupport
  8. def ===(right: Null): TripleEqualsInvocation[Null]

    Permalink
    Definition Classes
    TripleEqualsSupport
  9. def ===[T](right: T): TripleEqualsInvocation[T]

    Permalink
    Definition Classes
    TripleEqualsSupport
  10. object AddData

    Permalink

    Adds the given data to the stream.

    Adds the given data to the stream. Subsequent check answers will block until this data has been processed.

  11. object Assert

    Permalink
  12. object AssertOnQuery

    Permalink
  13. object AwaitEpoch

    Permalink
  14. object AwaitTerminationTester

    Permalink
  15. object CheckAnswer

    Permalink

    Checks to make sure that the current data stored in the sink matches the expectedAnswer.

    Checks to make sure that the current data stored in the sink matches the expectedAnswer. This operation automatically blocks until all added data has been processed.

  16. object CheckLastBatch

    Permalink

    Checks to make sure that the current data stored in the sink matches the expectedAnswer.

    Checks to make sure that the current data stored in the sink matches the expectedAnswer. This operation automatically blocks until all added data has been processed.

  17. object CheckNewAnswer

    Permalink
  18. object Execute

    Permalink

    Execute arbitrary code

  19. object IncrementEpoch

    Permalink
  20. object MultiAddData

    Permalink

    Adds data to multiple memory streams such that all the data will be made visible in the same batch.

    Adds data to multiple memory streams such that all the data will be made visible in the same batch. This is applicable only to MicroBatchExecution, as this coordination cannot be performed at the driver in ContinuousExecutions.

  21. object StopStream extends StreamAction with StreamMustBeRunning with Product with Serializable

    Permalink

    Stops the stream.

    Stops the stream. It must currently be running.

  22. def afterAll(): Unit

    Permalink
    Definition Classes
    StreamTestSparkFunSuite → BeforeAndAfterAll
  23. def alert: Alerter

    Permalink
    Attributes
    protected
    Definition Classes
    FunSuiteLike → Alerting
  24. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  25. macro def assert(condition: Boolean, clue: Any)(implicit prettifier: Prettifier, pos: Position): Assertion

    Permalink
    Definition Classes
    Assertions
  26. macro def assert(condition: Boolean)(implicit prettifier: Prettifier, pos: Position): Assertion

    Permalink
    Definition Classes
    Assertions
  27. def assertCached(query: Dataset[_], numCachedTables: Int = 1): Unit

    Permalink

    Asserts that a given Dataset will be executed using the given number of cached results.

    Asserts that a given Dataset will be executed using the given number of cached results.

    Definition Classes
    QueryTest
  28. macro def assertCompiles(code: String)(implicit pos: Position): Assertion

    Permalink
    Definition Classes
    Assertions
  29. macro def assertDoesNotCompile(code: String)(implicit pos: Position): Assertion

    Permalink
    Definition Classes
    Assertions
  30. def assertEmptyMissingInput(query: Dataset[_]): Unit

    Permalink

    Asserts that a given Dataset does not have missing inputs in all the analyzed plans.

    Asserts that a given Dataset does not have missing inputs in all the analyzed plans.

    Definition Classes
    QueryTest
  31. def assertResult(expected: Any)(actual: Any)(implicit prettifier: Prettifier, pos: Position): Assertion

    Permalink
    Definition Classes
    Assertions
  32. def assertResult(expected: Any, clue: Any)(actual: Any)(implicit prettifier: Prettifier, pos: Position): Assertion

    Permalink
    Definition Classes
    Assertions
  33. def assertThrows[T <: AnyRef](f: ⇒ Any)(implicit classTag: ClassTag[T], pos: Position): Assertion

    Permalink
    Definition Classes
    Assertions
  34. macro def assertTypeError(code: String)(implicit pos: Position): Assertion

    Permalink
    Definition Classes
    Assertions
  35. val assertionsHelper: AssertionsHelper

    Permalink
    Definition Classes
    Assertions
  36. macro def assume(condition: Boolean, clue: Any)(implicit prettifier: Prettifier, pos: Position): Assertion

    Permalink
    Definition Classes
    Assertions
  37. macro def assume(condition: Boolean)(implicit prettifier: Prettifier, pos: Position): Assertion

    Permalink
    Definition Classes
    Assertions
  38. def beforeAll(): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    BeforeAndAfterAll
  39. def canEvaluate(expr: Expression, plan: LogicalPlan): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    PredicateHelper
  40. def canEvaluateWithinJoin(expr: Expression): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    PredicateHelper
  41. def cancel(cause: Throwable)(implicit pos: Position): Nothing

    Permalink
    Definition Classes
    Assertions
  42. def cancel(message: String, cause: Throwable)(implicit pos: Position): Nothing

    Permalink
    Definition Classes
    Assertions
  43. def cancel(message: String)(implicit pos: Position): Nothing

    Permalink
    Definition Classes
    Assertions
  44. def cancel()(implicit pos: Position): Nothing

    Permalink
    Definition Classes
    Assertions
  45. def cancelAfter[T](timeout: Span)(fun: ⇒ T)(implicit signaler: Signaler, prettifier: Prettifier, pos: Position, timed: Timed[T]): T

    Permalink
    Definition Classes
    TimeLimits
  46. def checkAggregatesWithTol(dataFrame: DataFrame, expectedAnswer: Row, absTol: Double): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    QueryTest
  47. def checkAggregatesWithTol(dataFrame: DataFrame, expectedAnswer: Seq[Row], absTol: Double): Unit

    Permalink

    Runs the plan and makes sure the answer is within absTol of the expected result.

    Runs the plan and makes sure the answer is within absTol of the expected result.

    dataFrame

    the DataFrame to be executed

    expectedAnswer

    the expected result in a Seq of Rows.

    absTol

    the absolute tolerance between actual and expected answers.

    Attributes
    protected
    Definition Classes
    QueryTest
  48. def checkAnswer(df: ⇒ DataFrame, expectedAnswer: DataFrame): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    QueryTest
  49. def checkAnswer(df: ⇒ DataFrame, expectedAnswer: Row): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    QueryTest
  50. def checkAnswer(df: ⇒ DataFrame, expectedAnswer: Seq[Row]): Unit

    Permalink

    Runs the plan and makes sure the answer matches the expected result.

    Runs the plan and makes sure the answer matches the expected result.

    df

    the DataFrame to be executed

    expectedAnswer

    the expected result in a Seq of Rows.

    Attributes
    protected
    Definition Classes
    QueryTest
  51. def checkDataset[T](ds: ⇒ Dataset[T], expectedAnswer: T*): Unit

    Permalink

    Evaluates a dataset to make sure that the result of calling collect matches the given expected answer.

    Evaluates a dataset to make sure that the result of calling collect matches the given expected answer.

    Attributes
    protected
    Definition Classes
    QueryTest
  52. def checkDatasetUnorderly[T](ds: ⇒ Dataset[T], expectedAnswer: T*)(implicit arg0: Ordering[T]): Unit

    Permalink

    Evaluates a dataset to make sure that the result of calling collect matches the given expected answer, after sort.

    Evaluates a dataset to make sure that the result of calling collect matches the given expected answer, after sort.

    Attributes
    protected
    Definition Classes
    QueryTest
  53. def checkKeywordsExist(df: DataFrame, keywords: String*): Unit

    Permalink

    Runs the plan and makes sure the answer contains all of the keywords.

    Runs the plan and makes sure the answer contains all of the keywords.

    Definition Classes
    QueryTest
  54. def checkKeywordsNotExist(df: DataFrame, keywords: String*): Unit

    Permalink

    Runs the plan and makes sure the answer does NOT contain any of the keywords.

    Runs the plan and makes sure the answer does NOT contain any of the keywords.

    Definition Classes
    QueryTest
  55. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  56. def compareExpressions(e1: Expression, e2: Expression): Unit

    Permalink

    Fails the test if the two expressions do not match

    Fails the test if the two expressions do not match

    Attributes
    protected
    Definition Classes
    PlanTestBase
  57. def compareJoinOrder(plan1: LogicalPlan, plan2: LogicalPlan): Unit

    Permalink

    Fails the test if the join order in the two plans do not match

    Fails the test if the join order in the two plans do not match

    Attributes
    protected
    Definition Classes
    PlanTestBase
  58. def comparePlans(plan1: LogicalPlan, plan2: LogicalPlan, checkAnalysis: Boolean = true): Unit

    Permalink

    Fails the test if the two plans do not match

    Fails the test if the two plans do not match

    Attributes
    protected
    Definition Classes
    PlanTestBase
  59. def conversionCheckedConstraint[A, B](implicit equivalenceOfA: Equivalence[A], cnv: (B) ⇒ A): CanEqual[A, B]

    Permalink
    Definition Classes
    TripleEquals → TripleEqualsSupport
  60. def convertEquivalenceToAToBConstraint[A, B](equivalenceOfB: Equivalence[B])(implicit ev: <:<[A, B]): CanEqual[A, B]

    Permalink
    Definition Classes
    TripleEquals → TripleEqualsSupport
  61. def convertEquivalenceToAToBConversionConstraint[A, B](equivalenceOfB: Equivalence[B])(implicit ev: (A) ⇒ B): CanEqual[A, B]

    Permalink
    Definition Classes
    TripleEquals → TripleEqualsSupport
  62. def convertEquivalenceToBToAConstraint[A, B](equivalenceOfA: Equivalence[A])(implicit ev: <:<[B, A]): CanEqual[A, B]

    Permalink
    Definition Classes
    TripleEquals → TripleEqualsSupport
  63. def convertEquivalenceToBToAConversionConstraint[A, B](equivalenceOfA: Equivalence[A])(implicit ev: (B) ⇒ A): CanEqual[A, B]

    Permalink
    Definition Classes
    TripleEquals → TripleEqualsSupport
  64. def convertToCheckingEqualizer[T](left: T): CheckingEqualizer[T]

    Permalink
    Definition Classes
    TripleEquals → TripleEqualsSupport
  65. implicit def convertToEqualizer[T](left: T): Equalizer[T]

    Permalink
    Definition Classes
    TripleEquals → TripleEqualsSupport
  66. def defaultEquality[A]: Equality[A]

    Permalink
    Definition Classes
    TripleEqualsSupport
  67. implicit val defaultSignaler: Signaler

    Permalink
  68. val defaultTrigger: Trigger

    Permalink
    Attributes
    protected
  69. val defaultUseV2Sink: Boolean

    Permalink
    Attributes
    protected
  70. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  71. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  72. final def execute(testName: String, configMap: ConfigMap, color: Boolean, durations: Boolean, shortstacks: Boolean, fullstacks: Boolean, stats: Boolean): Unit

    Permalink
    Definition Classes
    Suite
  73. def expectedTestCount(filter: Filter): Int

    Permalink
    Definition Classes
    Suite
  74. def fail(cause: Throwable)(implicit pos: Position): Nothing

    Permalink
    Definition Classes
    Assertions
  75. def fail(message: String, cause: Throwable)(implicit pos: Position): Nothing

    Permalink
    Definition Classes
    Assertions
  76. def fail(message: String)(implicit pos: Position): Nothing

    Permalink
    Definition Classes
    Assertions
  77. def fail()(implicit pos: Position): Nothing

    Permalink
    Definition Classes
    Assertions
  78. def failAfter[T](timeout: Span)(fun: ⇒ T)(implicit signaler: Signaler, prettifier: Prettifier, pos: Position, timed: Timed[T]): T

    Permalink
    Definition Classes
    TimeLimits
  79. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  80. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  81. final def getTestResourceFile(file: String): File

    Permalink
    Attributes
    protected
    Definition Classes
    SparkFunSuite
  82. final def getTestResourcePath(file: String): String

    Permalink
    Attributes
    protected
    Definition Classes
    SparkFunSuite
  83. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  84. def ignore(testName: String, testTags: Tag*)(testFun: ⇒ Any)(implicit pos: Position): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    FunSuiteLike
  85. def info: Informer

    Permalink
    Attributes
    protected
    Definition Classes
    FunSuiteLike → Informing
  86. def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  87. def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  88. def intercept[T <: AnyRef](f: ⇒ Any)(implicit classTag: ClassTag[T], pos: Position): T

    Permalink
    Definition Classes
    Assertions
  89. val invokeBeforeAllAndAfterAllEvenIfNoTestsAreExpected: Boolean

    Permalink
    Definition Classes
    BeforeAndAfterAll
  90. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  91. def isTraceEnabled(): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  92. def log: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  93. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  94. def logDebug(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  95. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  96. def logError(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  97. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  98. def logInfo(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  99. def logName: String

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  100. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  101. def logTrace(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  102. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  103. def logWarning(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  104. def lowPriorityConversionCheckedConstraint[A, B](implicit equivalenceOfB: Equivalence[B], cnv: (A) ⇒ B): CanEqual[A, B]

    Permalink
    Definition Classes
    TripleEquals → TripleEqualsSupport
  105. def lowPriorityTypeCheckedConstraint[A, B](implicit equivalenceOfB: Equivalence[B], ev: <:<[A, B]): CanEqual[A, B]

    Permalink
    Definition Classes
    TripleEquals → TripleEqualsSupport
  106. def markup: Documenter

    Permalink
    Attributes
    protected
    Definition Classes
    FunSuiteLike → Documenting
  107. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  108. def nestedSuites: IndexedSeq[Suite]

    Permalink
    Definition Classes
    Suite
  109. def normalizeExprIds(plan: LogicalPlan): LogicalPlan

    Permalink

    Since attribute references are given globally unique ids during analysis, we must normalize them to check if two different queries are identical.

    Since attribute references are given globally unique ids during analysis, we must normalize them to check if two different queries are identical.

    Attributes
    protected
    Definition Classes
    PlanTestBase
  110. def normalizePlan(plan: LogicalPlan): LogicalPlan

    Permalink

    Normalizes plans: - Filter the filter conditions that appear in a plan.

    Normalizes plans: - Filter the filter conditions that appear in a plan. For instance, ((expr 1 && expr 2) && expr 3), (expr 1 && expr 2 && expr 3), (expr 3 && (expr 1 && expr 2) etc., will all now be equivalent. - Sample the seed will replaced by 0L. - Join conditions will be resorted by hashCode.

    Attributes
    protected
    Definition Classes
    PlanTestBase
  111. def note: Notifier

    Permalink
    Attributes
    protected
    Definition Classes
    FunSuiteLike → Notifying
  112. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  113. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  114. def pending: Assertion with PendingStatement

    Permalink
    Definition Classes
    Assertions
  115. def pendingUntilFixed(f: ⇒ Unit)(implicit pos: Position): Assertion with PendingStatement

    Permalink
    Definition Classes
    Assertions
  116. final def registerIgnoredTest(testText: String, testTags: Tag*)(testFun: ⇒ Any)(implicit pos: Position): Unit

    Permalink
    Definition Classes
    FunSuiteLike → TestRegistration
  117. final def registerTest(testText: String, testTags: Tag*)(testFun: ⇒ Any)(implicit pos: Position): Unit

    Permalink
    Definition Classes
    FunSuiteLike → TestRegistration
  118. def replaceAlias(condition: Expression, aliases: AttributeMap[Expression]): Expression

    Permalink
    Attributes
    protected
    Definition Classes
    PredicateHelper
  119. def rerunner: Option[String]

    Permalink
    Definition Classes
    Suite
  120. def run(testName: Option[String], args: Args): Status

    Permalink
    Definition Classes
    BeforeAndAfterAll → SuiteMixin
  121. def runNestedSuites(args: Args): Status

    Permalink
    Attributes
    protected
    Definition Classes
    Suite
  122. def runStressTest(ds: Dataset[Int], prepareActions: Seq[StreamAction], addData: (Seq[Int], Boolean) ⇒ StreamAction, iterations: Int): Unit

    Permalink

    Creates a stress test that randomly starts/stops/adds data/checks the result.

    Creates a stress test that randomly starts/stops/adds data/checks the result.

    ds

    a dataframe that executes + 1 on a stream of integers, returning the result

    prepareActions

    actions need to run before starting the stress test.

    addData

    an add data action that adds the given numbers to the stream, encoding them as needed

    iterations

    the iteration number

  123. def runStressTest(ds: Dataset[Int], addData: (Seq[Int]) ⇒ StreamAction, iterations: Int = 100): Unit

    Permalink

    Creates a stress test that randomly starts/stops/adds data/checks the result.

    Creates a stress test that randomly starts/stops/adds data/checks the result.

    ds

    a dataframe that executes + 1 on a stream of integers, returning the result

    addData

    an add data action that adds the given numbers to the stream, encoding them as needed

    iterations

    the iteration number

  124. def runTest(testName: String, args: Args): Status

    Permalink
    Attributes
    protected
    Definition Classes
    FunSuiteLike → TestSuite → Suite
  125. def runTests(testName: Option[String], args: Args): Status

    Permalink
    Attributes
    protected
    Definition Classes
    FunSuiteLike → Suite
  126. lazy val spark: SparkSession

    Permalink
    Definition Classes
    SparkFunSuite
  127. def splitConjunctivePredicates(condition: Expression): Seq[Expression]

    Permalink
    Attributes
    protected
    Definition Classes
    PredicateHelper
  128. def splitDisjunctivePredicates(condition: Expression): Seq[Expression]

    Permalink
    Attributes
    protected
    Definition Classes
    PredicateHelper
  129. val streamingTimeout: Span

    Permalink

    How long to wait for an active stream to catch up when checking a result.

  130. final val styleName: String

    Permalink
    Definition Classes
    FunSuiteLike → Suite
  131. final val succeed: Assertion

    Permalink
    Definition Classes
    Assertions
  132. def suiteId: String

    Permalink
    Definition Classes
    Suite
  133. def suiteName: String

    Permalink
    Definition Classes
    Suite
  134. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  135. def tags: Map[String, Set[String]]

    Permalink
    Definition Classes
    FunSuiteLike → Suite
  136. def test(testName: String, testTags: Tag*)(testFun: ⇒ Any)(implicit pos: Position): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    FunSuiteLike
  137. def testDataFor(testName: String, theConfigMap: ConfigMap): TestData

    Permalink
    Definition Classes
    FunSuiteLike → Suite
  138. def testNames: Set[String]

    Permalink
    Definition Classes
    FunSuiteLike → Suite
  139. def testStream(_stream: Dataset[_], outputMode: OutputMode = OutputMode.Append, useV2Sink: Boolean = defaultUseV2Sink)(actions: StreamAction*): Unit

    Permalink

    Executes the specified actions on the given streaming DataFrame and provides helpful error messages in the case of failures or incorrect answers.

    Executes the specified actions on the given streaming DataFrame and provides helpful error messages in the case of failures or incorrect answers.

    Note that if the stream is not explicitly started before an action that requires it to be running then it will be automatically started before performing any other actions.

  140. def testsFor(unit: Unit): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    FunSuiteLike
  141. def toString(): String

    Permalink
    Definition Classes
    FunSuite → AnyRef → Any
  142. def typeCheckedConstraint[A, B](implicit equivalenceOfA: Equivalence[A], ev: <:<[B, A]): CanEqual[A, B]

    Permalink
    Definition Classes
    TripleEquals → TripleEqualsSupport
  143. implicit def unconstrainedEquality[A, B](implicit equalityOfA: Equality[A]): CanEqual[A, B]

    Permalink
    Definition Classes
    TripleEquals → TripleEqualsSupport
  144. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  145. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  146. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  147. def withClue[T](clue: Any)(fun: ⇒ T): T

    Permalink
    Definition Classes
    Assertions
  148. final def withFixture(test: NoArgTest): Outcome

    Permalink

    Log the suite name and the test name before and after each test.

    Log the suite name and the test name before and after each test.

    Subclasses should never override this method. If they wish to run custom code before and after each test, they should mix in the {{org.scalatest.BeforeAndAfter}} trait instead.

    Attributes
    protected
    Definition Classes
    SparkFunSuite → TestSuite
  149. def withSQLConf(pairs: (String, String)*)(f: ⇒ Unit): Unit

    Permalink

    Sets all SQL configurations specified in pairs, calls f, and then restores all SQL configurations.

    Sets all SQL configurations specified in pairs, calls f, and then restores all SQL configurations.

    Attributes
    protected
    Definition Classes
    PlanTestBase

Deprecated Value Members

  1. final def execute: Unit

    Permalink
    Definition Classes
    Suite
    Annotations
    @deprecated
    Deprecated

    The parameterless execute method has been deprecated and will be removed in a future version of ScalaTest. Please invoke execute with empty parens instead: execute().

  2. def trap[T](f: ⇒ T): Throwable

    Permalink
    Definition Classes
    Assertions
    Annotations
    @deprecated
    Deprecated

    The trap method is no longer needed for demos in the REPL, which now abreviates stack traces, and will be removed in a future version of ScalaTest

Inherited from TimeLimits

Inherited from QueryTest

Inherited from PlanTest

Inherited from PlanTestBase

Inherited from PredicateHelper

Inherited from SparkFunSuite

Inherited from Logging

Inherited from BeforeAndAfterAll

Inherited from SuiteMixin

Inherited from FunSuite

Inherited from FunSuiteLike

Inherited from Documenting

Inherited from Alerting

Inherited from Notifying

Inherited from Informing

Inherited from TestRegistration

Inherited from TestSuite

Inherited from Suite

Inherited from Serializable

Inherited from Serializable

Inherited from Assertions

Inherited from TripleEquals

Inherited from TripleEqualsSupport

Inherited from AnyRef

Inherited from Any

Ungrouped