Packages

  • package root
    Definition Classes
    root
  • package org
    Definition Classes
    root
  • package scalatest

    ScalaTest's main traits, classes, and other members, including members supporting ScalaTest's DSL for the Scala interpreter.

    ScalaTest's main traits, classes, and other members, including members supporting ScalaTest's DSL for the Scala interpreter.

    Definition Classes
    org
  • package compatible
    Definition Classes
    scalatest
  • package concurrent

    ScalaTest's main traits, classes, and other members, including members supporting ScalaTest's DSL for the Scala interpreter.

    ScalaTest's main traits, classes, and other members, including members supporting ScalaTest's DSL for the Scala interpreter.

    Definition Classes
    scalatest
  • package enablers
    Definition Classes
    scalatest
  • package events
    Definition Classes
    scalatest
  • AlertProvided
  • DiscoveryCompleted
  • DiscoveryStarting
  • Event
  • ExceptionalEvent
  • Formatter
  • IndentedText
  • InfoProvided
  • LineInFile
  • Location
  • MarkupProvided
  • MotionToSuppress
  • NameInfo
  • NoteProvided
  • NotificationEvent
  • Ordinal
  • RecordableEvent
  • RunAborted
  • RunCompleted
  • RunStarting
  • RunStopped
  • ScopeClosed
  • ScopeOpened
  • ScopePending
  • SeeStackDepthException
  • SuiteAborted
  • SuiteCompleted
  • SuiteStarting
  • Summary
  • TestCanceled
  • TestFailed
  • TestIgnored
  • TestPending
  • TestStarting
  • TestSucceeded
  • TopOfClass
  • TopOfMethod
  • package exceptions
    Definition Classes
    scalatest
  • package featurespec
    Definition Classes
    scalatest
  • package fixture

    Package fixture deprecated types.

    Package fixture deprecated types.

    Definition Classes
    scalatest
  • package flatspec
    Definition Classes
    scalatest
  • package freespec
    Definition Classes
    scalatest
  • package funspec
    Definition Classes
    scalatest
  • package funsuite
    Definition Classes
    scalatest
  • package matchers
    Definition Classes
    scalatest
  • package path
    Definition Classes
    scalatest
  • package prop

    Scalatest support for Property-based testing.

    Scalatest support for Property-based testing.

    Introduction to Property-based Testing

    In traditional unit testing, you write tests that describe precisely what the test will do: create these objects, wire them together, call these functions, assert on the results, and so on. It is clear and deterministic, but also limited, because it only covers the exact situations you think to test. In most cases, it is not feasible to test all of the possible combinations of data that might arise in real-world use.

    Property-based testing works the other way around. You describe properties -- rules that you expect your classes to live by -- and describe how to test those properties. The test system then generates relatively large amounts of synthetic data (with an emphasis on edge cases that tend to make things break), so that you can see if the properties hold true in these situations.

    As a result, property-based testing is scientific in the purest sense: you are stating a hypothesis about how things should work (the property), and the system is trying to falsify that hypothesis. If the tests pass, that doesn't prove the property holds, but it at least gives you some confidence that you are probably correct.

    Property-based testing is deliberately a bit random: while the edge cases get tried upfront, the system also usually generates a number of random values to try out. This makes things a bit non-deterministic -- each run will be tried with somewhat different data. To make it easier to debug, and to build regression tests, the system provides tools to re-run a failed test with precisely the same data.

    Background

    TODO: Bill should insert a brief section on QuickCheck, ScalaCheck, etc, and how this system is similar and different.

    Using Property Checks

    In order to use the tools described here, you should import this package:

    import org.scalatest._
import org.scalatest.prop._

    This library is designed to work well with the types defined in Scalactic, and some functions take types such as PosZInt as parameters. So it can also be helpful to import those with:

    import org.scalactic.anyvals._

    In order to call forAll, the function that actually performs property checks, you will need to either extend or import GeneratorDrivenPropertyChecks, like this:

    class DocExamples extends FlatSpec with Matchers with GeneratorDrivenPropertyChecks {

    There's nothing special about FlatSpec, though -- you may use any of ScalaTest's styles with property checks. GeneratorDrivenPropertyChecks extends CommonGenerators, so it also provides access to the many utilities found there.

    What Does a Property Look Like?

    Let's check a simple property of Strings -- that if you concatenate a String to itself, its length will be doubled:

    "Strings" should "have the correct length when doubled" in {
  forAll { (s: String) =>
    val s2 = s * 2
    s2.length should equal (s.length * 2)
  }
}

    (Note that the examples here are all using the FlatSpec style, but will work the same way with any of ScalaTest's styles.)

    As the name of the tests suggests, the property we are testing is the length of a String that has been doubled.

    The test begins with forAll. This is usually the way you'll want to begin property checks, and that line can be read as, "For all Strings, the following should be true".

    The test harness will generate a number of Strings, with various contents and lengths. For each one, we compute s * 2. (* is a function on String, which appends the String to itself as many times as you specify.) And then we check that the length of the doubled String is twice the length of the original one.

    Using Specific Generators

    Let's try a more general version of this test, multiplying arbitrary Strings by arbitrary multipliers:

    "Strings" should "have the correct length when multiplied" in {
  forAll { (s: String, n: PosZInt) =>
    val s2 = s * n.value
    s2.length should equal (s.length * n.value)
  }
}

    Again, you can read the first line of the test as "For all Strings, and all non-negative Integers, the following should be true". (PosZInt is a type defined in Scalactic, which can be any positive integer, including zero. It is appropriate to use here, since multiplying a String by a negative number doesn't make sense.)

    This intuitively makes sense, but when we try to run it, we get a JVM Out of Memory error! Why? Because the test system tries to test with the "edge cases" first, and one of the more important edge cases is Int.MaxValue. It is trying to multiply a String by that, which is far larger than the memory of even a big computer, and crashing.

    So we want to constrain our test to sane values of n, so that it doesn't crash. We can do this by using more specific Generators.

    When we write a forAll test like the above, ScalaTest has to generate the values to be tested -- the semi-random Strings, Ints and other types that you are testing. It does this by calling on an implicit Generator for the desired type. The Generator generates values to test, starting with the edge cases and then moving on to randomly-selected values.

    ScalaTest has built-in Generators for many major types, including String and PosZInt, but these Generators are generic: they will try any value, including values that can break your test, as shown above. But it also provides tools to let you be more specific.

    Here is the fixed version of the above test:

    "Strings" should "have the correct length when multiplied" in {
  forAll(strings, posZIntsBetween(0, 1000))
  { (s: String, n: PosZInt) =>
    val s2 = s * n.value
    s2.length should equal (s.length * n.value)
  }
}

    This is using a variant of forAll, which lets you specify the Generators to use instead of just picking the implicit one. CommonGenerators.strings is the built-in Generator for Strings, the same one you were getting implicitly. (The other built-ins can be found in CommonGenerators. They are mixed into GeneratorDrivenPropertyChecks, so they are readily available.)

    But CommonGenerators.posZIntsBetween is a function that creates a Generator that selects from the given values. In this case, it will create a Generator that only creates numbers from 0 to 1000 -- small enough to not blow up our computer's memory. If you try this test, this runs correctly.

    The moral of the story is that, while using the built-in Generators is very convenient, and works most of the time, you should think about the data you are trying to test, and pick or create a more-specific Generator when the test calls for it.

    CommonGenerators contains many functions that are helpful in common cases. In particular:

    • xxsBetween (where xxs might be Int, Long, Float or most other significant numeric types) gives you a value of the desired type in the given range, as in the posZIntsBetween() example above.
    • CommonGenerators.specificValue and CommonGenerators.specificValues create Generators that produce either one specific value every time, or one of several values randomly. This is useful for enumerations and types that behave like enumerations.
    • CommonGenerators.evenly and CommonGenerators.frequency create higher-level Generators that call other Generators, either more or less equally or with a distribution you define.

    Testing Your Own Types

    Testing the built-in types isn't very interesting, though. Usually, you have your own types that you want to check the properties of. So let's build up an example piece by piece.

    Say you have this simple type:

    sealed trait Shape {
  def area: Double
}
case class Rectangle(width: Int, height: Int) extends Shape {
  require(width > 0)
  require(height > 0)
  def area: Double = width * height
}

    Let's confirm a nice straightforward property that is surely true: that the area is greater than zero:

    "Rectangles" should "have a positive area" in {
   forAll { (w: PosInt, h: PosInt) =>
     val rect = Rectangle(w, h)
     rect.area should be > 0.0
   }
 }

    Note that, even though our class takes ordinary Ints as parameters (and checks the values at runtime), it is actually easier to generate the legal values using Scalactic's PosInt type.

    This should work, right? Actually, it doesn't -- if we run it a few times, we quickly hit an error!

    [info] Rectangles
[info] - should have a positive area *** FAILED ***
[info]   GeneratorDrivenPropertyCheckFailedException was thrown during property evaluation.
[info]    (DocExamples.scala:42)
[info]     Falsified after 2 successful property evaluations.
[info]     Location: (DocExamples.scala:42)
[info]     Occurred when passed generated values (
[info]       None = PosInt(399455539),
[info]       None = PosInt(703518968)
[info]     )
[info]     Init Seed: 1568878346200

    TODO: fix the above error to reflect the better errors we should get when we merge in the code being forward-ported from 3.0.5.

    Looking at it, we can see that the numbers being used are pretty large. What happens when we multiply them together?

    scala> 399455539 * 703518968
res0: Int = -2046258840

    We're hitting an Int overflow problem here: the numbers are too big to multiply together and still get an Int. So we have to fix our area function:

    case class Rectangle(width: Int, height: Int) extends Shape {
  require(width > 0)
  require(height > 0)
  def area: Double = width.toLong * height.toLong
}

    Now, when we run our property check, it consistently passes. Excellent -- we've caught a bug, because ScalaTest tried sufficiently large numbers.

    Composing Your Own Generators

    Doing things as shown above works, but having to generate the parameters and construct a Rectangle every time is a nuisance. What we really want is to create our own Generator that just hands us Rectangles, the same way we can do for PosInt. Fortunately, this is easy.

    Generators can be composed in for comprehensions. So we can create our own Generator for Rectangle like this:

    implicit val rectGenerator = for {
  w <- posInts
  h <- posInts
}
  yield Rectangle(w, h)

    Taking that line by line:

    w <- posInts

    CommonGenerators.posInts is the built-in Generator for positive Ints. So this line puts a randomly-generated positive Int in w, and

    h <- posInts

    this line puts another one in h. Finally, this line:

    yield Rectangle(w, h)

    combines w and h to make a Rectangle.

    That's pretty much all you need in order to build any normal case class -- just build it out of the Generators for the type of each field. (And if the fields are complex data structures themselves, build Generators for them the same way, until you are just using primitives.)

    Now, our property check becomes simpler:

    "Generated Rectangles" should "have a positive area" in {
   forAll { (rect: Rectangle) =>
     rect.area should be > 0.0
   }
 }

    That's about as close to plain English as we can reasonably hope for!

    Filtering Values with whenever()

    Sometimes, not all of your generated values make sense for the property you want to check -- you know (via external information) that some of these values will never come up. In cases like this, you can create a custom Generator that only creates the values you do want, but it's often easier to just use Whenever.whenever. (Whenever is mixed into GeneratorDrivenPropertyChecks, so this is available when you need it.)

    The Whenever.whenever function can be used inside of GeneratorDrivenPropertyChecks.forAll. It says that only the filtered values should be used, and anything else should be discarded. For example, look at this property:

    "Fractions" should "get smaller when squared" in {
  forAll { (n: Float) =>
    whenever(n > 0 && n < 1) {
      (n * n) should be < n
    }
  }
}

    We are testing a property of numbers less than 1, so we filter away everything that is not the numbers we want. This property check succeeds, because we've screened out the values that would make it fail.

    Discard Limits

    You shouldn't push Whenever.whenever too far, though. This system is all about trying random data, but if too much of the random data simply isn't usable, you can't get valid answers, and the system tracks that.

    For example, consider this apparently-reasonable test:

    "Space Chars" should "not also be letters" in {
  forAll { (c: Char) =>
    whenever (c.isSpaceChar) {
      assert(!c.isLetter)
    }
  }
}

    Although the property is true, this test will fail with an error like this:

    [info] Lowercase Chars
[info] - should upper-case correctly *** FAILED ***
[info]   Gave up after 0 successful property evaluations. 49 evaluations were discarded.
[info]   Init Seed: 1568855247784

    Because the vast majority of Chars are not spaces, nearly all of the generated values are being discarded. As a result, the system gives up after a while. In cases like this, you usually should write a custom Generator instead.

    The proportion of how many discards to permit, relative to the number of successful checks, is configuration-controllable. See GeneratorDrivenPropertyChecks for more details.

    Randomization

    The point of Generator is to create pseudo-random values for checking properties. But it turns out to be very inconvenient if those values are actually random -- that would mean that, when a property check fails occasionally, you have no good way to invoke that specific set of circumstances again for debugging. We want "randomness", but we also want it to be deterministic, and reproducible when you need it.

    To support this, all "randomness" in ScalaTest's property checking system uses the Randomizer class. You start by creating a Randomizer using an initial seed value, and call that to get your "random" value. Each call to a Randomizer function returns a new Randomizer, which you should use to fetch the next value.

    GeneratorDrivenPropertyChecks.forAll uses Randomizer under the hood: each time you run a forAll-based test, it will automatically create a new Randomizer, which by default is seeded based on the current system time. You can override this, as discussed below.

    Since Randomizer is actually deterministic (the "random" values are unobvious, but will always be the same given the same initial seed), this means that re-running a test with the same seed will produce the same values.

    If you need random data for your own Generators and property checks, you should use Randomizer in the same way; that way, your tests will also be re-runnable, when needed for debugging.

    Debugging, and Re-running a Failed Property Check

    In Testing Your Own Types above, we found to our surprise that the property check failed with this error:

    [info] Rectangles
[info] - should have a positive area *** FAILED ***
[info]   GeneratorDrivenPropertyCheckFailedException was thrown during property evaluation.
[info]    (DocExamples.scala:42)
[info]     Falsified after 2 successful property evaluations.
[info]     Location: (DocExamples.scala:42)
[info]     Occurred when passed generated values (
[info]       None = PosInt(399455539),
[info]       None = PosInt(703518968)
[info]     )
[info]     Init Seed: 1568878346200

    There must be a bug here -- but once we've fixed it, how can we make sure that we are re-testing exactly the same case that failed?

    This is where the pseudo-random nature of Randomizer comes in, and why it is so important to use it consistently. So long as all of our "random" data comes from that, then all we need to do is re-run with the same seed.

    That's why the Init Seed shown in the message above is crucial. We can re-use that seed -- and therefore get exactly the same "random" data -- by using the -S flag to ScalaTest.

    So you can run this command in sbt to re-run exactly the same property check:

    testOnly *DocExamples -- -z "have a positive area" -S 1568878346200

    Taking that apart:

    • testOnly *DocExamples says that we only want to run suites whose paths end with DocExamples
    • -z "have a positive area" says to only run tests whose names include that string.
    • -S 1568878346200 says to run all tests with a "random" seed of 1568878346200

    By combining these flags, you can re-run exactly the property check you need, with the right random seed to make sure you are re-creating the failed test. You should get exactly the same failure over and over until you fix the bug, and then you can confirm your fix with confidence.

    Configuration

    In general, forAll() works well out of the box. But you can tune several configuration parameters when needed. See GeneratorDrivenPropertyChecks for info on how to set configuration parameters for your test.

    Table-Driven Properties

    Sometimes, you want something in between traditional hard-coded unit tests and Generator-driven, randomized tests. Instead, you sometimes want to check your properties against a specific set of inputs.

    (This is particularly useful for regression tests, when you have found certain inputs that have caused problems in the past, and want to make sure that they get consistently re-tested.)

    ScalaTest supports these, by mixing in TableDrivenPropertyChecks. See the documentation for that class for the full details.

    Definition Classes
    scalatest
  • package propspec
    Definition Classes
    scalatest
  • package tagobjects
    Definition Classes
    scalatest
  • package time
    Definition Classes
    scalatest
  • package tools
    Definition Classes
    scalatest
  • package words
    Definition Classes
    scalatest
  • package wordspec
    Definition Classes
    scalatest

package events

Ordering
  1. Alphabetic
Visibility
  1. Public
  2. All

Type Members

  1. final case class AlertProvided(ordinal: Ordinal, message: String, nameInfo: Option[NameInfo], throwable: Option[Throwable] = None, formatter: Option[Formatter] = None, location: Option[Location] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with NotificationEvent with Product with Serializable

    Event used to provide alert notifications.

    Event used to provide alert notifications.

    To create instances of this class you may use the factory method. For example, given a report function named report, you could fire an AlertProvided event like this:

    report(AlertProvided(ordinal, message, Some(NameInfo(suiteName, suiteId, Some(thisSuite.getClass.getName), Some(testName)))))

    AlertProvided differs from InfoProvided in that unlike InfoProvided, AlertProvided isn't a RecordableEvent. If fired becase of an alert call from a test, for example, the AlertProvided will immediately be sent to the reporters rather than being stored and sent in the recordedEvents field of the test completion event. Thus, AlertProvided enables "status notifications" to be provided while tests are happening. The difference between AlertProvided and NoteProvided, which is also a "status notification" fired immediately during tests, is that AlertProvided is intended for warnings, where as NoteProvided is just for information. As an illustration, AlertProvided messages are displayed in yellow, NoteProvided in green, in the stdout, stderr, and file reporters.

    An AlertProvided event may be fired from anywhere. In this respect AlertProvided is different from events for which it is defined whether they are fired in the context of a suite or test. If fired in the context of a test, the AlertProvided event should include a NameInfo in which testName is defined. If fired in the context of a suite, but not a test, the AlertProvided event should include a NameInfo in which testName is not defined. If fired within the context of neither a suite nor a test, the nameInfo of the AlertProvided event (an Option[NameInfo]) should be None.

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    message

    a localized message suitable for presenting to the user

    nameInfo

    an optional NameInfo that if defined, provides names for the suite and optionally the test in the context of which the information was provided

    throwable

    an optional Throwable

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    An optional Location that provides information indicating where in the source code an event originated.

    payload

    an optional object that can be used to pass custom information to the reporter about the AlertProvided event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  2. final case class DiscoveryCompleted(ordinal: Ordinal, duration: Option[Long] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with Product with Serializable

    Event that indicates a runner has completed searching for suites.

    Event that indicates a runner has completed searching for suites.

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    duration

    an optional amount of time, in milliseconds, that was required by the run that has completed

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  3. final case class DiscoveryStarting(ordinal: Ordinal, configMap: ConfigMap, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with Product with Serializable

    Event that indicates a runner is beginning search for suites to run.

    Event that indicates a runner is beginning search for suites to run.

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    configMap

    a ConfigMap of key-value pairs that can be used by custom Reporters

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  4. sealed abstract class Event extends Ordered[Event] with Product with Serializable

    A base class for the events that can be passed to the report function passed to the execute method of a Suite.

  5. sealed trait ExceptionalEvent extends Event

    Marker trait for test failed and test canceled events.

  6. sealed abstract class Formatter extends Product with Serializable

    Abstract class for the optional formatter objects that must be passed to the Events reported during a ScalaTest run.

  7. final case class IndentedText(formattedText: String, rawText: String, indentationLevel: Int) extends Formatter with Product with Serializable

    A Formatter providing information that enables reporters to create more stylized output.

    A Formatter providing information that enables reporters to create more stylized output.

    An example is that specification-style suites, such as FunSpec, generate output that reads more like a specification, for instance:

    A Stack (when newly created)
- should be empty
- should complain when popped

    This output might be generated by ScalaTest's standard out reporter. Each of these lines would be taken from the IndentedText's formattedText parameter. Were this same run to be reported in HTML or in a GUI, the output would be based on the rawText and the indentationLevel. Here's what the IndentedText values would be for each event:

    • InfoProvided reported with an: 
        IndentedText(
    formattedText = "A Stack (when newly created)",
    rawText = "A Stack (when newly created)",
    indentationLevel = 0
  )
    • TestSucceeded reported with an: 
        IndentedText(
    formattedText = "- should be empty",
    rawText = "should be empty",
    indentationLevel = 1
  )
    • TestSucceeded reported with an: 
        IndentedText(
    formattedText = "- should complain when popped",
    rawText = "should complain when popped",
    indentationLevel = 1
  )

    One possible way this information could be presented in HTML, for example, is this:

    A Stack (when newly created)

    • should be empty
    • should complain when popped
    formattedText

    a localized string suitable for presenting to a user by printing it straight to an output stream

    rawText

    a localized string suitable for presenting to the user after in some way being indented by the value specified as the indentationLevel parameter

    indentationLevel

    a zero or positive integer representing an indentation level for the indented text

    Exceptions thrown

    IllegalArgumentException if the specified indentationLevel is less than zero

  8. final case class InfoProvided(ordinal: Ordinal, message: String, nameInfo: Option[NameInfo], throwable: Option[Throwable] = None, formatter: Option[Formatter] = None, location: Option[Location] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with RecordableEvent with Product with Serializable

    Event used to provide information that is not appropriate to report via any other Event.

    Event used to provide information that is not appropriate to report via any other Event.

    To create instances of this class you may use the factory method provided in its companion object. For example, given a report function named report, you could fire a InfoProvided event like this:

    report(InfoProvided(ordinal, message, Some(NameInfo(suiteName, suiteId, Some(thisSuite.getClass.getName), Some(testName)))))

    An InfoProvided event may be fired from anywhere. In this respect InfoProvided is different from events for which it is defined whether they are fired in the context of a suite or test. If fired in the context of a test, the InfoProvided event should include a NameInfo in which testName is defined. If fired in the context of a suite, but not a test, the InfoProvided event should include a NameInfo in which testName is not defined. If fired within the context of neither a suite nor a test, the nameInfo of the InfoProvided event (an Option[NameInfo]) should be None.

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    message

    a localized message suitable for presenting to the user

    nameInfo

    an optional NameInfo that if defined, provides names for the suite and optionally the test in the context of which the information was provided

    throwable

    an optional Throwable

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    An optional Location that provides information indicating where in the source code an event originated.

    payload

    an optional object that can be used to pass custom information to the reporter about the InfoProvided event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  9. final case class LineInFile(lineNumber: Int, fileName: String, filePathname: Option[String]) extends Location with Product with Serializable

    An arbitrary line number in a named source file.

    An arbitrary line number in a named source file.

    lineNumber

    the line number

    fileName

    the source file's simple name

    filePathname

    an optional fully qualified pathname of the source file

  10. sealed abstract class Location extends Product with Serializable

    Location in source code indicating where in the source code an event originated.

  11. final case class MarkupProvided(ordinal: Ordinal, text: String, nameInfo: Option[NameInfo], formatter: Option[Formatter] = None, location: Option[Location] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with RecordableEvent with Product with Serializable

    Event used to provide markup text for document-style reports.

    Event used to provide markup text for document-style reports.

    To create instances of this class you may use the factory method. For example, given a report function named report, you could fire a MarkupProvided event like this:

    report(MarkupProvided(ordinal, text, Some(NameInfo(suiteName, suiteId, Some(thisSuite.getClass.getName), Some(testName)))))

    A MarkupProvided event may be fired from anywhere. In this respect MarkupProvided is different from the other events, for which it is defined whether they are fired in the context of a suite or test. If fired in the context of a test, the MarkupProvided event should include a NameInfo in which testName is defined. If fired in the context of a suite, but not a test, the MarkupProvided event should include a NameInfo in which testName is not defined. If fired within the context of neither a suite nor a test, the nameInfo of the MarkupProvided event (an Option[NameInfo]) should be None.

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    text

    a snippet of markup text (in Markdown format)

    nameInfo

    an optional NameInfo that if defined, provides names for the suite and optionally the test in the context of which the information was provided

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    An optional Location that provides information indicating where in the source code an event originated.

    payload

    an optional object that can be used to pass custom information to the reporter about the MarkupProvided event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  12. final case class NameInfo(suiteName: String, suiteId: String, suiteClassName: Option[String], testName: Option[String]) extends Product with Serializable

    Class that holds information about names for the information events InfoProvided, MarkupProvided, ScopeOpened, ScopeClosed, ScopePending, AlertProvided and NoteProvided.

    Class that holds information about names for the information events InfoProvided, MarkupProvided, ScopeOpened, ScopeClosed, ScopePending, AlertProvided and NoteProvided.

    An information event may be fired from anywhere. In this respect these events are different from the other events, for which it is defined whether they are fired in the context of a suite or test. If fired in the context of a test, an information event event should include a NameInfo in which testName is defined. If fired in the context of a suite, but not a test, the InfoProvided event should include a NameInfo in which testName is not defined. If fired within the context of neither a suite nor a test, the nameInfo of the InfoProvided event (an Option[NameInfo]) should be None.

    If either suiteClassName or testName is defined, then suiteName and suiteId must be defined. The suite class name parameter is optional even if a suite name is provided by passing a Some as suiteName, because suites in ScalaTest are an abstraction that need not necessarily correspond to one class. Nevertheless, in most cases each suite will correspond to a class, and when it does, the fully qualified name of that class should be reported by passing a Some for suiteClassName. One use for this bit of information is JUnit integration, because the "name" provided to a JUnit org.junit.runner.Description appears to usually include a fully qualified class name by convention.

    suiteName

    an optional name of the suite about which an information event was fired

    suiteId

    an optional string ID for the suite about which an information event was fired, intended to be unique across all suites in a run

    suiteClassName

    an optional fully qualifed Suite class name about which the information was provided

    testName

    an optional test name information

  13. final case class NoteProvided(ordinal: Ordinal, message: String, nameInfo: Option[NameInfo], throwable: Option[Throwable] = None, formatter: Option[Formatter] = None, location: Option[Location] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with NotificationEvent with Product with Serializable

    Event used to provide notifications.

    Event used to provide notifications.

    To create instances of this class you may use the factory method. For example, given a report function named report, you could fire a NoteProvided event like this:

    report(NoteProvided(ordinal, message, Some(NameInfo(suiteName, suiteId, Some(thisSuite.getClass.getName), Some(testName)))))

    NoteProvided differs from InfoProvided in that unlike InfoProvided, NoteProvided isn't a RecordableEvent. If fired because of a note call from a test, for example, the NoteProvided will immediately be sent to the reporters rather than being stored and sent in the recordedEvents field of the test completion event. Thus, NoteProvided enables "status notifications" to be provided while tests are happening. The difference between NoteProvided and AlertProvided, which is also a "status notification" fired immediately during tests, is that AlertProvided is intended for warnings, where as NoteProvided is just for information. As an illustration, AlertProvided messages are displayed in yellow, NoteProvided in green, in the stdout, stderr, and file reporters.

    An NoteProvided event may be fired from anywhere. In this respect NoteProvided is different from events for which it is defined whether they are fired in the context of a suite or test. If fired in the context of a test, the NoteProvided event should include a NameInfo in which testName is defined. If fired in the context of a suite, but not a test, the NoteProvided event should include a NameInfo in which testName is not defined. If fired within the context of neither a suite nor a test, the nameInfo of the NoteProvided event (an Option[NameInfo]) should be None.

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    message

    a localized message suitable for presenting to the user

    nameInfo

    an optional NameInfo that if defined, provides names for the suite and optionally the test in the context of which the information was provided

    throwable

    an optional Throwable

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    An optional Location that provides information indicating where in the source code an event originated.

    payload

    an optional object that can be used to pass custom information to the reporter about the NoteProvided event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  14. sealed trait NotificationEvent extends Event

    Marker trait for the "notification" events NoteProvided and AlertProvided.

  15. final class Ordinal extends Ordered[Ordinal] with Serializable

    Class used to specify a sequential order for events reported during a test run, so they can be arranged in that order in a report even if the events were fired in some other order during concurrent or distributed execution.

    Class used to specify a sequential order for events reported during a test run, so they can be arranged in that order in a report even if the events were fired in some other order during concurrent or distributed execution.

    An Ordinal is an immutable object holding a run stamp and a sequence of stamps. The run stamp is an integer that identifies a particular run. All events reported during the same run should share the same run stamp. By contrast, each event reported during a particular run should have a different stamp sequence. One use case for the run stamp is that the initial run from ScalaTest's GUI will have run stamp 0. Subsequent reruns will have run stamps 1, 2, 3, etc., so that reports in the GUI can simply be sorted in "ordinal" order. Another use case is a set of servers used to run multiple tests simultaneously in a distributed fashion. The run stamp can be used to identify the run to which an event belongs.

    The stamp sequence is designed to allow a sequential order of events to be specified during concurrent execution of ScalaTest suites. ScalaTest's model for concurrent execution is that the suites that make up a run may be executed concurrently, but the tests within a single suite will be executed sequentially. In addition to tests, suites may contain nested suites. The default implementation of execute in class Suite will first invoke runNestedSuites and then runTests. If no Distributor is passed to execute, the runNestedSuites method will execute the nested suites sequentially via the same thread that invoked runNestedSuites. As a result, suites will by default executed in depth first order when executed sequentially. If a Distributor is passed to execute, the runNestedSuites method will simply put its nested suites into the Distributor and return. Some other threads or processes must then execute those nested suites. Given the default implementations of execute and runNestedSuites described here, the Ordinal will allow the events from a concurrent run to be sorted in the same depth-first order that the events from a corresponding sequential run would arrive.

    Each event reported during a run should be given a unique Ordinal. An Ordinal is required by all Event subclasses, instances of which are used to send information to the report function passed to a Suite's execute method. The first Ordinal for a run can be produced by passing a run stamp to Ordinal's lone public constructor:

    val firstOrdinal = new Ordinal(99)

    The run stamp can be any integer. The Ordinal created in this way can be passed along with the first reported event of the run, such as a RunStarting event. Thereafter, new Ordinals for the same run can be obtained by calling either next or nextNewOldPair on the previously obtained Ordinal. In other words, given an Ordinal, you can obtain the next Ordinal by invoking one of these two "next" methods on the Ordinal you have in hand. Before executing a new Suite, the nextNewOldPair method should be invoked. This will return two new Ordinals, one for the new Suite about to be executed, and one for the currently executing entity (either a Suite or some sort of test runner). At any other time, the next Ordinal can be obtained by simply invoking next on the current Ordinal.

    You can convert an Ordinal to a List by invoking toList on it. The resulting List will contain the run stamp as its first element, and the contents of its stamps sequence as the subsequent elements. The stamps sequence will initially be composed of a single element with the value 0. Thus, toList invoked on the firstOrdinal shown above will result in:

    firstOrdinal.toList // results in: List(99, 0)

    Each time next is invoked, the rightmost integer returned by toList will increment:

    val secondOrdinal = firstOrdinal.next
secondOrdinal.toList // results in: List(99, 1)

val thirdOrdinal = secondOrdinal.next
thirdOrdinal.toList  // result is : List(99, 2)

    When nextNewOldPair is invoked the result will be a tuple whose first element is the first Ordinal for the new Suite about to be executed (for example, a nested Suite of the currently executing Suite). The second element is the next Ordinal for the currently executing Suite or other entity:

    val (nextForNewSuite, nextForThisRunner) = thirdOrdinal.nextNewOldPair
nextForNewSuite.toList   // results in: (99, 2, 0)
nextForThisRunner.toList // results in: (99, 3)

    The toList method of the Ordinal for the new suite starts with the same sequence of elements as the Ordinal from which it was created, but has one more element, a 0, appended at the end. Subsequent invocations of next on this series of Ordinals will increment that last element:

    val newSuiteOrdinal2 = nextForNewSuite.next
newSuiteOrdinal2.toList // results in: List(99, 2, 1)

val newSuiteOrdinal3 = newSuiteOrdinal2.next
newSuiteOrdinal3.toList  // result is : List(99, 2, 2)

    This behavior allows events fired by Suite running concurrently to be reordered in a pre-determined sequence after all the events have been reported. The ordering of two Ordinals can be determined by first comparing the first element of the Lists obtained by invoking toList on both Ordinals. These values represent the runStamp. If one run stamp is a lower number than the other, that Ordinal comes first. For example, an Ordinal with a run stamp of 98 is ordered before an Ordinal with a run stamp of 99. If the run stamps are equal, the next number in the list is inspected. As with the run stamps, an Ordinal with a lower number is ordered before an Ordinal with a higher number. If two corresponding elements are equal, the next pair of elements will be inspected. This will continue no down the length of the Lists until a position is found where the element values are not equal, or the end of one or both of the Lists are reached. If the two Lists are identical all the way to the end, and both Lists have the same lengths, then the Ordinals are equal. (Equal Ordinals will not happen if correctly used by creating a new Ordinal for each fired event and each new Suite.). If the two Lists are identical all the way to the end of one, but the other List is longer (has more elements), then the shorter list is ordered before the longer one.

    As an example, here are some Ordinal List forms in order:

    List(99, 0)
List(99, 1)
List(99, 2)
List(99, 2, 0)
List(99, 2, 1)
List(99, 2, 2)
List(99, 2, 2, 0)
List(99, 2, 2, 1)
List(99, 2, 2, 2)
List(99, 2, 3)
List(99, 2, 4)
List(99, 2, 4, 0)
List(99, 2, 4, 1)
List(99, 2, 4, 2)
List(99, 3)
List(99, 4)
List(99, 4, 0)
List(99, 4, 1)
List(99, 5)

  16. sealed trait RecordableEvent extends Event

    Marker trait for test completed event's recordedEvents.

  17. final case class RunAborted(ordinal: Ordinal, message: String, throwable: Option[Throwable], duration: Option[Long] = None, summary: Option[Summary] = None, formatter: Option[Formatter] = None, location: Option[Location] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with Product with Serializable

    Event that indicates a runner encountered an error while attempting to run a suite of tests.

    Event that indicates a runner encountered an error while attempting to run a suite of tests.

    For example, object Runner reports RunAborted if the execute method of any of the run's starting Suites completes abruptly with a Throwable.

    ScalaTest's Runner fires a RunAborted report with an empty summary, because the reporter is responsible for keeping track of the total number of tests reported as succeeded, failed, ignored, and pending. ScalaTest's internal reporter replaces the RunAborted with a new one that is identical except that is has a defined summary.

    To create instances of this class you may use the factory method. For example, given a report function named report, you could fire a RunAborted event like this:

    report(RunAborted(ordinal, message, Some(exception)))

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    message

    a localized message suitable for presenting to the user

    throwable

    an optional Throwable that, if a Some, indicates why the run has aborted, or a Throwable created to capture stack trace information about the problem.

    duration

    an optional amount of time, in milliseconds, that was required by the run that has aborted

    summary

    an optional Summary of the number of tests that were reported as succeeded, failed, ignored, and pending

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    An optional Location that provides information indicating where in the source code an event originated.

    payload

    an optional object that can be used to pass custom information to the reporter about the RunAborted event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  18. final case class RunCompleted(ordinal: Ordinal, duration: Option[Long] = None, summary: Option[Summary] = None, formatter: Option[Formatter] = None, location: Option[Location] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with Product with Serializable

    Event that indicates a runner has completed running a suite of tests.

    Event that indicates a runner has completed running a suite of tests.

    Suite's execute method takes a Stopper, whose stopRequested method indicates a stop was requested. If true is returned by stopRequested while a suite of tests is running, the execute method should promptly return even if that suite hasn't finished running all of its tests.

    If a stop was requested via the Stopper. Runner will report RunStopped when the execute method of the run's starting Suite returns. If a stop is not requested, Runner will report RunCompleted when the last execute method of the run's starting Suites returns.

    ScalaTest's Runner fires a RunCompleted report with an empty summary, because the reporter is responsible for keeping track of the total number of tests reported as succeeded, failed, ignored, pending and canceled. ScalaTest's internal reporter replaces the RunCompleted with a new one that is identical except that is has a defined summary.

    To create instances of this class you may use the factory method. For example, given a report function named report, you could fire a RunCompleted event like this:

    report(RunCompleted(ordinal))

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    duration

    an optional amount of time, in milliseconds, that was required by the run that has completed

    summary

    an optional Summary of the number of tests that were reported as succeeded, failed, ignored, pending and canceled

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    An optional Location that provides information indicating where in the source code an event originated.

    payload

    an optional object that can be used to pass custom information to the reporter about the RunCompleted event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  19. final case class RunStarting(ordinal: Ordinal, testCount: Int, configMap: ConfigMap, formatter: Option[Formatter] = None, location: Option[Location] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with Product with Serializable

    Event that indicates a runner is about run a suite of tests.

    Event that indicates a runner is about run a suite of tests.

    For example, object Runner reports RunStarting to indicate that the first execute method of a run's initial Suite is about to be invoked.

    To create instances of this class you may use the factory method. For example, given a report function named report, you could fire a RunStarting event like this:

    report(RunStarting(ordinal, testCount))

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    testCount

    the number of tests expected during this run

    configMap

    a ConfigMap of key-value pairs that can be used by custom Reporters

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    An optional Location that provides information indicating where in the source code an event originated.

    payload

    an optional object that can be used to pass custom information to the reporter about the RunStarting event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

    Exceptions thrown

    IllegalArgumentException if testCount is less than zero.

  20. final case class RunStopped(ordinal: Ordinal, duration: Option[Long] = None, summary: Option[Summary] = None, formatter: Option[Formatter] = None, location: Option[Location] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with Product with Serializable

    Event that indicates a runner has stopped running a suite of tests prior to completion, likely because of a stop request.

    Event that indicates a runner has stopped running a suite of tests prior to completion, likely because of a stop request.

    Suite's execute method takes a Stopper, whose stopRequested method indicates a stop was requested. If true is returned by stopRequested while a suite of tests is running, the execute method should promptly return even if that suite hasn't finished running all of its tests.

    If a stop was requested via the Stopper. Runner will report RunStopped when the execute method of the run's starting Suite returns. If a stop is not requested, Runner will report RunCompleted when the last execute method of the run's starting Suites returns.

    ScalaTest's Runner fires a RunStopped report with an empty summary, because the reporter is responsible for keeping track of the total number of tests reported as succeeded, failed, ignored, pending and canceled. ScalaTest's internal reporter replaces the RunStopped with a new one that is identical except that is has a defined summary.

    To create instances of this class you may use the factory method. For example, given a report function named report, you could fire a RunStopped event like this:

    report(RunStopped(ordinal))

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    duration

    an optional amount of time, in milliseconds, that was required by the run that has stopped

    summary

    an optional summary of the number of tests that were reported as succeeded, failed, ignored pending and canceled

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    An optional Location that provides information indicating where in the source code an event originated.

    payload

    an optional object that can be used to pass custom information to the reporter about the RunStopped event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  21. final case class ScopeClosed(ordinal: Ordinal, message: String, nameInfo: NameInfo, formatter: Option[Formatter] = None, location: Option[Location] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with Product with Serializable

    Event that indicates a scope has been closed.

    Event that indicates a scope has been closed.

    To create instances of this class you may use the factory method. For example, given a report function named report, you could fire a ScopeClosed event like this:

    report(ScopeClosed(ordinal, message, Some(NameInfo(suiteName, suiteId, Some(thisSuite.getClass.getName), Some(testName)))))

    A ScopeClosed event may be fired from within suites or tests. If fired in the context of a test, the ScopeClosed event should include a NameInfo in which testName is defined. If fired in the context of a suite, but not a test, the ScopeClosed event should include a NameInfo in which testName is not defined.

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    message

    a localized message suitable for presenting to the user

    nameInfo

    a NameInfo that provides names for the suite and optionally the test in the context of which the scope was closed

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    An optional Location that provides information indicating where in the source code an event originated.

    payload

    an optional object that can be used to pass custom information to the reporter about the ScopeClosed event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  22. final case class ScopeOpened(ordinal: Ordinal, message: String, nameInfo: NameInfo, formatter: Option[Formatter] = None, location: Option[Location] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with Product with Serializable

    Event that indicates a new scope has been opened.

    Event that indicates a new scope has been opened.

    To create instances of this class you may use the factory method. For example, given a report function named report, you could fire a ScopeOpened event like this:

    report(ScopeOpened(ordinal, message, Some(NameInfo(suiteName, suiteId, Some(thisSuite.getClass.getName), Some(testName)))))

    A ScopeOpened event may be fired from within suites or tests. If fired in the context of a test, the ScopeOpened event should include a NameInfo in which testName is defined. If fired in the context of a suite, but not a test, the ScopeOpened event should include a NameInfo in which testName is not defined.

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    message

    a localized message suitable for presenting to the user

    nameInfo

    a NameInfo that provides names for the suite and optionally the test in the context of which the scope was opened

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    An optional Location that provides information indicating where in the source code an event originated.

    payload

    an optional object that can be used to pass custom information to the reporter about the ScopeOpened event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  23. final case class ScopePending(ordinal: Ordinal, message: String, nameInfo: NameInfo, formatter: Option[Formatter] = None, location: Option[Location] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with Product with Serializable

    Event that indicates a scope is pending.

    Event that indicates a scope is pending.

    To create instances of this class you may use the factory method. For example, given a report function named report, you could fire a ScopePending event like this:

    report(ScopePending(ordinal, message, Some(NameInfo(suiteName, Some(thisSuite.getClass.getName), Some(testName)))))

    A ScopePending event is fired from within suites, and not tests. The ScopePending event should include a NameInfo in which testName is not defined.

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    message

    a localized message suitable for presenting to the user

    nameInfo

    a NameInfo that provides names for the suite and optionally the test in the context of which the scope was closed

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    An optional Location that provides information indicating where in the source code an event originated.

    payload

    an optional object that can be used to pass custom information to the reporter about the ScopePending event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  24. final case class SuiteAborted(ordinal: Ordinal, message: String, suiteName: String, suiteId: String, suiteClassName: Option[String], throwable: Option[Throwable] = None, duration: Option[Long] = None, formatter: Option[Formatter] = None, location: Option[Location] = None, rerunner: Option[String] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with ExceptionalEvent with Product with Serializable

    Event that indicates the execution of a suite of tests has aborted, likely because of an error, prior to completion.

    Event that indicates the execution of a suite of tests has aborted, likely because of an error, prior to completion.

    For example, trait Suite and object Runner use SuiteAborted to report that the execute method of a Suite has completed abruptly with a RuntimeException.

    To create instances of this class you may use the factory method. For example, given a report function named report, you could fire a SuiteAborted event like this:

    report(SuiteAborted(ordinal, userFriendlyName, message, suiteName, Some(thisSuite.getClass.getName)))

    The suite class name parameter is optional, because suites in ScalaTest are an abstraction that need not necessarily correspond to one class. Nevertheless, in most cases each suite will correspond to a class, and when it does, the fully qualified name of that class should be reported by passing a Some for suiteClassName. One use for this bit of information is JUnit integration, because the "name" provided to a JUnit org.junit.runner.Description appears to usually include a fully qualified class name by convention.

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run suite name, suitable for presenting to the user

    message

    a localized message suitable for presenting to the user

    suiteName

    a localized name identifying the suite that has aborted, suitable for presenting to the user

    suiteId

    a string ID for the suite that has aborted, intended to be unique across all suites in a run

    suiteClassName

      an optional fully qualifed Suite class name containing the suite that has aborted

    throwable

    an optional Throwable that, if a Some, indicates why the suite has aborted, or a Throwable created to capture stack trace information about the problem.

    duration

    an optional amount of time, in milliseconds, that was required to execute the suite that has aborted

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    an optional Location that provides information indicating where in the source code an event originated.

    rerunner

    an optional String giving the fully qualified name of the class that can be used to rerun the suite that has aborted. (If None is passed, the suite cannot be rerun.)

    payload

    an optional object that can be used to pass custom information to the reporter about the SuiteAborted event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  25. final case class SuiteCompleted(ordinal: Ordinal, suiteName: String, suiteId: String, suiteClassName: Option[String], duration: Option[Long] = None, formatter: Option[Formatter] = None, location: Option[Location] = None, rerunner: Option[String] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with Product with Serializable

    Event that indicates a suite of tests has completed executing.

    Event that indicates a suite of tests has completed executing.

    For example, trait Suite and object Runner use SuiteCompleted to report that the execute method of a Suite has returned normally (without throwing a RuntimeException).

    To create instances of this class you may use the factory method. For example, given a report function named report, you could fire a SuiteCompleted event like this:

    report(SuiteCompleted(ordinal, userFriendlyName, suiteName, Some(thisSuite.getClass.getName)))

    The suite class name parameter is optional, because suites in ScalaTest are an abstraction that need not necessarily correspond to one class. Nevertheless, in most cases each suite will correspond to a class, and when it does, the fully qualified name of that class should be reported by passing a Some for suiteClassName. One use for this bit of information is JUnit integration, because the "name" provided to a JUnit org.junit.runner.Description appears to usually include a fully qualified class name by convention.

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    suiteName

    a localized name identifying the suite that has completed, suitable for presenting to the user

    suiteId

    a string ID for the suite that has completed, intended to be unique across all suites in a run

    suiteClassName

    an optional fully qualifed Suite class name containing the suite that has completed

    duration

    an optional amount of time, in milliseconds, that was required to execute the suite that has completed

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    An optional Location that provides information indicating where in the source code an event originated.

    rerunner

    an optional String giving the fully qualified name of the class that can be used to rerun the suite that has completed. (If None is passed, the suite cannot be rerun.)

    payload

    an optional object that can be used to pass custom information to the reporter about the SuiteCompleted event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  26. final case class SuiteStarting(ordinal: Ordinal, suiteName: String, suiteId: String, suiteClassName: Option[String], formatter: Option[Formatter] = None, location: Option[Location] = None, rerunner: Option[String] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with Product with Serializable

    Event that indicates a suite of tests is about to start executing.

    Event that indicates a suite of tests is about to start executing.

    For example, trait Suite and object Runner use SuiteStarting to report that the execute method of a Suite is about to be invoked.

    To create instances of this class you may use the factory method. For example, given a report function named report, you could fire a SuiteStarting event like this:

    report(SuiteStarting(ordinal, userFriendlyName, suiteName, Some(thisSuite.getClass.getName)))

    The suite class name parameter is optional, because suites in ScalaTest are an abstraction that need not necessarily correspond to one class. Nevertheless, in most cases each suite will correspond to a class, and when it does, the fully qualified name of that class should be reported by passing a Some for suiteClassName. One use for this bit of information is JUnit integration, because the "name" provided to a JUnit org.junit.runner.Description appears to usually include a fully qualified class name by convention.

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    suiteName

    a localized name identifying the suite that is starting, suitable for presenting to the user

    suiteId

    a string ID for the suite that is starting, intended to be unique across all suites in a run XXX

    suiteClassName

    an optional fully qualifed Suite class name of the suite that is starting

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    An optional Location that provides information indicating where in the source code an event originated.

    rerunner

    an optional String giving the fully qualified name of the class that can be used to rerun the suite that is starting. (If None is passed, the suite cannot be rerun.)

    payload

    an optional object that can be used to pass custom information to the reporter about the SuiteStarting event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  27. final case class Summary(testsSucceededCount: Int, testsFailedCount: Int, testsIgnoredCount: Int, testsPendingCount: Int, testsCanceledCount: Int, suitesCompletedCount: Int, suitesAbortedCount: Int, scopesPendingCount: Int) extends Product with Serializable

    Class each of whose instances hold summary information about one ScalaTest run.

    Class each of whose instances hold summary information about one ScalaTest run.

    testsSucceededCount

    the number of tests that were reported as succeeded during the run

    testsFailedCount

    the number of tests that were reported as failed during the run

    testsIgnoredCount

    the number of tests that were were reported as ignored during the run

    testsPendingCount

    the number of tests that were reported as pending during the run

    testsCanceledCount

    the number of tests that were reported as canceled during the run

    suitesCompletedCount

    the number of suites that were reported as completed during the run

    suitesAbortedCount

    the number of suites that were reported as aborted during the run

    scopesPendingCount

    the number of scopes that were reported as pending during the run

  28. final case class TestCanceled(ordinal: Ordinal, message: String, suiteName: String, suiteId: String, suiteClassName: Option[String], testName: String, testText: String, recordedEvents: IndexedSeq[RecordableEvent], throwable: Option[Throwable] = None, duration: Option[Long] = None, formatter: Option[Formatter] = None, location: Option[Location] = None, rerunner: Option[String] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with ExceptionalEvent with Product with Serializable

    Event that indicates a test was canceled, i.e., it couldn't run because some precondition was not met.

    Event that indicates a test was canceled, i.e., it couldn't run because some precondition was not met.

    To create instances of this class you may use the factory methods. For example, given a report function named report, you could fire a TestCanceled event like this:

    report(TestCanceled(ordinal, userFriendlyName, suiteName, Some(thisSuite.getClass.getName), testName))

    The suite class name parameter is optional, because suites in ScalaTest are an abstraction that need not necessarily correspond to one class. Nevertheless, in most cases each suite will correspond to a class, and when it does, the fully qualified name of that class should be reported by passing a Some for suiteClassName. One use for this bit of information is JUnit integration, because the "name" provided to a JUnit org.junit.runner.Description appears to usually include a fully qualified class name by convention.

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    message

    a localized message suitable for presenting to the user

    suiteName

    a localized name identifying the suite containing the test that was canceled, suitable for presenting to the user

    suiteId

    a string ID for the suite containing the test that was canceled, intended to be unique across all suites in a run

    suiteClassName

    an optional fully qualifed Suite class name containing the test that was canceled

    testName

    the name of the test that was canceled

    testText

    the text of the test that was canceled (may be the test name, or a suffix of the test name)

    recordedEvents

    recorded events in the test.

    throwable

    an optional Throwable that, if a Some, indicates why the test was canceled, or a Throwable created to capture stack trace information about the problem.

    duration

    an optional amount of time, in milliseconds, that was required to run the test that was canceled

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    An optional Location that provides information indicating where in the source code an event originated.

    rerunner

    an optional String giving the fully qualified name of the class that can be used to rerun the test that has canceled. (If None is passed, the test cannot be rerun.)

    payload

    an optional object that can be used to pass custom information to the reporter about the TestCanceled event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  29. final case class TestFailed(ordinal: Ordinal, message: String, suiteName: String, suiteId: String, suiteClassName: Option[String], testName: String, testText: String, recordedEvents: IndexedSeq[RecordableEvent], analysis: IndexedSeq[String], throwable: Option[Throwable] = None, duration: Option[Long] = None, formatter: Option[Formatter] = None, location: Option[Location] = None, rerunner: Option[String] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with ExceptionalEvent with Product with Serializable

    Event that indicates a suite (or other entity) has completed running a test that failed.

    Event that indicates a suite (or other entity) has completed running a test that failed.

    For example, trait Suite uses TestFailed to report that a test method of a Suite completed abruptly with an Exception.

    To create instances of this class you may use the factory method. For example, given a report function named report, you could fire a TestFailed event like this:

    report(TestFailed(ordinal, userFriendlyName, message, suiteName, Some(thisSuite.getClass.getName), testName))

    The suite class name parameter is optional, because suites in ScalaTest are an abstraction that need not necessarily correspond to one class. Nevertheless, in most cases each suite will correspond to a class, and when it does, the fully qualified name of that class should be reported by passing a Some for suiteClassName. One use for this bit of information is JUnit integration, because the "name" provided to a JUnit org.junit.runner.Description appears to usually include a fully qualified class name by convention.

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    message

    a localized message suitable for presenting to the user

    suiteName

    a localized name identifying the suite containing the test that has failed, suitable for presenting to the user

    suiteId

    a string ID for the suite containing the test that has failed, intended to be unique across all suites in a run

    suiteClassName

    an optional fully qualifed Suite class name containing the test that has failed

    testName

    the name of the test that has failed

    testText

    the text of the test that has failed (may be the test name, or a suffix of the test name)

    recordedEvents

    recorded events in the test.

    throwable

    an optional Throwable that, if a Some, indicates why the test has failed, or a Throwable created to capture stack trace information about the problem.

    duration

    an optional amount of time, in milliseconds, that was required to run the test that has failed

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    An optional Location that provides information indicating where in the source code an event originated.

    rerunner

    an optional String giving the fully qualified name of the class that can be used to rerun the test that has failed. (If None is passed, the test cannot be rerun.)

    payload

    an optional object that can be used to pass custom information to the reporter about the TestFailed event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  30. final case class TestIgnored(ordinal: Ordinal, suiteName: String, suiteId: String, suiteClassName: Option[String], testName: String, testText: String, formatter: Option[Formatter] = None, location: Option[Location] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with Product with Serializable

    Event that indicates a suite (or other entity) has ignored a test.

    Event that indicates a suite (or other entity) has ignored a test.

    For example, trait Suite uses TestIgnored to report that a test method of a Suite was ignored because it was annotated with @Ignore. Ignored tests will not be run, but will usually be reported as reminder to fix the broken test.

    To create instances of this class you may use the factory method. For example, given a report function named report, you could fire a TestIgnored event like this:

    report(TestIgnored(ordinal, userFriendlyName, suiteName, Some(thisSuite.getClass.getName), testName))

    The suite class name parameter is optional, because suites in ScalaTest are an abstraction that need not necessarily correspond to one class. Nevertheless, in most cases each suite will correspond to a class, and when it does, the fully qualified name of that class should be reported by passing a Some for suiteClassName. One use for this bit of information is JUnit integration, because the "name" provided to a JUnit org.junit.runner.Description appears to usually include a fully qualified class name by convention.

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    suiteName

    a localized name identifying the suite containing the test that was ignored, suitable for presenting to the user

    suiteId

    a string ID for the suite containing the test that was ignored, intended to be unique across all suites in a run

    suiteClassName

    an optional fully qualifed Suite class name containing the test that was ignored

    testName

    the name of the test that was ignored

    testText

    the text of the test that was ignored (may be the test name, or a suffix of the test name)

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    An optional Location that provides information indicating where in the source code an event originated.

    payload

    an optional object that can be used to pass custom information to the reporter about the TestIgnored event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  31. final case class TestPending(ordinal: Ordinal, suiteName: String, suiteId: String, suiteClassName: Option[String], testName: String, testText: String, recordedEvents: IndexedSeq[RecordableEvent], duration: Option[Long] = None, formatter: Option[Formatter] = None, location: Option[Location] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with Product with Serializable

    Event that indicates a test is pending, i.e., it hasn't yet been implemented.

    Event that indicates a test is pending, i.e., it hasn't yet been implemented.

    To create instances of this class you may use the factory method. For example, given a report function named report, you could fire a TestPending event like this:

    report(TestPending(ordinal, userFriendlyName, suiteName, Some(thisSuite.getClass.getName), testName))

    The suite class name parameter is optional, because suites in ScalaTest are an abstraction that need not necessarily correspond to one class. Nevertheless, in most cases each suite will correspond to a class, and when it does, the fully qualified name of that class should be reported by passing a Some for suiteClassName. One use for this bit of information is JUnit integration, because the "name" provided to a JUnit org.junit.runner.Description appears to usually include a fully qualified class name by convention.

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    suiteName

    a localized name identifying the suite containing the test that is pending, suitable for presenting to the user

    suiteId

    a string ID for the suite containing the test that is pending, intended to be unique across all suites in a run

    suiteClassName

    an optional fully qualifed Suite class name containing the test that is pending

    testName

    the name of the test that is pending

    testText

    the text of the test that is pending (may be the test name, or a suffix of the test name)

    recordedEvents

    recorded events in the test.

    duration

    an optional amount of time, in milliseconds, that was required to run the test that is pending

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    An optional Location that provides information indicating where in the source code an event originated.

    payload

    an optional object that can be used to pass custom information to the reporter about the TestPending event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  32. final case class TestStarting(ordinal: Ordinal, suiteName: String, suiteId: String, suiteClassName: Option[String], testName: String, testText: String, formatter: Option[Formatter] = None, location: Option[Location] = None, rerunner: Option[String] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with Product with Serializable

    Event that indicates a suite (or other entity) is about to start running a test.

    Event that indicates a suite (or other entity) is about to start running a test.

    For example, trait Suite uses TestStarting to report that a test method of a Suite is about to be invoked.

    To create instances of this class you may use the factory method. For example, given a report function named report, you could fire a TestStarting event like this:

    report(TestStarting(ordinal, userFriendlyName, suiteName, Some(thisSuite.getClass.getName), testName))

    The suite class name parameter is optional, because suites in ScalaTest are an abstraction that need not necessarily correspond to one class. Nevertheless, in most cases each suite will correspond to a class, and when it does, the fully qualified name of that class should be reported by passing a Some for suiteClassName. One use for this bit of information is JUnit integration, because the "name" provided to a JUnit org.junit.runner.Description appears to usually include a fully qualified class name by convention.

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    suiteName

    a localized name identifying the suite containing the test that is starting, suitable for presenting to the user

    suiteId

    a string ID for the suite containing the test that is starting, intended to be unique across all suites in a run

    suiteClassName

    an optional fully qualifed Suite class name containing the test that is starting

    testName

    the name of the test that is starting

    testText

    the text of the test that is starting (may be the test name, or a suffix of the test name)

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    An optional Location that provides information indicating where in the source code an event originated.

    rerunner

    an optional String giving the fully qualified name of the class that can be used to rerun the test that is starting. (If None is passed, the test cannot be rerun.)

    payload

    an optional object that can be used to pass custom information to the reporter about the TestStarting event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  33. final case class TestSucceeded(ordinal: Ordinal, suiteName: String, suiteId: String, suiteClassName: Option[String], testName: String, testText: String, recordedEvents: IndexedSeq[RecordableEvent], duration: Option[Long] = None, formatter: Option[Formatter] = None, location: Option[Location] = None, rerunner: Option[String] = None, payload: Option[Any] = None, threadName: String = Thread.currentThread.getName, timeStamp: Long = new Date).getTime) extends Event with Product with Serializable

    Event that indicates a suite (or other entity) has completed running a test that succeeded.

    Event that indicates a suite (or other entity) has completed running a test that succeeded.

    For example, trait Suite uses TestSucceeded to report that a test method of a Suite returned normally (without throwing an Exception).

    To create instances of this class you may use the factory method. For example, given a report function named report, you could fire a TestSucceeded event like this:

    report(TestSucceeded(ordinal, userFriendlyName, suiteName, Some(thisSuite.getClass.getName), testName))

    The suite class name parameter is optional, because suites in ScalaTest are an abstraction that need not necessarily correspond to one class. Nevertheless, in most cases each suite will correspond to a class, and when it does, the fully qualified name of that class should be reported by passing a Some for suiteClassName. One use for this bit of information is JUnit integration, because the "name" provided to a JUnit org.junit.runner.Description appears to usually include a fully qualified class name by convention.

    ordinal

    an Ordinal that can be used to place this event in order in the context of other events reported during the same run

    suiteName

    a localized name identifying the suite containing the test that has succeeded, suitable for presenting to the user

    suiteId

    a string ID for the suite containing the test that has succeeded, intended to be unique across all suites in a run

    suiteClassName

    an optional fully qualifed Suite class name containing the test that has succeeded

    testName

    the name of the test that has succeeded

    testText

    the text of the test that has succeeded (may be the test name, or a suffix of the test name)

    recordedEvents

    recorded events in the test.

    duration

    an optional amount of time, in milliseconds, that was required to run the test that has succeeded

    formatter

    an optional Formatter that provides extra information that can be used by reporters in determining how to present this event to the user

    location

    An optional Location that provides information indicating where in the source code an event originated.

    rerunner

    an optional String giving the fully qualified name of the class that can be used to rerun the test that has succeeded. (If None is passed, the test cannot be rerun.)

    payload

    an optional object that can be used to pass custom information to the reporter about the TestSucceeded event

    threadName

    a name for the Thread about whose activity this event was reported

    timeStamp

    a Long indicating the time this event was reported, expressed in terms of the number of milliseconds since the standard base time known as "the epoch": January 1, 1970, 00:00:00 GMT

  34. final case class TopOfClass(className: String) extends Location with Product with Serializable

    The location in a source file where the class whose by the fully qualified name is passed as className is declared.

    The location in a source file where the class whose by the fully qualified name is passed as className is declared.

    className

    the fully qualified class name

  35. final case class TopOfMethod(className: String, methodId: String) extends Location with Product with Serializable

    The location in a source file where the method identified by the passed methodId in the class whose fully qualified name is pased as className is declared.

    The location in a source file where the method identified by the passed methodId in the class whose fully qualified name is pased as className is declared. The methodId is obtained by calling toGenericString on the java.lang.reflect.Method object representing the method.

    className

    the fully qualified class name

    methodId

    the method ID, obtained by calling toGenericString on the java.lang.reflect.Method object representing the method

Value Members

  1. object MotionToSuppress extends Formatter with Product with Serializable

    A Formatter that indicates reporters may wish to suppress reporting of an Event.

    A Formatter that indicates reporters may wish to suppress reporting of an Event. "Suppress" means that the event won't be reported to the user.

    An example is that specification-style suites, such as FunSpec, generate output that reads more like a specification. One aspect of this is that generally only a single event should be reported for each test, so that output can appear like this:

    A Stack (when newly created)
- should be empty
- should complain when popped

    ScalaTest suites should generate two events per test, a TestStarting event and either a TestSucceeded or a TestFailed event. The FunSpec trait does report both events, but passes a MotionToSuppress along with the TestStarting event. As a result, The TestStarting events have no effect on the output. Each TestSucceeded or TestFailed event, which is sent with an IndentedText formatter instead of a MotionToSuppress, will generate output, such as "- should be empty".

    Reporters may choose to ignore a MotionToSuppress. For example, an XML reporter may want to report everything about every event that is fired during a concurrent run, so that the events can be reordered later by reading the complete, but unordered, information from an XML file. In this case, the XML reporter would actually report events that were fired with a MotionToSuppress, including indicating that the report included a motion to suppress.

  2. object SeeStackDepthException extends Location with Product with Serializable

    Indicates the location should be taken from the stack depth exception, included elsewhere in the event that contained this location.

Ungrouped