Packages

  • package root
    Definition Classes
    root
  • package org
    Definition Classes
    root
  • package scalatest

    ScalaTest's main traits, classes, and other members, including members supporting ScalaTest's DSL for the Scala interpreter.

    ScalaTest's main traits, classes, and other members, including members supporting ScalaTest's DSL for the Scala interpreter.

    Definition Classes
    org
  • package compatible
    Definition Classes
    scalatest
  • package concurrent

    ScalaTest's main traits, classes, and other members, including members supporting ScalaTest's DSL for the Scala interpreter.

    ScalaTest's main traits, classes, and other members, including members supporting ScalaTest's DSL for the Scala interpreter.

    Definition Classes
    scalatest
  • package enablers
    Definition Classes
    scalatest
  • package events
    Definition Classes
    scalatest
  • package exceptions
    Definition Classes
    scalatest
  • package featurespec
    Definition Classes
    scalatest
  • package fixture

    Package fixture deprecated types.

    Package fixture deprecated types.

    Definition Classes
    scalatest
  • package flatspec
    Definition Classes
    scalatest
  • package freespec
    Definition Classes
    scalatest
  • package funspec
    Definition Classes
    scalatest
  • package funsuite
    Definition Classes
    scalatest
  • package matchers
    Definition Classes
    scalatest
  • package path
    Definition Classes
    scalatest
  • FreeSpec
  • FreeSpecLike
  • FunSpec
  • FunSpecLike
  • package prop

    Scalatest support for Property-based testing.

    Scalatest support for Property-based testing.

    Introduction to Property-based Testing

    In traditional unit testing, you write tests that describe precisely what the test will do: create these objects, wire them together, call these functions, assert on the results, and so on. It is clear and deterministic, but also limited, because it only covers the exact situations you think to test. In most cases, it is not feasible to test all of the possible combinations of data that might arise in real-world use.

    Property-based testing works the other way around. You describe properties -- rules that you expect your classes to live by -- and describe how to test those properties. The test system then generates relatively large amounts of synthetic data (with an emphasis on edge cases that tend to make things break), so that you can see if the properties hold true in these situations.

    As a result, property-based testing is scientific in the purest sense: you are stating a hypothesis about how things should work (the property), and the system is trying to falsify that hypothesis. If the tests pass, that doesn't prove the property holds, but it at least gives you some confidence that you are probably correct.

    Property-based testing is deliberately a bit random: while the edge cases get tried upfront, the system also usually generates a number of random values to try out. This makes things a bit non-deterministic -- each run will be tried with somewhat different data. To make it easier to debug, and to build regression tests, the system provides tools to re-run a failed test with precisely the same data.

    Background

    TODO: Bill should insert a brief section on QuickCheck, ScalaCheck, etc, and how this system is similar and different.

    Using Property Checks

    In order to use the tools described here, you should import this package:

    import org.scalatest._
import org.scalatest.prop._

    This library is designed to work well with the types defined in Scalactic, and some functions take types such as PosZInt as parameters. So it can also be helpful to import those with:

    import org.scalactic.anyvals._

    In order to call forAll, the function that actually performs property checks, you will need to either extend or import GeneratorDrivenPropertyChecks, like this:

    class DocExamples extends FlatSpec with Matchers with GeneratorDrivenPropertyChecks {

    There's nothing special about FlatSpec, though -- you may use any of ScalaTest's styles with property checks. GeneratorDrivenPropertyChecks extends CommonGenerators, so it also provides access to the many utilities found there.

    What Does a Property Look Like?

    Let's check a simple property of Strings -- that if you concatenate a String to itself, its length will be doubled:

    "Strings" should "have the correct length when doubled" in {
  forAll { (s: String) =>
    val s2 = s * 2
    s2.length should equal (s.length * 2)
  }
}

    (Note that the examples here are all using the FlatSpec style, but will work the same way with any of ScalaTest's styles.)

    As the name of the tests suggests, the property we are testing is the length of a String that has been doubled.

    The test begins with forAll. This is usually the way you'll want to begin property checks, and that line can be read as, "For all Strings, the following should be true".

    The test harness will generate a number of Strings, with various contents and lengths. For each one, we compute s * 2. (* is a function on String, which appends the String to itself as many times as you specify.) And then we check that the length of the doubled String is twice the length of the original one.

    Using Specific Generators

    Let's try a more general version of this test, multiplying arbitrary Strings by arbitrary multipliers:

    "Strings" should "have the correct length when multiplied" in {
  forAll { (s: String, n: PosZInt) =>
    val s2 = s * n.value
    s2.length should equal (s.length * n.value)
  }
}

    Again, you can read the first line of the test as "For all Strings, and all non-negative Integers, the following should be true". (PosZInt is a type defined in Scalactic, which can be any positive integer, including zero. It is appropriate to use here, since multiplying a String by a negative number doesn't make sense.)

    This intuitively makes sense, but when we try to run it, we get a JVM Out of Memory error! Why? Because the test system tries to test with the "edge cases" first, and one of the more important edge cases is Int.MaxValue. It is trying to multiply a String by that, which is far larger than the memory of even a big computer, and crashing.

    So we want to constrain our test to sane values of n, so that it doesn't crash. We can do this by using more specific Generators.

    When we write a forAll test like the above, ScalaTest has to generate the values to be tested -- the semi-random Strings, Ints and other types that you are testing. It does this by calling on an implicit Generator for the desired type. The Generator generates values to test, starting with the edge cases and then moving on to randomly-selected values.

    ScalaTest has built-in Generators for many major types, including String and PosZInt, but these Generators are generic: they will try any value, including values that can break your test, as shown above. But it also provides tools to let you be more specific.

    Here is the fixed version of the above test:

    "Strings" should "have the correct length when multiplied" in {
  forAll(strings, posZIntsBetween(0, 1000))
  { (s: String, n: PosZInt) =>
    val s2 = s * n.value
    s2.length should equal (s.length * n.value)
  }
}

    This is using a variant of forAll, which lets you specify the Generators to use instead of just picking the implicit one. CommonGenerators.strings is the built-in Generator for Strings, the same one you were getting implicitly. (The other built-ins can be found in CommonGenerators. They are mixed into GeneratorDrivenPropertyChecks, so they are readily available.)

    But CommonGenerators.posZIntsBetween is a function that creates a Generator that selects from the given values. In this case, it will create a Generator that only creates numbers from 0 to 1000 -- small enough to not blow up our computer's memory. If you try this test, this runs correctly.

    The moral of the story is that, while using the built-in Generators is very convenient, and works most of the time, you should think about the data you are trying to test, and pick or create a more-specific Generator when the test calls for it.

    CommonGenerators contains many functions that are helpful in common cases. In particular:

    • xxsBetween (where xxs might be Int, Long, Float or most other significant numeric types) gives you a value of the desired type in the given range, as in the posZIntsBetween() example above.
    • CommonGenerators.specificValue and CommonGenerators.specificValues create Generators that produce either one specific value every time, or one of several values randomly. This is useful for enumerations and types that behave like enumerations.
    • CommonGenerators.evenly and CommonGenerators.frequency create higher-level Generators that call other Generators, either more or less equally or with a distribution you define.

    Testing Your Own Types

    Testing the built-in types isn't very interesting, though. Usually, you have your own types that you want to check the properties of. So let's build up an example piece by piece.

    Say you have this simple type:

    sealed trait Shape {
  def area: Double
}
case class Rectangle(width: Int, height: Int) extends Shape {
  require(width > 0)
  require(height > 0)
  def area: Double = width * height
}

    Let's confirm a nice straightforward property that is surely true: that the area is greater than zero:

    "Rectangles" should "have a positive area" in {
   forAll { (w: PosInt, h: PosInt) =>
     val rect = Rectangle(w, h)
     rect.area should be > 0.0
   }
 }

    Note that, even though our class takes ordinary Ints as parameters (and checks the values at runtime), it is actually easier to generate the legal values using Scalactic's PosInt type.

    This should work, right? Actually, it doesn't -- if we run it a few times, we quickly hit an error!

    [info] Rectangles
[info] - should have a positive area *** FAILED ***
[info]   GeneratorDrivenPropertyCheckFailedException was thrown during property evaluation.
[info]    (DocExamples.scala:42)
[info]     Falsified after 2 successful property evaluations.
[info]     Location: (DocExamples.scala:42)
[info]     Occurred when passed generated values (
[info]       None = PosInt(399455539),
[info]       None = PosInt(703518968)
[info]     )
[info]     Init Seed: 1568878346200

    TODO: fix the above error to reflect the better errors we should get when we merge in the code being forward-ported from 3.0.5.

    Looking at it, we can see that the numbers being used are pretty large. What happens when we multiply them together?

    scala> 399455539 * 703518968
res0: Int = -2046258840

    We're hitting an Int overflow problem here: the numbers are too big to multiply together and still get an Int. So we have to fix our area function:

    case class Rectangle(width: Int, height: Int) extends Shape {
  require(width > 0)
  require(height > 0)
  def area: Double = width.toLong * height.toLong
}

    Now, when we run our property check, it consistently passes. Excellent -- we've caught a bug, because ScalaTest tried sufficiently large numbers.

    Composing Your Own Generators

    Doing things as shown above works, but having to generate the parameters and construct a Rectangle every time is a nuisance. What we really want is to create our own Generator that just hands us Rectangles, the same way we can do for PosInt. Fortunately, this is easy.

    Generators can be composed in for comprehensions. So we can create our own Generator for Rectangle like this:

    implicit val rectGenerator = for {
  w <- posInts
  h <- posInts
}
  yield Rectangle(w, h)

    Taking that line by line:

    w <- posInts

    CommonGenerators.posInts is the built-in Generator for positive Ints. So this line puts a randomly-generated positive Int in w, and

    h <- posInts

    this line puts another one in h. Finally, this line:

    yield Rectangle(w, h)

    combines w and h to make a Rectangle.

    That's pretty much all you need in order to build any normal case class -- just build it out of the Generators for the type of each field. (And if the fields are complex data structures themselves, build Generators for them the same way, until you are just using primitives.)

    Now, our property check becomes simpler:

    "Generated Rectangles" should "have a positive area" in {
   forAll { (rect: Rectangle) =>
     rect.area should be > 0.0
   }
 }

    That's about as close to plain English as we can reasonably hope for!

    Filtering Values with whenever()

    Sometimes, not all of your generated values make sense for the property you want to check -- you know (via external information) that some of these values will never come up. In cases like this, you can create a custom Generator that only creates the values you do want, but it's often easier to just use Whenever.whenever. (Whenever is mixed into GeneratorDrivenPropertyChecks, so this is available when you need it.)

    The Whenever.whenever function can be used inside of GeneratorDrivenPropertyChecks.forAll. It says that only the filtered values should be used, and anything else should be discarded. For example, look at this property:

    "Fractions" should "get smaller when squared" in {
  forAll { (n: Float) =>
    whenever(n > 0 && n < 1) {
      (n * n) should be < n
    }
  }
}

    We are testing a property of numbers less than 1, so we filter away everything that is not the numbers we want. This property check succeeds, because we've screened out the values that would make it fail.

    Discard Limits

    You shouldn't push Whenever.whenever too far, though. This system is all about trying random data, but if too much of the random data simply isn't usable, you can't get valid answers, and the system tracks that.

    For example, consider this apparently-reasonable test:

    "Space Chars" should "not also be letters" in {
  forAll { (c: Char) =>
    whenever (c.isSpaceChar) {
      assert(!c.isLetter)
    }
  }
}

    Although the property is true, this test will fail with an error like this:

    [info] Lowercase Chars
[info] - should upper-case correctly *** FAILED ***
[info]   Gave up after 0 successful property evaluations. 49 evaluations were discarded.
[info]   Init Seed: 1568855247784

    Because the vast majority of Chars are not spaces, nearly all of the generated values are being discarded. As a result, the system gives up after a while. In cases like this, you usually should write a custom Generator instead.

    The proportion of how many discards to permit, relative to the number of successful checks, is configuration-controllable. See GeneratorDrivenPropertyChecks for more details.

    Randomization

    The point of Generator is to create pseudo-random values for checking properties. But it turns out to be very inconvenient if those values are actually random -- that would mean that, when a property check fails occasionally, you have no good way to invoke that specific set of circumstances again for debugging. We want "randomness", but we also want it to be deterministic, and reproducible when you need it.

    To support this, all "randomness" in ScalaTest's property checking system uses the Randomizer class. You start by creating a Randomizer using an initial seed value, and call that to get your "random" value. Each call to a Randomizer function returns a new Randomizer, which you should use to fetch the next value.

    GeneratorDrivenPropertyChecks.forAll uses Randomizer under the hood: each time you run a forAll-based test, it will automatically create a new Randomizer, which by default is seeded based on the current system time. You can override this, as discussed below.

    Since Randomizer is actually deterministic (the "random" values are unobvious, but will always be the same given the same initial seed), this means that re-running a test with the same seed will produce the same values.

    If you need random data for your own Generators and property checks, you should use Randomizer in the same way; that way, your tests will also be re-runnable, when needed for debugging.

    Debugging, and Re-running a Failed Property Check

    In Testing Your Own Types above, we found to our surprise that the property check failed with this error:

    [info] Rectangles
[info] - should have a positive area *** FAILED ***
[info]   GeneratorDrivenPropertyCheckFailedException was thrown during property evaluation.
[info]    (DocExamples.scala:42)
[info]     Falsified after 2 successful property evaluations.
[info]     Location: (DocExamples.scala:42)
[info]     Occurred when passed generated values (
[info]       None = PosInt(399455539),
[info]       None = PosInt(703518968)
[info]     )
[info]     Init Seed: 1568878346200

    There must be a bug here -- but once we've fixed it, how can we make sure that we are re-testing exactly the same case that failed?

    This is where the pseudo-random nature of Randomizer comes in, and why it is so important to use it consistently. So long as all of our "random" data comes from that, then all we need to do is re-run with the same seed.

    That's why the Init Seed shown in the message above is crucial. We can re-use that seed -- and therefore get exactly the same "random" data -- by using the -S flag to ScalaTest.

    So you can run this command in sbt to re-run exactly the same property check:

    testOnly *DocExamples -- -z "have a positive area" -S 1568878346200

    Taking that apart:

    • testOnly *DocExamples says that we only want to run suites whose paths end with DocExamples
    • -z "have a positive area" says to only run tests whose names include that string.
    • -S 1568878346200 says to run all tests with a "random" seed of 1568878346200

    By combining these flags, you can re-run exactly the property check you need, with the right random seed to make sure you are re-creating the failed test. You should get exactly the same failure over and over until you fix the bug, and then you can confirm your fix with confidence.

    Configuration

    In general, forAll() works well out of the box. But you can tune several configuration parameters when needed. See GeneratorDrivenPropertyChecks for info on how to set configuration parameters for your test.

    Table-Driven Properties

    Sometimes, you want something in between traditional hard-coded unit tests and Generator-driven, randomized tests. Instead, you sometimes want to check your properties against a specific set of inputs.

    (This is particularly useful for regression tests, when you have found certain inputs that have caused problems in the past, and want to make sure that they get consistently re-tested.)

    ScalaTest supports these, by mixing in TableDrivenPropertyChecks. See the documentation for that class for the full details.

    Definition Classes
    scalatest
  • package refspec
    Definition Classes
    scalatest
  • package tagobjects
    Definition Classes
    scalatest
  • package tags
    Definition Classes
    scalatest
  • package time
    Definition Classes
    scalatest
  • package tools
    Definition Classes
    scalatest
  • package words
    Definition Classes
    scalatest

package path

Type Members

  1. class FreeSpec extends FreeSpecLike

    A sister class to org.scalatest.FreeSpec that isolates tests by running each test in its own instance of the test class, and for each test, only executing the path leading to that test.

    A sister class to org.scalatest.FreeSpec that isolates tests by running each test in its own instance of the test class, and for each test, only executing the path leading to that test.

    Class path.FreeSpec behaves similarly to class org.scalatest.FreeSpec, except that tests are isolated based on their path. The purpose of path.FreeSpec is to facilitate writing specification-style tests for mutable objects in a clear, boilerpate-free way. To test mutable objects, you need to mutate them. Using a path class, you can make a statement in text, then implement that statement in code (including mutating state), and nest and combine these test/code pairs in any way you wish. Each test will only see the side effects of code that is in blocks that enclose the test. Here's an example:

    import org.scalatest.path
import org.scalatest.matchers.Matchers
import scala.collection.mutable.ListBuffer

class ExampleSpec extends path.FreeSpec with Matchers {

  "A ListBuffer" - {

    val buf = ListBuffer.empty[Int] // This implements "A ListBuffer"

    "should be empty when created" in {

      // This test sees:
      //   val buf = ListBuffer.empty[Int]
      // So buf is: ListBuffer()

      buf should be ('empty)
    }

    "when 1 is appended" - {

      buf += 1 // This implements "when 1 is appended", etc...

      "should contain 1" in {

        // This test sees:
        //   val buf = ListBuffer.empty[Int]
        //   buf += 1
        // So buf is: ListBuffer(1)

        buf.remove(0) should equal (1)
        buf should be ('empty)
      }

      "when 2 is appended" - {

        buf += 2

        "should contain 1 and 2" in {

          // This test sees:
          //   val buf = ListBuffer.empty[Int]
          //   buf += 1
          //   buf += 2
          // So buf is: ListBuffer(1, 2)

          buf.remove(0) should equal (1)
          buf.remove(0) should equal (2)
          buf should be ('empty)
        }

        "when 2 is removed" - {

          buf -= 2

          "should contain only 1 again" in {

            // This test sees:
            //   val buf = ListBuffer.empty[Int]
            //   buf += 1
            //   buf += 2
            //   buf -= 2
            // So buf is: ListBuffer(1)

            buf.remove(0) should equal (1)
            buf should be ('empty)
          }
        }

        "when 3 is appended" - {

          buf += 3

          "should contain 1, 2, and 3" in {

            // This test sees:
            //   val buf = ListBuffer.empty[Int]
            //   buf += 1
            //   buf += 2
            //   buf += 3
            // So buf is: ListBuffer(1, 2, 3)

            buf.remove(0) should equal (1)
            buf.remove(0) should equal (2)
            buf.remove(0) should equal (3)
            buf should be ('empty)
          }
        }
      }

      "when 88 is appended" - {

        buf += 88

        "should contain 1 and 88" in {

          // This test sees:
          //   val buf = ListBuffer.empty[Int]
          //   buf += 1
          //   buf += 88
          // So buf is: ListBuffer(1, 88)

          buf.remove(0) should equal (1)
          buf.remove(0) should equal (88)
          buf should be ('empty)
        }
      }
    }

    "should have size 0 when created" in {

      // This test sees:
      //   val buf = ListBuffer.empty[Int]
      // So buf is: ListBuffer()

      buf should have size 0
    }
  }
}

    Note that the above class is organized by writing a bit of specification text that opens a new block followed by, at the top of the new block, some code that "implements" or "performs" what is described in the text. This is repeated as the mutable object (here, a ListBuffer), is prepared for the enclosed tests. For example:

    "A ListBuffer" - {
  val buf = ListBuffer.empty[Int]

    Or:

    "when 2 is appended" - {
  buf += 2

    Note also that although each test mutates the ListBuffer, none of the other tests observe those side effects:

    "should contain 1" in {

  buf.remove(0) should equal (1)
  // ...
}

"when 2 is appended" - {

  buf += 2

  "should contain 1 and 2" in {

    // This test does not see the buf.remove(0) from the previous test,
    // so the first element in the ListBuffer is again 1
    buf.remove(0) should equal (1)
    buf.remove(0) should equal (2)

    This kind of isolation of tests from each other is a consequence of running each test in its own instance of the test class, and can also be achieved by simply mixing OneInstancePerTest into a regular org.scalatest.FreeSpec. However, path.FreeSpec takes isolation one step further: a test in a path.FreeSpec does not observe side effects performed outside tests in earlier blocks that do not enclose it. Here's an example:

    "when 2 is removed" - {

  buf -= 2

  // ...
}

"when 3 is appended" - {

  buf += 3

  "should contain 1, 2, and 3" in {

    // This test does not see the buf -= 2 from the earlier "when 2 is removed" block,
    // because that block does not enclose this test, so the second element in the
    // ListBuffer is still 2
    buf.remove(0) should equal (1)
    buf.remove(0) should equal (2)
    buf.remove(0) should equal (3)

    Running the full ExampleSpec, shown above, in the Scala interpeter would give you:

    scala> import org.scalatest._
import org.scalatest._

scala> run(new ExampleSpec)
ExampleSpec:
A ListBuffer
- should be empty when created
  when 1 is appended
  - should contain 1
    when 2 is appended
    - should contain 1 and 2
      when 2 is removed
      - should contain only 1 again
      when 3 is appended
      - should contain 1, 2, and 3
    when 88 is appended
    - should contain 1 and 88
- should have size 0 when created

    Note: class path.FreeSpec's approach to isolation was inspired in part by the specsy framework, written by Esko Luontola.

    Shared fixtures

    A test fixture is objects or other artifacts (such as files, sockets, database connections, etc.) used by tests to do their work. If a fixture is used by only one test, then the definitions of the fixture objects can be local to the method. If multiple tests need to share an immutable fixture, you can simply assign them to instance variables. If multiple tests need to share mutable fixture objects or vars, there's one and only one way to do it in a path.FreeSpec: place the mutable objects lexically before the test. Any mutations needed by the test must be placed lexically before and/or after the test. As used here, "Lexically before" means that the code needs to be executed during construction of that test's instance of the test class to reach the test (or put another way, the code is along the "path to the test.") "Lexically after" means that the code needs to be executed to exit the constructor after the test has been executed.

    The reason lexical placement is the one and only one way to share fixtures in a path.FreeSpec is because all of its lifecycle methods are overridden and declared final. Thus you can't mix in BeforeAndAfter or BeforeAndAfterEach, because both override runTest, which is final in a path.FreeSpec. You also can't override withFixture, because path.FreeSpec extends Suite not TestSuite, where withFixture is defined. In short:

    In a path.FreeSpec, if you need some code to execute before a test, place that code lexically before the test. If you need some code to execute after a test, place that code lexically after the test.

    The reason the life cycle methods are final, by the way, is to prevent users from attempting to combine a path.FreeSpec's approach to isolation with other ways ScalaTest provides to share fixtures or execute tests, because doing so could make the resulting test code hard to reason about. A path.FreeSpec's execution model is a bit magical, but because it executes in one and only one way, users should be able to reason about the code. To help you visualize how a path.FreeSpec is executed, consider the following variant of ExampleSpec that includes print statements:

    import org.scalatest.path
import org.scalatest.matchers.Matchers
import scala.collection.mutable.ListBuffer

class ExampleSpec extends path.FreeSpec with Matchers {

  println("Start of: ExampleSpec")
  "A ListBuffer" - {

    println("Start of: A ListBuffer")
    val buf = ListBuffer.empty[Int]

    "should be empty when created" in {

      println("In test: should be empty when created; buf is: " + buf)
      buf should be ('empty)
    }

    "when 1 is appended" - {

      println("Start of: when 1 is appended")
      buf += 1

      "should contain 1" in {

        println("In test: should contain 1; buf is: " + buf)
        buf.remove(0) should equal (1)
        buf should be ('empty)
      }

      "when 2 is appended" - {

        println("Start of: when 2 is appended")
        buf += 2

        "should contain 1 and 2" in {

          println("In test: should contain 1 and 2; buf is: " + buf)
          buf.remove(0) should equal (1)
          buf.remove(0) should equal (2)
          buf should be ('empty)
        }

        "when 2 is removed" - {

          println("Start of: when 2 is removed")
          buf -= 2

          "should contain only 1 again" in {

            println("In test: should contain only 1 again; buf is: " + buf)
            buf.remove(0) should equal (1)
            buf should be ('empty)
          }

          println("End of: when 2 is removed")
        }

        "when 3 is appended" - {

          println("Start of: when 3 is appended")
          buf += 3

          "should contain 1, 2, and 3" in {

            println("In test: should contain 1, 2, and 3; buf is: " + buf)
            buf.remove(0) should equal (1)
            buf.remove(0) should equal (2)
            buf.remove(0) should equal (3)
            buf should be ('empty)
          }
          println("End of: when 3 is appended")
        }

        println("End of: when 2 is appended")
      }

      "when 88 is appended" - {

        println("Start of: when 88 is appended")
        buf += 88

        "should contain 1 and 88" in {

          println("In test: should contain 1 and 88; buf is: " + buf)
          buf.remove(0) should equal (1)
          buf.remove(0) should equal (88)
          buf should be ('empty)
        }

        println("End of: when 88 is appended")
      }

      println("End of: when 1 is appended")
    }

    "should have size 0 when created" in {

      println("In test: should have size 0 when created; buf is: " + buf)
      buf should have size 0
    }

    println("End of: A ListBuffer")
  }
  println("End of: ExampleSpec")
  println()
}

    Running the above version of ExampleSpec in the Scala interpreter will give you output similar to:

    scala> import org.scalatest._
import org.scalatest._

scala> run(new ExampleSpec)
ExampleSpec:
Start of: ExampleSpec
Start of: A ListBuffer
In test: should be empty when created; buf is: ListBuffer()
End of: A ListBuffer
End of: ExampleSpec

Start of: ExampleSpec
Start of: A ListBuffer
Start of: when 1 is appended
In test: should contain 1; buf is: ListBuffer(1)
ExampleSpec:
End of: when 1 is appended
End of: A ListBuffer
End of: ExampleSpec

Start of: ExampleSpec
Start of: A ListBuffer
Start of: when 1 is appended
Start of: when 2 is appended
In test: should contain 1 and 2; buf is: ListBuffer(1, 2)
End of: when 2 is appended
End of: when 1 is appended
End of: A ListBuffer
End of: ExampleSpec

Start of: ExampleSpec
Start of: A ListBuffer
Start of: when 1 is appended
Start of: when 2 is appended
Start of: when 2 is removed
In test: should contain only 1 again; buf is: ListBuffer(1)
End of: when 2 is removed
End of: when 2 is appended
End of: when 1 is appended
End of: A ListBuffer
End of: ExampleSpec

Start of: ExampleSpec
Start of: A ListBuffer
Start of: when 1 is appended
Start of: when 2 is appended
Start of: when 3 is appended
In test: should contain 1, 2, and 3; buf is: ListBuffer(1, 2, 3)
End of: when 3 is appended
End of: when 2 is appended
End of: when 1 is appended
End of: A ListBuffer
End of: ExampleSpec

Start of: ExampleSpec
Start of: A ListBuffer
Start of: when 1 is appended
Start of: when 88 is appended
In test: should contain 1 and 88; buf is: ListBuffer(1, 88)
End of: when 88 is appended
End of: when 1 is appended
End of: A ListBuffer
End of: ExampleSpec

Start of: ExampleSpec
Start of: A ListBuffer
In test: should have size 0 when created; buf is: ListBuffer()
End of: A ListBuffer
End of: ExampleSpec

A ListBuffer
- should be empty when created
  when 1 is appended
  - should contain 1
    when 2 is appended
    - should contain 1 and 2
      when 2 is removed
      - should contain only 1 again
      when 3 is appended
      - should contain 1, 2, and 3
    when 88 is appended
    - should contain 1 and 88
- should have size 0 when created

    Note that each test is executed in order of appearance in the path.FreeSpec, and that only those println statements residing in blocks that enclose the test being run are executed. Any println statements in blocks that do not form the "path" to a test are not executed in the instance of the class that executes that test.

    How it executes

    To provide its special brand of test isolation, path.FreeSpec executes quite differently from its sister class in org.scalatest. An org.scalatest.FreeSpec registers tests during construction and executes them when run is invoked. An org.scalatest.path.FreeSpec, by contrast, runs each test in its own instance while that instance is being constructed. During construction, it registers not the tests to run, but the results of running those tests. When run is invoked on a path.FreeSpec, it reports the registered results and does not run the tests again. If run is invoked a second or third time, in fact, a path.FreeSpec will each time report the same results registered during construction. If you want to run the tests of a path.FreeSpec anew, you'll need to create a new instance and invoke run on that.

    A path.FreeSpec will create one instance for each "leaf" node it contains. The main kind of leaf node is a test, such as:

    // One instance will be created for each test
"should be empty when created" in {
  buf should be ('empty)
}

    However, an empty scope (a scope that contains no tests or nested scopes) is also a leaf node:

     // One instance will be created for each empty scope
"when 99 is added" - {
  // A scope is "empty" and therefore a leaf node if it has no
  // tests or nested scopes, though it may have other code (which
  // will be executed in the instance created for that leaf node)
  buf += 99
}

    The tests will be executed sequentially, in the order of appearance. The first test (or empty scope, if that is first) will be executed when a class that mixes in path.FreeSpec is instantiated. Only the first test will be executed during this initial instance, and of course, only the path to that test. Then, the first time the client uses the initial instance (by invoking one of run, expectedTestsCount, tags, or testNames on the instance), the initial instance will, before doing anything else, ensure that any remaining tests are executed, each in its own instance.

    To ensure that the correct path is taken in each instance, and to register its test results, the initial path.FreeSpec instance must communicate with the other instances it creates for running any subsequent leaf nodes. It does so by setting a thread-local variable prior to creating each instance (a technique suggested by Esko Luontola). Each instance of path.FreeSpec checks the thread-local variable. If the thread-local is not set, it knows it is an initial instance and therefore executes every block it encounters until it discovers, and executes the first test (or empty scope, if that's the first leaf node). It then discovers, but does not execute the next leaf node, or discovers there are no other leaf nodes remaining to execute. It communicates the path to the next leaf node, if any, and the result of running the test it did execute, if any, back to the initial instance. The initial instance repeats this process until all leaf nodes have been executed and all test results registered.

    Ignored tests

    You mark a test as ignored in an org.scalatest.path.FreeSpec in the same manner as in an org.scalatest.FreeSpec. Please see the Ignored tests section in its documentation for more information.

    Note that a separate instance will be created for an ignored test, and the path to the ignored test will be executed in that instance, but the test function itself will not be executed. Instead, a TestIgnored event will be fired.

    Informers

    You output information using Informers in an org.scalatest.path.FreeSpec in the same manner as in an org.scalatest.FreeSpec. Please see the Informers section in its documentation for more information.

    Pending tests

    You mark a test as pending in an org.scalatest.path.FreeSpec in the same manner as in an org.scalatest.FreeSpec. Please see the Pending tests section in its documentation for more information.

    Note that a separate instance will be created for a pending test, and the path to the ignored test will be executed in that instance, as well as the test function (up until it completes abruptly with a TestPendingException).

    Tagging tests

    You can place tests into groups by tagging them in an org.scalatest.path.FreeSpec in the same manner as in an org.scalatest.FreeSpec. Please see the Tagging tests section in its documentation for more information.

    Note that one difference between this class and its sister class org.scalatest.FreeSpec is that because tests are executed at construction time, rather than each time run is invoked, an org.scalatest.path.FreeSpec will always execute all non-ignored tests. When run is invoked on a path.FreeSpec, if some tests are excluded based on tags, the registered results of running those tests will not be reported. (But those tests will have already run and the results registered.) By contrast, because an org.scalatest.FreeSpec only executes tests after run has been called, and at that time the tags to include and exclude are known, only tests selected by the tags will be executed.

    In short, in an org.scalatest.FreeSpec, tests not selected by the tags to include and exclude specified for the run (via the Filter passed to run) will not be executed. In an org.scalatest.path.FreeSpec, by contrast, all non-ignored tests will be executed, each during the construction of its own instance, and tests not selected by the tags to include and exclude specified for a run will not be reported. (One upshot of this is that if you have tests that you want to tag as being slow so you can sometimes exclude them during a run, you probably don't want to put them in a path.FreeSpec. Because in a path.Freespec the slow tests will be run regardless, with only their registered results not being reported if you exclude slow tests during a run.)

    Shared tests

    You can factor out shared tests in an org.scalatest.path.FreeSpec in the same manner as in an org.scalatest.FreeSpec. Please see the Shared tests section in its documentation for more information.

    Nested suites

    Nested suites are not allowed in a path.FreeSpec. Because a path.FreeSpec executes tests eagerly at construction time, registering the results of those test runs and reporting them later when run is invoked, the order of nested suites versus test runs would be different in a org.scalatest.path.FreeSpec than in an org.scalatest.FreeSpec. In org.scalatest.FreeSpec's implementation of run, nested suites are executed then tests are executed. A org.scalatest.path.FreeSpec with nested suites would execute these in the opposite order: first tests then nested suites. To help make path.FreeSpec code easier to reason about by giving readers of one less difference to think about, nested suites are not allowed. If you want to add nested suites to a path.FreeSpec, you can instead wrap them all in a Suites object. They will be executed in the order of appearance (unless a Distributor is passed, in which case they will execute in parallel).

    Durations

    Many ScalaTest events include a duration that indicates how long the event being reported took to execute. For example, a TestSucceeded event provides a duration indicating how long it took for that test to execute. A SuiteCompleted event provides a duration indicating how long it took for that entire suite of tests to execute.

    In the test completion events fired by a path.FreeSpec (TestSucceeded, TestFailed, or TestPending), the durations reported refer to the time it took for the tests to run. This time is registered with the test results and reported along with the test results each time run is invoked. By contrast, the suite completion events fired for a path.FreeSpec represent the amount of time it took to report the registered results. (These events are not fired by path.FreeSpec, but instead by the entity that invokes run on the path.FreeSpec.) As a result, the total time for running the tests of a path.FreeSpec, calculated by summing the durations of all the individual test completion events, may be greater than the duration reported for executing the entire suite.

  2. trait FreeSpecLike extends Suite with OneInstancePerTest with Informing with Notifying with Alerting with Documenting

    Implementation trait for class path.FreeSpec, which is a sister class to org.scalatest.FreeSpec that isolates tests by running each test in its own instance of the test class, and for each test, only executing the path leading to that test.

    Implementation trait for class path.FreeSpec, which is a sister class to org.scalatest.FreeSpec that isolates tests by running each test in its own instance of the test class, and for each test, only executing the path leading to that test.

    path.FreeSpec is a class, not a trait, to minimize compile time given there is a slight compiler overhead to mixing in traits compared to extending classes. If you need to mix the behavior of path.FreeSpec into some other class, you can use this trait instead, because class path.FreeSpec does nothing more than extend this trait and add a nice toString implementation.

    See the documentation of the class for a detailed overview of path.FreeSpec.

  3. class FunSpec extends FunSpecLike

    A sister class to org.scalatest.FunSpec that isolates tests by running each test in its own instance of the test class, and for each test, only executing the path leading to that test.

    A sister class to org.scalatest.FunSpec that isolates tests by running each test in its own instance of the test class, and for each test, only executing the path leading to that test.

    Class path.FunSpec behaves similarly to class org.scalatest.FunSpec, except that tests are isolated based on their path. The purpose of path.FunSpec is to facilitate writing specification-style tests for mutable objects in a clear, boilerpate-free way. To test mutable objects, you need to mutate them. Using a path class, you can make a statement in text, then implement that statement in code (including mutating state), and nest and combine these test/code pairs in any way you wish. Each test will only see the side effects of code that is in blocks that enclose the test. Here's an example:

    import org.scalatest.path
import org.scalatest.matchers.Matchers
import scala.collection.mutable.ListBuffer

class ExampleSpec extends path.FunSpec with Matchers {

  describe("A ListBuffer") {

    val buf = ListBuffer.empty[Int] // This implements "A ListBuffer"

    it("should be empty when created") {

      // This test sees:
      //   val buf = ListBuffer.empty[Int]
      // So buf is: ListBuffer()

      buf should be ('empty)
    }

    describe("when 1 is appended") {

      buf += 1 // This implements "when 1 is appended", etc...

      it("should contain 1") {

        // This test sees:
        //   val buf = ListBuffer.empty[Int]
        //   buf += 1
        // So buf is: ListBuffer(1)

        buf.remove(0) should equal (1)
        buf should be ('empty)
      }

      describe("when 2 is appended") {

        buf += 2

        it("should contain 1 and 2") {

          // This test sees:
          //   val buf = ListBuffer.empty[Int]
          //   buf += 1
          //   buf += 2
          // So buf is: ListBuffer(1, 2)

          buf.remove(0) should equal (1)
          buf.remove(0) should equal (2)
          buf should be ('empty)
        }

        describe("when 2 is removed") {

          buf -= 2

          it("should contain only 1 again") {

            // This test sees:
            //   val buf = ListBuffer.empty[Int]
            //   buf += 1
            //   buf += 2
            //   buf -= 2
            // So buf is: ListBuffer(1)

            buf.remove(0) should equal (1)
            buf should be ('empty)
          }
        }

        describe("when 3 is appended") {

          buf += 3

          it("should contain 1, 2, and 3") {

            // This test sees:
            //   val buf = ListBuffer.empty[Int]
            //   buf += 1
            //   buf += 2
            //   buf += 3
            // So buf is: ListBuffer(1, 2, 3)

            buf.remove(0) should equal (1)
            buf.remove(0) should equal (2)
            buf.remove(0) should equal (3)
            buf should be ('empty)
          }
        }
      }

      describe("when 88 is appended") {

        buf += 88

        it("should contain 1 and 88") {

          // This test sees:
          //   val buf = ListBuffer.empty[Int]
          //   buf += 1
          //   buf += 88
          // So buf is: ListBuffer(1, 88)

          buf.remove(0) should equal (1)
          buf.remove(0) should equal (88)
          buf should be ('empty)
        }
      }
    }

    it("should have size 0 when created") {

      // This test sees:
      //   val buf = ListBuffer.empty[Int]
      // So buf is: ListBuffer()

      buf should have size 0
    }
  }
}

    Note that the above class is organized by writing a bit of specification text that opens a new block followed by, at the top of the new block, some code that "implements" or "performs" what is described in the text. This is repeated as the mutable object (here, a ListBuffer), is prepared for the enclosed tests. For example:

    describe("A ListBuffer") {
  val buf = ListBuffer.empty[Int]

    Or:

    describe("when 2 is appended") {
  buf += 2

    Note also that although each test mutates the ListBuffer, none of the other tests observe those side effects:

    it("should contain 1") {

  buf.remove(0) should equal (1)
  // ...
}

describe("when 2 is appended") {

  buf += 2

  it("should contain 1 and 2") {

    // This test does not see the buf.remove(0) from the previous test,
    // so the first element in the ListBuffer is again 1
    buf.remove(0) should equal (1)
    buf.remove(0) should equal (2)

    This kind of isolation of tests from each other is a consequence of running each test in its own instance of the test class, and can also be achieved by simply mixing OneInstancePerTest into a regular org.scalatest.FunSpec. However, path.FunSpec takes isolation one step further: a test in a path.FunSpec does not observe side effects performed outside tests in earlier blocks that do not enclose it. Here's an example:

    describe("when 2 is removed") {

  buf -= 2

  // ...
}

describe("when 3 is appended") {

  buf += 3

  it("should contain 1, 2, and 3") {

    // This test does not see the buf -= 2 from the earlier "when 2 is removed" block,
    // because that block does not enclose this test, so the second element in the
    // ListBuffer is still 2
    buf.remove(0) should equal (1)
    buf.remove(0) should equal (2)
    buf.remove(0) should equal (3)

    Running the full ExampleSpec, shown above, in the Scala interpeter would give you:

    scala> import org.scalatest._
import org.scalatest._

scala> run(new ExampleSpec)
ExampleSpec:
A ListBuffer
- should be empty when created
  when 1 is appended
  - should contain 1
    when 2 is appended
    - should contain 1 and 2
      when 2 is removed
      - should contain only 1 again
      when 3 is appended
      - should contain 1, 2, and 3
    when 88 is appended
    - should contain 1 and 88
- should have size 0 when created

    Note: class path.FunSpec's approach to isolation was inspired in part by the specsy framework, written by Esko Luontola.

    Shared fixtures

    A test fixture is objects or other artifacts (such as files, sockets, database connections, etc.) used by tests to do their work. If a fixture is used by only one test, then the definitions of the fixture objects can be local to the method. If multiple tests need to share an immutable fixture, you can simply assign them to instance variables. If multiple tests need to share mutable fixture objects or vars, there's one and only one way to do it in a path.FunSpec: place the mutable objects lexically before the test. Any mutations needed by the test must be placed lexically before and/or after the test. As used here, "Lexically before" means that the code needs to be executed during construction of that test's instance of the test class to reach the test (or put another way, the code is along the "path to the test.") "Lexically after" means that the code needs to be executed to exit the constructor after the test has been executed.

    The reason lexical placement is the one and only one way to share fixtures in a path.FunSpec is because all of its lifecycle methods are overridden and declared final. Thus you can't mix in BeforeAndAfter or BeforeAndAfterEach, because both override runTest, which is final in a path.FunSpec. You also can't override withFixture, because path.FreeSpec extends Suite not TestSuite, where withFixture is defined. In short:

    In a path.FunSpec, if you need some code to execute before a test, place that code lexically before the test. If you need some code to execute after a test, place that code lexically after the test.

    The reason the life cycle methods are final, by the way, is to prevent users from attempting to combine a path.FunSpec's approach to isolation with other ways ScalaTest provides to share fixtures or execute tests, because doing so could make the resulting test code hard to reason about. A path.FunSpec's execution model is a bit magical, but because it executes in one and only one way, users should be able to reason about the code. To help you visualize how a path.FunSpec is executed, consider the following variant of ExampleSpec that includes print statements:

    import org.scalatest.path
import org.scalatest.matchers.Matchers
import scala.collection.mutable.ListBuffer

class ExampleSpec extends path.FunSpec with Matchers {

  println("Start of: ExampleSpec")
  describe("A ListBuffer") {

    println("Start of: A ListBuffer")
    val buf = ListBuffer.empty[Int]

    it("should be empty when created") {

      println("In test: should be empty when created; buf is: " + buf)
      buf should be ('empty)
    }

    describe("when 1 is appended") {

      println("Start of: when 1 is appended")
      buf += 1

      it("should contain 1") {

        println("In test: should contain 1; buf is: " + buf)
        buf.remove(0) should equal (1)
        buf should be ('empty)
      }

      describe("when 2 is appended") {

        println("Start of: when 2 is appended")
        buf += 2

        it("should contain 1 and 2") {

          println("In test: should contain 1 and 2; buf is: " + buf)
          buf.remove(0) should equal (1)
          buf.remove(0) should equal (2)
          buf should be ('empty)
        }

        describe("when 2 is removed") {

          println("Start of: when 2 is removed")
          buf -= 2

          it("should contain only 1 again") {

            println("In test: should contain only 1 again; buf is: " + buf)
            buf.remove(0) should equal (1)
            buf should be ('empty)
          }

          println("End of: when 2 is removed")
        }

        describe("when 3 is appended") {

          println("Start of: when 3 is appended")
          buf += 3

          it("should contain 1, 2, and 3") {

            println("In test: should contain 1, 2, and 3; buf is: " + buf)
            buf.remove(0) should equal (1)
            buf.remove(0) should equal (2)
            buf.remove(0) should equal (3)
            buf should be ('empty)
          }
          println("End of: when 3 is appended")
        }

        println("End of: when 2 is appended")
      }

      describe("when 88 is appended") {

        println("Start of: when 88 is appended")
        buf += 88

        it("should contain 1 and 88") {

          println("In test: should contain 1 and 88; buf is: " + buf)
          buf.remove(0) should equal (1)
          buf.remove(0) should equal (88)
          buf should be ('empty)
        }

        println("End of: when 88 is appended")
      }

      println("End of: when 1 is appended")
    }

    it("should have size 0 when created") {

      println("In test: should have size 0 when created; buf is: " + buf)
      buf should have size 0
    }

    println("End of: A ListBuffer")
  }
  println("End of: ExampleSpec")
  println()
}

    Running the above version of ExampleSpec in the Scala interpreter will give you output similar to:

    scala> import org.scalatest._
import org.scalatest._

scala> run(new ExampleSpec)
ExampleSpec:
Start of: ExampleSpec
Start of: A ListBuffer
In test: should be empty when created; buf is: ListBuffer()
End of: A ListBuffer
End of: ExampleSpec

Start of: ExampleSpec
Start of: A ListBuffer
Start of: when 1 is appended
In test: should contain 1; buf is: ListBuffer(1)
ExampleSpec:
End of: when 1 is appended
End of: A ListBuffer
End of: ExampleSpec

Start of: ExampleSpec
Start of: A ListBuffer
Start of: when 1 is appended
Start of: when 2 is appended
In test: should contain 1 and 2; buf is: ListBuffer(1, 2)
End of: when 2 is appended
End of: when 1 is appended
End of: A ListBuffer
End of: ExampleSpec

Start of: ExampleSpec
Start of: A ListBuffer
Start of: when 1 is appended
Start of: when 2 is appended
Start of: when 2 is removed
In test: should contain only 1 again; buf is: ListBuffer(1)
End of: when 2 is removed
End of: when 2 is appended
End of: when 1 is appended
End of: A ListBuffer
End of: ExampleSpec

Start of: ExampleSpec
Start of: A ListBuffer
Start of: when 1 is appended
Start of: when 2 is appended
Start of: when 3 is appended
In test: should contain 1, 2, and 3; buf is: ListBuffer(1, 2, 3)
End of: when 3 is appended
End of: when 2 is appended
End of: when 1 is appended
End of: A ListBuffer
End of: ExampleSpec

Start of: ExampleSpec
Start of: A ListBuffer
Start of: when 1 is appended
Start of: when 88 is appended
In test: should contain 1 and 88; buf is: ListBuffer(1, 88)
End of: when 88 is appended
End of: when 1 is appended
End of: A ListBuffer
End of: ExampleSpec

Start of: ExampleSpec
Start of: A ListBuffer
In test: should have size 0 when created; buf is: ListBuffer()
End of: A ListBuffer
End of: ExampleSpec

A ListBuffer
- should be empty when created
  when 1 is appended
  - should contain 1
    when 2 is appended
    - should contain 1 and 2
      when 2 is removed
      - should contain only 1 again
      when 3 is appended
      - should contain 1, 2, and 3
    when 88 is appended
    - should contain 1 and 88
- should have size 0 when created

    Note that each test is executed in order of appearance in the path.FunSpec, and that only those println statements residing in blocks that enclose the test being run are executed. Any println statements in blocks that do not form the "path" to a test are not executed in the instance of the class that executes that test.

    How it executes

    To provide its special brand of test isolation, path.FunSpec executes quite differently from its sister class in org.scalatest. An org.scalatest.FunSpec registers tests during construction and executes them when run is invoked. An org.scalatest.path.FunSpec, by contrast, runs each test in its own instance while that instance is being constructed. During construction, it registers not the tests to run, but the results of running those tests. When run is invoked on a path.FunSpec, it reports the registered results and does not run the tests again. If run is invoked a second or third time, in fact, a path.FunSpec will each time report the same results registered during construction. If you want to run the tests of a path.FunSpec anew, you'll need to create a new instance and invoke run on that.

    A path.FunSpec will create one instance for each "leaf" node it contains. The main kind of leaf node is a test, such as:

    // One instance will be created for each test
it("should be empty when created") {
  buf should be ('empty)
}

    However, an empty scope (a scope that contains no tests or nested scopes) is also a leaf node:

     // One instance will be created for each empty scope
describe("when 99 is added") {
  // A scope is "empty" and therefore a leaf node if it has no
  // tests or nested scopes, though it may have other code (which
  // will be executed in the instance created for that leaf node)
  buf += 99
}

    The tests will be executed sequentially, in the order of appearance. The first test (or empty scope, if that is first) will be executed when a class that mixes in path.FunSpec is instantiated. Only the first test will be executed during this initial instance, and of course, only the path to that test. Then, the first time the client uses the initial instance (by invoking one of run, expectedTestsCount, tags, or testNames on the instance), the initial instance will, before doing anything else, ensure that any remaining tests are executed, each in its own instance.

    To ensure that the correct path is taken in each instance, and to register its test results, the initial path.FunSpec instance must communicate with the other instances it creates for running any subsequent leaf nodes. It does so by setting a thread-local variable prior to creating each instance (a technique suggested by Esko Luontola). Each instance of path.FunSpec checks the thread-local variable. If the thread-local is not set, it knows it is an initial instance and therefore executes every block it encounters until it discovers, and executes the first test (or empty scope, if that's the first leaf node). It then discovers, but does not execute the next leaf node, or discovers there are no other leaf nodes remaining to execute. It communicates the path to the next leaf node, if any, and the result of running the test it did execute, if any, back to the initial instance. The initial instance repeats this process until all leaf nodes have been executed and all test results registered.

    Ignored tests

    You mark a test as ignored in an org.scalatest.path.FunSpec in the same manner as in an org.scalatest.FunSpec. Please see the Ignored tests section in its documentation for more information.

    Note that a separate instance will be created for an ignored test, and the path to the ignored test will be executed in that instance, but the test function itself will not be executed. Instead, a TestIgnored event will be fired.

    Informers

    You output information using Informers in an org.scalatest.path.FunSpec in the same manner as in an org.scalatest.FunSpec. Please see the Informers section in its documentation for more information.

    Pending tests

    You mark a test as pending in an org.scalatest.path.FunSpec in the same manner as in an org.scalatest.FunSpec. Please see the Pending tests section in its documentation for more information.

    Note that a separate instance will be created for a pending test, and the path to the ignored test will be executed in that instance, as well as the test function (up until it completes abruptly with a TestPendingException).

    Tagging tests

    You can place tests into groups by tagging them in an org.scalatest.path.FunSpec in the same manner as in an org.scalatest.FunSpec. Please see the Tagging tests section in its documentation for more information.

    Note that one difference between this class and its sister class org.scalatest.FunSpec is that because tests are executed at construction time, rather than each time run is invoked, an org.scalatest.path.FunSpec will always execute all non-ignored tests. When run is invoked on a path.FunSpec, if some tests are excluded based on tags, the registered results of running those tests will not be reported. (But those tests will have already run and the results registered.) By contrast, because an org.scalatest.FunSpec only executes tests after run has been called, and at that time the tags to include and exclude are known, only tests selected by the tags will be executed.

    In short, in an org.scalatest.FunSpec, tests not selected by the tags to include and exclude specified for the run (via the Filter passed to run) will not be executed. In an org.scalatest.path.FunSpec, by contrast, all non-ignored tests will be executed, each during the construction of its own instance, and tests not selected by the tags to include and exclude specified for a run will not be reported. (One upshot of this is that if you have tests that you want to tag as being slow so you can sometimes exclude them during a run, you probably don't want to put them in a path.FunSpec. Because in a path.Freespec the slow tests will be run regardless, with only their registered results not being reported if you exclude slow tests during a run.)

    Shared tests

    You can factor out shared tests in an org.scalatest.path.FunSpec in the same manner as in an org.scalatest.FunSpec. Please see the Shared tests section in its documentation for more information.

    Nested suites

    Nested suites are not allowed in a path.FunSpec. Because a path.FunSpec executes tests eagerly at construction time, registering the results of those test runs and reporting them later when run is invoked, the order of nested suites versus test runs would be different in a org.scalatest.path.FunSpec than in an org.scalatest.FunSpec. In org.scalatest.FunSpec's implementation of run, nested suites are executed then tests are executed. A org.scalatest.path.FunSpec with nested suites would execute these in the opposite order: first tests then nested suites. To help make path.FunSpec code easier to reason about by giving readers of one less difference to think about, nested suites are not allowed. If you want to add nested suites to a path.FunSpec, you can instead wrap them all in a Suites object. They will be executed in the order of appearance (unless a Distributor is passed, in which case they will execute in parallel).

    Durations

    Many ScalaTest events include a duration that indicates how long the event being reported took to execute. For example, a TestSucceeded event provides a duration indicating how long it took for that test to execute. A SuiteCompleted event provides a duration indicating how long it took for that entire suite of tests to execute.

    In the test completion events fired by a path.FunSpec (TestSucceeded, TestFailed, or TestPending), the durations reported refer to the time it took for the tests to run. This time is registered with the test results and reported along with the test results each time run is invoked. By contrast, the suite completion events fired for a path.FunSpec represent the amount of time it took to report the registered results. (These events are not fired by path.FunSpec, but instead by the entity that invokes run on the path.FunSpec.) As a result, the total time for running the tests of a path.FunSpec, calculated by summing the durations of all the individual test completion events, may be greater than the duration reported for executing the entire suite.

  4. trait FunSpecLike extends Suite with OneInstancePerTest with Informing with Notifying with Alerting with Documenting

    Implementation trait for class path.FunSpec, which is a sister class to org.scalatest.FunSpec that isolates tests by running each test in its own instance of the test class, and for each test, only executing the path leading to that test.

    Implementation trait for class path.FunSpec, which is a sister class to org.scalatest.FunSpec that isolates tests by running each test in its own instance of the test class, and for each test, only executing the path leading to that test.

    path.FunSpec is a class, not a trait, to minimize compile time given there is a slight compiler overhead to mixing in traits compared to extending classes. If you need to mix the behavior of path.FunSpec into some other class, you can use this trait instead, because class path.FunSpec does nothing more than extend this trait and add a nice toString implementation.

    See the documentation of the class for a detailed overview of path.FunSpec.

Ungrouped