scalatest
Trait providing an apply method to which alert messages about a running suite of tests can be reported.
Trait providing an apply method to which alert messages about a running suite of tests can be reported.
An Alerter is essentially
used to wrap a Reporter and provide easy ways to send alert messages
to that Reporter via an AlertProvided event.
Alerter contains an apply method that takes a string and
an optional payload object of type Any.
The Alerter will forward the passed alert message string to the
Reporter as the message parameter, and the optional
payload object as the payload parameter, of an AlertProvided event.
For insight into the differences between Alerter, Notifier, and Informer, see the
main documentation for trait Alerting.
Trait that contains the alert method, which can be used to send an alert to the reporter.
Trait that contains the alert method, which can be used to send an alert to the reporter.
One difference between alert and the info method of Informer is that
info messages provided during a test are recorded and sent as part of test completion event, whereas
alert messages are sent right away as AlertProvided messages. For long-running tests,
alert allows you to send "alert notifications" to the reporter right away, so users can be made aware
of potential problems being experienced by long-running tests. By contrast, info messages will only be seen by the user after the
test has completed, and are more geared towards specification (such as Given/When/Then messages) than notification.
The difference between alert and the update method of Updating is
that alert is intended to be used
for warnings or notifications of potential problems, whereas update is just for status updates.
In string reporters for which ANSI color is enabled, update notifications are shown in green and alert notifications
in yellow.
Trait providing an implicit conversion that allows clues to be placed after a block of code.
Trait providing an implicit conversion that allows clues to be placed after a block of code.
You can use the withClue construct provided by Assertions, which is
extended by every style trait in ScalaTest, to add extra information to reports of failed or canceled tests.
The withClue from Assertions places the "clue string" at the front, both
in the code and in the resulting message:
withClue("This is a prepended clue;") {
1 + 1 should equal (3)
}
The above expression will yield the failure message:
This is a prepended clue; 2 did not equal 3
If you mix in this trait, or import its members via its companion object, you can alternatively place the clue string at the end, like this:
{ 1 + 1 should equal (3) } withClue "now the clue comes after"
The above expression will yield the failure message:
2 did not equal 3 now the clue comes after
If no space is already present, either at the beginning of the clue string or at the end
of the current message, a space will be placed between the two, unless the clue string
starts with one of the punctuation characters: comma (,), period (.),
or semicolon (;). For example, the failure message in the above example
includes an extra space inserted between 3 and now.
By contrast this code, which has a clue string starting with comma:
{ 1 + 1 should equal (3) } withClue ", now the clue comes after"
Will yield a failure message with no extra inserted space:
2 did not equal 3, now the clue comes after
The withClue method will only append the clue string to the detail
message of exception types that mix in the ModifiableMessage trait.
See the documentation for ModifiableMessage for more
information.
Note: the reason this functionality is not provided by Assertions directly, like the
prepended withClue construct, is because appended clues require an implicit conversion.
ScalaTest only gives you one implicit conversion by default in any test class to minimize the
potential for conflicts with other implicit conversions you may be using. All other implicit conversions,
including the one provided by this trait, you must explicitly invite into your code through inheritance
or an import.
Arguments bundle passed to four of ScalaTest's lifecycle methods: run, runNestedSuites,
runTests, and runTest.
Arguments bundle passed to four of ScalaTest's lifecycle methods: run, runNestedSuites,
runTests, and runTest.
The signatures of these methods, defined in trait Suite, are:
def run(testName: Option[String], args: Args) def runNestedSuites(args: Args) def runTests(testName: Option[String], args: Args) def runTest(testName: String, args: Args)
The purpose of bundling these arguments into an Args object instead of passing them in individually is to make the signature
of these four lifecycle methods easier to read, write, and remember, as well as to make the methods more pleasant to override in user code.
the Reporter to which results will be reported
the Stopper that will be consulted to determine whether to stop execution early.
a Filter with which to filter tests based on their tags
a ConfigMap of key-value pairs that can be used by the executing Suite of tests.
an optional Distributor, into which to put nested Suites to be executed
by another entity, such as concurrently by a pool of threads. If None, nested Suites will be executed sequentially.
a Tracker tracking Ordinals being fired by the current thread.
a (possibly empty) Set of Strings specifying the run's chosen styles
a flag used to pass information between run methods
in OneInstancePerTest and ParallelTestExecution.
an optional DistributedTestSorter used by ParallelTestExecution to sort the events
for the parallel-executed tests of one suite back into sequential order on the fly, with a timeout in case a test takes too long to complete
an optional DistributedSuiteSorter used by ParallelTestExecution to ensure the events
for the parallel-executed suites are sorted back into sequential order, with a timeout in case a suite takes to long to complete, even when tests are executed in parallel
NullArgumentException if any passed parameter is null.
Trait that contains ScalaTest's basic assertion methods.
Trait that contains ScalaTest's basic assertion methods.
You can use the assertions provided by this trait in any ScalaTest Suite,
because Suite
mixes in this trait. This trait is designed to be used independently of anything else in ScalaTest, though, so you
can mix it into anything. (You can alternatively import the methods defined in this trait. For details, see the documentation
for the Assertions companion object.
In any Scala program, you can write assertions by invoking assert and passing in a Boolean expression,
such as:
val left = 2 val right = 1 assert(left == right)
If the passed expression is true, assert will return normally. If false,
Scala's assert will complete abruptly with an AssertionError. This behavior is provided by
the assert method defined in object Predef, whose members are implicitly imported into every
Scala source file. This Assertions trait defines another assert method that hides the
one in Predef. It behaves the same, except that if false is passed it throws
TestFailedException instead of AssertionError.
Why? Because unlike AssertionError, TestFailedException carries information about exactly
which item in the stack trace represents
the line of test code that failed, which can help users more quickly find an offending line of code in a failing test.
In addition, ScalaTest's assert provides better error messages than Scala's assert.
If you pass the previous Boolean expression, left == right to assert in a ScalaTest test,
a failure will be reported that, because assert is implemented as a macro,
includes reporting the left and right values.
For example, given the same code as above but using ScalaTest assertions:
import org.scalatest.Assertions._ val left = 2 val right = 1 assert(left == right)
The detail message in the thrown TestFailedException from this assert
will be: "2 did not equal 1".
ScalaTest's assert macro works by recognizing patterns in the AST of the expression passed to assert and,
for a finite set of common expressions, giving an error message that an equivalent ScalaTest matcher
expression would give. Here are some examples, where a is 1, b is 2, c is 3, d
is 4, xs is List(a, b, c), and num is 1.0:
assert(a == b || c >= d)
// Error message: 1 did not equal 2, and 3 was not greater than or equal to 4
assert(xs.exists(_ == 4))
// Error message: List(1, 2, 3) did not contain 4
assert("hello".startsWith("h") && "goodbye".endsWith("y"))
// Error message: "hello" started with "h", but "goodbye" did not end with "y"
assert(num.isInstanceOf[Int])
// Error message: 1.0 was not instance of scala.Int
assert(Some(2).isEmpty)
// Error message: Some(2) was not empty
For expressions that are not recognized, the macro currently prints out a string
representation of the (desugared) AST and adds "was false". Here are some examples of
error messages for unrecognized expressions:
assert(None.isDefined) // Error message: scala.None.isDefined was false assert(xs.exists(i => i > 10)) // Error message: xs.exists(((i: Int) => i.>(10))) was false
You can augment the standard error message by providing a String as a second argument
to assert, like this:
val attempted = 2 assert(attempted == 1, "Execution was attempted " + left + " times instead of 1 time")
Using this form of assert, the failure report will be more specific to your problem domain, thereby
helping you debug the problem. This Assertions trait also mixes in the
TripleEquals, which gives you a === operator
that allows you to customize Equality, perform equality checks with numeric
Tolerance, and enforce type constraints at compile time with
sibling traits TypeCheckedTripleEquals and
ConversionCheckedTripleEquals.
Although the assert macro provides a natural, readable extension to Scala's assert mechanism that
provides good error messages, as the operands become lengthy, the code becomes less readable. In addition, the error messages
generated for == and === comparisons
don't distinguish between actual and expected values. The operands are just called left and right,
because if one were named expected and the other actual, it would be difficult for people to
remember which was which. To help with these limitations of assertions, Suite includes a method called assertResult that
can be used as an alternative to assert. To use assertResult, you place
the expected value in parentheses after assertResult, followed by curly braces containing code
that should result in the expected value. For example:
val a = 5
val b = 2
assertResult(2) {
a - b
}
In this case, the expected value is 2, and the code being tested is a - b. This assertion will fail, and
the detail message in the TestFailedException will read, "Expected 2, but got 3."
If you just need the test to fail, you can write:
fail()
Or, if you want the test to fail with a message, write:
fail("I've got a bad feeling about this")
In async style tests, you must end your test body with either Future[Assertion] or
Assertion. ScalaTest's assertions (including matcher expressions) have result type
Assertion, so ending with an assertion will satisfy the compiler.
If a test body or function body passed to Future.map does
not end with type Assertion, however, you can fix the type error by placing
succeed at the end of the
test or function body:
succeed // Has type Assertion
Sometimes you need to test whether a method throws an expected exception under certain circumstances, such as when invalid arguments are passed to the method. You can do this in the JUnit 3 style, like this:
val s = "hi"
try {
s.charAt(-1)
fail()
}
catch {
case _: IndexOutOfBoundsException => // Expected, so continue
}
If charAt throws IndexOutOfBoundsException as expected, control will transfer
to the catch case, which does nothing. If, however, charAt fails to throw an exception,
the next statement, fail(), will be run. The fail method always completes abruptly with
a TestFailedException, thereby signaling a failed test.
To make this common use case easier to express and read, ScalaTest provides two methods:
assertThrows and intercept.
Here's how you use assertThrows:
val s = "hi"
assertThrows[IndexOutOfBoundsException] { // Result type: Assertion
s.charAt(-1)
}
This code behaves much like the previous example. If charAt throws an instance of IndexOutOfBoundsException,
assertThrows will return Succeeded. But if charAt completes normally, or throws a different
exception, assertThrows will complete abruptly with a TestFailedException.
The intercept method behaves the same as assertThrows, except that instead of returning Succeeded,
intercept returns the caught exception so that you can inspect it further if you wish. For example, you may need
to ensure that data contained inside the exception have expected values. Here's an example:
val s = "hi"
val caught =
intercept[IndexOutOfBoundsException] { // Result type: IndexOutOfBoundsException
s.charAt(-1)
}
assert(caught.getMessage.indexOf("-1") != -1)
Often when creating libraries you may wish to ensure that certain arrangements of code that
represent potential “user errors” do not compile, so that your library is more error resistant.
ScalaTest's Assertions trait includes the following syntax for that purpose:
assertDoesNotCompile("val a: String = 1")
If you want to ensure that a snippet of code does not compile because of a type error (as opposed to a syntax error), use:
assertTypeError("val a: String = 1")
Note that the assertTypeError call will only succeed if the given snippet of code does not
compile because of a type error. A syntax error will still result on a thrown TestFailedException.
If you want to state that a snippet of code does compile, you can make that more obvious with:
assertCompiles("val a: Int = 1")
Although the previous three constructs are implemented with macros that determine at compile time whether the snippet of code represented by the string does or does not compile, errors are reported as test failures at runtime.
Trait Assertions also provides methods that allow you to cancel a test.
You would cancel a test if a resource required by the test was unavailable. For example, if a test
requires an external database to be online, and it isn't, the test could be canceled to indicate
it was unable to run because of the missing database. Such a test assumes a database is
available, and you can use the assume method to indicate this at the beginning of
the test, like this:
assume(database.isAvailable)
For each overloaded assert method, trait Assertions provides an
overloaded assume method with an identical signature and behavior, except the
assume methods throw TestCanceledException whereas the
assert methods throw TestFailedException. As with assert,
assume hides a Scala method in Predef that performs a similar
function, but throws AssertionError. And just as you can with assert,
you will get an error message extracted by a macro from the AST passed to assume, and can
optionally provide a clue string to augment this error message. Here are some examples:
assume(database.isAvailable, "The database was down again") assume(database.getAllUsers.count === 9)
For each overloaded fail method, there's a corresponding cancel method
with an identical signature and behavior, except the cancel methods throw
TestCanceledException whereas the fail methods throw
TestFailedException. Thus if you just need to cancel a test, you can write:
cancel()
If you want to cancel the test with a message, just place the message in the parentheses:
cancel("Can't run the test because no internet connection was found")
If you want more information that is provided by default by the methods if this trait,
you can supply a "clue" string in one of several ways.
The extra information (or "clues") you provide will
be included in the detail message of the thrown exception. Both
assert and assertResult provide a way for a clue to be
included directly, intercept does not.
Here's an example of clues provided directly in assert:
assert(1 + 1 === 3, "this is a clue")
and in assertResult:
assertResult(3, "this is a clue") { 1 + 1 }
The exceptions thrown by the previous two statements will include the clue
string, "this is a clue", in the exception's detail message.
To get the same clue in the detail message of an exception thrown
by a failed intercept call requires using withClue:
withClue("this is a clue") {
intercept[IndexOutOfBoundsException] {
"hi".charAt(-1)
}
}
The withClue method will only prepend the clue string to the detail
message of exception types that mix in the ModifiableMessage trait.
See the documentation for ModifiableMessage for more information.
If you wish to place a clue string after a block of code, see the documentation for
AppendedClues.
Note: ScalaTest's assertTypeError construct is in part inspired by the illTyped macro
of shapeless.
Trait declaring methods that can be used to register by-name test functions that
have result type Future[Assertion].
Trait declaring methods that can be used to register by-name test functions that
have result type Future[Assertion].
The base trait of ScalaTest's asynchronous testing styles, which defines a
withFixture lifecycle method that accepts as its parameter a test function
that returns a FutureOutcome.
The base trait of ScalaTest's asynchronous testing styles, which defines a
withFixture lifecycle method that accepts as its parameter a test function
that returns a FutureOutcome.
The withFixture method add by this trait has the
following signature and implementation:
def withFixture(test: NoArgAsyncTest): FutureOutcome = {
test()
}
This trait enables testing of asynchronous code without blocking. Instead of returning
Outcome like TestSuite's
withFixture, this trait's withFixture method returns a
FutureOutcome. Similarly, the apply method of test function interface,
NoArgAsyncTest, returns FutureOutcome:
// In trait NoArgAsyncTest: def apply(): FutureOutcome
The withFixture method supports async testing, because when the test function returns,
the test body has not necessarily finished execution.
The recommended way to ensure cleanup is performed after a test body finishes execution is
to use a complete-lastly clause, syntax that is defined in trait
CompleteLastly, which this trait extends.
Using cleanup-lastly will ensure that cleanup will occur whether
FutureOutcome-producing code completes abruptly by throwing an exception, or returns
normally yielding a FutureOutcome. In the latter case,
complete-lastly will
register the cleanup code to execute asynchronously when the FutureOutcome completes.
The withFixture method is designed to be stacked, and to enable this, you should always call the super implementation
of withFixture, and let it invoke the test function rather than invoking the test function directly. In other words, instead of writing
“test()”, you should write “super.withFixture(test)”. Thus, the recommended
structure of a withFixture implementation that performs cleanup looks like this:
// Your implementation
override def withFixture(test: NoArgAsyncTest) = {
// Perform setup here
complete {
super.withFixture(test) // Invoke the test function
} lastly {
// Perform cleanup here
}
}
If you have no cleanup to perform, you can write withFixture like this instead:
// Your implementation
override def withFixture(test: NoArgAsyncTest) = {
// Perform setup here
super.withFixture(test) // Invoke the test function
}
The test function and withFixture method returns a
FutureOutcome,
a ScalaTest class that wraps a Scala Future[Outcome] and offers methods
more specific to asynchronous test outcomes. In a Scala Future, any exception
results in a scala.util.Failure. In a FutureOutcome, a
thrown TestPendingException always results in a Pending,
a thrown TestCanceledException always results in a Canceled,
and any other exception, so long as it isn't suite-aborting, results in a
Failed. This is true of the asynchronous test code itself that's represented by
the FutureOutcome and any transformation or callback registered on the
FutureOutcome in withFixture.
If you want to perform an action only for certain outcomes, you'll need to
register code performing that action on the FutureOutcome using
one of FutureOutcome's callback registration methods:
onSucceededThen - executed if the Outcome is a Succeeded.onFailedThen - executed if the Outcome is a Failed.onCanceledThen - executed if the Outcome is a Canceled.onPendingThen - executed if the Outcome is a Pending.onOutcomeThen - executed on any Outcome (i.e., no
suite-aborting exception is thrown).onAbortedThen - executed if a suite-aborting exception is thrown.onCompletedThen - executed whether the result is an Outcome
or a thrown suite-aborting exception.For example, if you want to perform an action if a test fails, you'd register the
callback using onFailedThen, like this:
// Your implementation
override def withFixture(test: NoArgAsyncTest) = {
// Perform setup here
val futureOutcome = super.withFixture(test) // Invoke the test function
futureOutcome onFailedThen { ex =>
// perform action that you want to occur
// only if a test fails here
}
}
Note that all callback registration methods, such as onFailedThen used in the
previous example, return a new FutureOutcome that won't complete until the
the original FutureOutcome and the callback has completed. If the callback
throws an exception, the resulting FutureOutcome will represent that exception.
For example, if a FutureOutcome results in Failed, but a callback
registered on that FutureOutcome with onFailedThen throws TestPendingException, the
result of the FutureOutcome returned by onFailedThen will
be Pending.
Lastly, if you want to change the outcome in some way in withFixture, you'll need to use
the change method of FutureOutcome, like this:
// Your implementation
override def withFixture(test: NoArgAsyncTest) = {
// Perform setup here
val futureOutcome = super.withFixture(test) // Invoke the test function
futureOutcome change { outcome =>
// transform the outcome into a new outcome here
}
}
Trait defining abstract "lifecycle" methods that are implemented in AsyncTestSuite
and can be overridden in stackable modification traits.
Trait defining abstract "lifecycle" methods that are implemented in AsyncTestSuite
and can be overridden in stackable modification traits.
The main use case for this trait is to override withFixture in a mixin trait.
Here's an example:
trait Builder extends AsyncTestSuiteMixin { this: AsyncTestSuite =>
final val builder = new ThreadSafeStringBuilder
abstract override def withFixture(test: NoArgAsyncTest) = {
builder.append("ScalaTest is ")
complete {
super.withFixture(test) // To be stackable, must call super.withFixture
} lastly {
builder.clear()
}
}
}
Trait that can be mixed into suites that need code executed before and after running each test.
Trait that can be mixed into suites that need code executed before and after running each test.
Recommended Usage:
Use trait BeforeAndAfter when you need to perform the same side-effects before and/or after tests, rather than at the beginning
or end of tests. Note: For more insight into where BeforeAndAfter fits into the big picture, see the
Shared fixtures section in the documentation for your chosen style trait.
|
A test fixture is composed of the objects and other artifacts (files, sockets, database
connections, etc.) tests use to do their work.
When multiple tests need to work with the same fixtures, it is important to try and avoid
duplicating the fixture code across those tests. The more code duplication you have in your
tests, the greater drag the tests will have on refactoring the actual production code.
Trait BeforeAndAfter offers one way to eliminate such code duplication:
a before clause that will register code to be run before each test,
and an after clause that will register code to be run after.
Here's an example:
package org.scalatest.examples.flatspec.beforeandafter
import org.scalatest._
import collection.mutable.ListBuffer
class ExampleSpec extends FlatSpec with BeforeAndAfter {
val builder = new StringBuilder
val buffer = new ListBuffer[String]
before {
builder.append("ScalaTest is ")
}
after {
builder.clear()
buffer.clear()
}
"Testing" should "be easy" in {
builder.append("easy!")
assert(builder.toString === "ScalaTest is easy!")
assert(buffer.isEmpty)
buffer += "sweet"
}
it should "be fun" in {
builder.append("fun!")
assert(builder.toString === "ScalaTest is fun!")
assert(buffer.isEmpty)
}
}
The before and after methods can each only be called once per Suite,
and cannot be invoked after run has been invoked. If either of the registered before or after functions
complete abruptly with an exception, it will be reported as an aborted suite and no more tests will be attempted in that suite.
Note that the only way before and after code can communicate with test code is via some side-effecting mechanism, commonly by
reassigning instance vars or by changing the state of mutable objects held from instance vals (as in this example). If using
instance vars or mutable objects held from instance vals you wouldn't be able to run tests in parallel in the same instance
of the test class unless you synchronized access to the shared, mutable state. This is why ScalaTest's ParallelTestExecution trait extends
OneInstancePerTest. By running each test in its own instance of the class, each test has its own copy of the instance variables, so you
don't need to synchronize. Were you to mix ParallelTestExecution into the ExampleSuite above, the tests would run in parallel just fine
without any synchronization needed on the mutable StringBuilder and ListBuffer[String] objects.
Although BeforeAndAfter provides a minimal-boilerplate way to execute code before and after tests, it isn't designed to enable stackable
traits, because the order of execution would be non-obvious. If you want to factor out before and after code that is common to multiple test suites, you
should use trait BeforeAndAfterEach instead.
The advantage this trait has over BeforeAndAfterEach is that its syntax is more concise.
The main disadvantage is that it is not stackable, whereas BeforeAndAfterEach is. I.e.,
you can write several traits that extend BeforeAndAfterEach and provide beforeEach methods
that include a call to super.beforeEach, and mix them together in various combinations. By contrast,
only one call to the before registration function is allowed in a suite or spec that mixes
in BeforeAndAfter. In addition, BeforeAndAfterEach allows you to access
the config map and test name via the TestData passed to its beforeEach and
afterEach methods, whereas BeforeAndAfter
gives you no access to the config map.
Stackable trait that can be mixed into suites that need methods invoked before and after executing the suite.
Stackable trait that can be mixed into suites that need methods invoked before and after executing the suite.
This trait allows code to be executed before and/or after all the tests and nested suites of a
suite are run. This trait overrides run and calls the
beforeAll method, then calls super.run. After the super.run
invocation completes, whether it returns normally or completes abruptly with an exception,
this trait's run method will invoke afterAll.
Trait BeforeAndAfterAll defines beforeAll
and afterAll methods that take no parameters. This trait's implementation of these
methods do nothing.
For example, the following ExampleSpec mixes in BeforeAndAfterAll and
in beforeAll, creates and writes to a temp file.
Each test class, ExampleSpec and all its nested
suites--OneSpec, TwoSpec, RedSpec,
and BlueSpec--tests that the file exists. After all of the nested suites
have executed, afterAll is invoked, which
deletes the file.
(Note: if you're unfamiliar with the withFixture(OneArgTest) approach to shared fixtures, check out
the documentation for trait fixture.FlatSpec.)
package org.scalatest.examples.beforeandafterall
import org.scalatest._
import java.io._
trait TempFileExistsSpec extends fixture.FlatSpecLike {
protected val tempFileName = "tmp.txt"
type FixtureParam = File
override def withFixture(test: OneArgTest) = {
val file = new File(tempFileName)
withFixture(test.toNoArgTest(file)) // loan the fixture to the test
}
"The temp file" should ("exist in " + suiteName) in { file =>
assert(file.exists)
}
}
class OneSpec extends TempFileExistsSpec
class TwoSpec extends TempFileExistsSpec
class RedSpec extends TempFileExistsSpec
class BlueSpec extends TempFileExistsSpec
class ExampleSpec extends Suites(
new OneSpec,
new TwoSpec,
new RedSpec,
new BlueSpec
) with TempFileExistsSpec with BeforeAndAfterAll {
// Set up the temp file needed by the test, taking
// a file name from the config map
override def beforeAll() {
val writer = new FileWriter(tempFileName)
try writer.write("Hello, suite of tests!")
finally writer.close()
}
// Delete the temp file
override def afterAll() {
val file = new File(tempFileName)
file.delete()
}
}
If you do supply a mapping for "tempFileName" in the config map, you'll see that the temp file is available to all the tests:
scala> org.scalatest.run(new ExampleSpec)
ExampleSpec:
OneSpec:
The temp file
- should exist in OneSpec
TwoSpec:
The temp file
- should exist in TwoSpec
RedSpec:
The temp file
- should exist in RedSpec
BlueSpec:
The temp file
- should exist in BlueSpec
The temp file
- should exist in ExampleSpec
Note: this trait uses the Status result of Suite's "run" methods
to ensure that the code in afterAll is executed after
all the tests and nested suites are executed even if a Distributor is passed.
Note that it is not guaranteed that afterAll is invoked from the same thread as beforeAll,
so if there's any shared state between beforeAll and afterAll you'll need to make sure they are
synchronized correctly.
Trait that can be mixed into suites that need methods that make use of the config map invoked before and/or after executing the suite.
Trait that can be mixed into suites that need methods that make use of the config map invoked before and/or after executing the suite.
This trait allows code to be executed before and/or after all the tests and nested suites of a
suite are run. This trait overrides run and calls the
beforeAll(ConfigMap) method, then calls super.run. After the super.run
invocation completes, whether it returns normally or completes abruptly with an exception,
this trait's run method will invoke afterAll(ConfigMap).
Note that this trait differs from BeforeAndAfterAll in that it gives
the beforeAll and afterAll code access to the config map. If you don't need
the config map, use BeforeAndAfterAll instead.
Trait BeforeAndAfterAllConfigMap defines beforeAll
and afterAll methods that take a configMap.
This trait's implemention of each method does nothing.
For example, the following ExampleSpec mixes in BeforeAndAfterAllConfigMap and
in beforeAll, creates and writes to a temp file, taking the name of the temp file
from the configMap. This same configMap is then passed to the run
methods of the nested suites, OneSpec, TwoSpec, RedSpec,
and BlueSpec, so those suites can access the filename and, therefore, the file's
contents. After all of the nested suites have executed, afterAll is invoked, which
again grabs the file name from the configMap and deletes the file. Each of these five
test classes extend trait TempFileExistsSpec, which defines a test that ensures the temp file exists.
(Note: if you're unfamiliar with the withFixture(OneArgTest) approach to shared fixtures, check out
the documentation for trait fixture.FlatSpec.)
package org.scalatest.examples.beforeandafterallconfigmap
import org.scalatest._
import java.io._
trait TempFileExistsSpec extends fixture.FlatSpec {
type FixtureParam = File
override def withFixture(test: OneArgTest) = {
val fileName = test.configMap.getRequired[String]("tempFileName")
val file = new File(fileName)
withFixture(test.toNoArgTest(file)) // loan the fixture to the test
}
"The temp file" should ("exist in " + suiteName) in { file =>
assert(file.exists)
}
}
class OneSpec extends TempFileExistsSpec
class TwoSpec extends TempFileExistsSpec
class RedSpec extends TempFileExistsSpec
class BlueSpec extends TempFileExistsSpec
class ExampleSpec extends Suites(
new OneSpec,
new TwoSpec,
new RedSpec,
new BlueSpec
) with TempFileExistsSpec with BeforeAndAfterAllConfigMap {
private val tempFileName = "tempFileName"
// Set up the temp file needed by the test, taking
// a file name from the config map
override def beforeAll(cm: ConfigMap) {
assume(
cm.isDefinedAt(tempFileName),
"must place a temp file name in the config map under the key: " + tempFileName
)
val fileName = cm.getRequired[String](tempFileName)
val writer = new FileWriter(fileName)
try writer.write("Hello, suite of tests!")
finally writer.close()
}
// Delete the temp file
override def afterAll(cm: ConfigMap) {
val fileName = cm.getRequired[String]("tempFileName")
val file = new File(fileName)
file.delete()
}
}
Running the above class in the interpreter will give an error if you don't supply a mapping for "tempFileName" in the config map:
scala> org.scalatest.run(new ExampleSpec) ExampleSpec: Exception encountered when invoking run on a suite. *** ABORTED *** Exception encountered when invoking run on a suite. (:30) *** RUN ABORTED *** An exception or error caused a run to abort: must place a temp file name in the config map under the key: tempFileName ( :30)
If you do supply a mapping for "tempFileName" in the config map, you'll see that the temp file is available to all the tests:
scala> (new ExampleSpec).execute(configMap = ConfigMap("tempFileName" -> "tmp.txt"))
ExampleSpec:
OneSpec:
The temp file
- should exist in OneSpec
TwoSpec:
The temp file
- should exist in TwoSpec
RedSpec:
The temp file
- should exist in RedSpec
BlueSpec:
The temp file
- should exist in BlueSpec
The temp file
- should exist in ExampleSpec
Note: As of 2.0.M5, this trait uses the newly added Status result of Suite's "run" methods
to ensure that the code in afterAll is executed after
all the tests and nested suites are executed even if a Distributor is passed.
Note that it is not guaranteed that afterAll is invoked from the same thread as beforeAll,
so if there's any shared state between beforeAll and afterAll you'll need to make sure they are
synchronized correctly.
Stackable trait that can be mixed into suites that need code executed before and/or after running each test.
Stackable trait that can be mixed into suites that need code executed before and/or after running each test.
Recommended Usage:
Use trait BeforeAndAfterEach when you want to stack traits that perform side-effects before and/or after tests, rather
than at the beginning or end of tests.
Note: For more insight into where BeforeAndAfterEach fits into the big picture, see the
Shared fixtures section in the documentation for your chosen style trait.
|
A test fixture is composed of the objects and other artifacts (files, sockets, database
connections, etc.) tests use to do their work.
When multiple tests need to work with the same fixtures, it is important to try and avoid
duplicating the fixture code across those tests. The more code duplication you have in your
tests, the greater drag the tests will have on refactoring the actual production code, and
the slower your compile will likely be.
Trait BeforeAndAfterEach offers one way to eliminate such code duplication:
a beforeEach method that will be run before each test (like JUnit's setUp),
and an afterEach method that will be run after (like JUnit's tearDown).
Here's an example:
package org.scalatest.examples.flatspec.composingbeforeandaftereach
import org.scalatest._
import collection.mutable.ListBuffer
trait Builder extends BeforeAndAfterEach { this: Suite =>
val builder = new StringBuilder
override def beforeEach() {
builder.append("ScalaTest is ")
super.beforeEach() // To be stackable, must call super.beforeEach
}
override def afterEach() {
try {
super.afterEach() // To be stackable, must call super.afterEach
}
finally {
builder.clear()
}
}
}
trait Buffer extends BeforeAndAfterEach { this: Suite =>
val buffer = new ListBuffer[String]
override def afterEach() {
try {
super.afterEach() // To be stackable, must call super.afterEach
}
finally {
buffer.clear()
}
}
}
class ExampleSpec extends FlatSpec with Builder with Buffer {
"Testing" should "be easy" in {
builder.append("easy!")
assert(builder.toString === "ScalaTest is easy!")
assert(buffer.isEmpty)
buffer += "sweet"
}
it should "be fun" in {
builder.append("fun!")
assert(builder.toString === "ScalaTest is fun!")
assert(buffer.isEmpty)
buffer += "clear"
}
}
To get the same ordering as withFixture, place your super.beforeEach call at the end of each
beforeEach method, and the super.afterEach call at the beginning of each afterEach
method, as shown in the previous example. It is a good idea to invoke super.afterEach in a try
block and perform cleanup in a finally clause, as shown in the previous example, because this ensures the
cleanup code is performed even if super.afterEach throws an exception.
The main advantage of BeforeAndAfterEach over BeforeAndAfter is that BeforeAndAfterEach.
enables trait stacking.
The main disadvantage of BeforeAndAfterEach compared to BeforeAndAfter is that BeforeAndAfterEach
requires more boilerplate. If you don't need trait stacking, use BeforeAndAfter instead
of BeforeAndAfterEach.
If you want to make use of test data (the test name, config map, etc.) in your beforeEach
or afterEach method, use trait BeforeAndAfterEachTestData instead.
Stackable trait that can be mixed into suites that need code that makes use of test data (test name, tags, config map, etc.) executed before and/or after running each test.
Stackable trait that can be mixed into suites that need code that makes use of test data (test name, tags, config map, etc.) executed before and/or after running each test.
Recommended Usage:
Use trait BeforeAndAfterEachTestData when you want to stack traits that perform side-effects before and/or after tests, rather
than at the beginning or end of tests, when you need access to any test data (such as the config map) in the before and/or after code.
Note: For more insight into where BeforeAndAfterEachTestData fits into the big picture, see the
Shared fixtures section in the documentation for your chosen style trait.
|
A test fixture is composed of the objects and other artifacts (files, sockets, database
connections, etc.) tests use to do their work.
When multiple tests need to work with the same fixtures, it is important to try and avoid
duplicating the fixture code across those tests. The more code duplication you have in your
tests, the greater drag the tests will have on refactoring the actual production code.
Trait BeforeAndAfterEachTestData offers one way to eliminate such code duplication:
a beforeEach(TestData) method that will be run before each test (like JUnit's setUp),
and an afterEach(TestData) method that will be run after (like JUnit's tearDown).
Here's an example:
package org.scalatest.examples.flatspec.composingbeforeandaftereachtestdata
import org.scalatest._
import collection.mutable.ListBuffer
trait Builder extends BeforeAndAfterEachTestData { this: Suite =>
val builder = new StringBuilder
override def beforeEach(td: TestData) {
builder.append(td.name)
super.beforeEach(td) // To be stackable, must call super.beforeEach(TestData)
}
override def afterEach(td: TestData) {
try {
super.afterEach(td) // To be stackable, must call super.afterEach(TestData)
}
finally {
builder.clear()
}
}
}
trait Buffer extends BeforeAndAfterEachTestData { this: Suite =>
val buffer = new ListBuffer[String]
override def afterEach(td: TestData) {
try {
super.afterEach(td) // To be stackable, must call super.afterEach(TestData)
}
finally {
buffer.clear()
}
}
}
class ExampleSpec extends FlatSpec with Builder with Buffer {
"Testing" should "be easy" in {
builder.append("!")
assert(builder.toString === "Testing should be easy!")
assert(buffer.isEmpty)
buffer += "sweet"
}
it should "be fun" in {
builder.append("!")
assert(builder.toString === "Testing should be fun!")
assert(buffer.isEmpty)
buffer += "clear"
}
}
To get the same ordering as withFixture, place your super.beforeEach(TestData) call at the end of each
beforeEach(TestData) method, and the super.afterEach(TestData) call at the beginning of each afterEach(TestData)
method, as shown in the previous example. It is a good idea to invoke super.afterEach(TestData) in a try
block and perform cleanup in a finally clause, as shown in the previous example, because this ensures the
cleanup code is performed even if super.afterEach(TestData) throws an exception.
Besides enabling trait stacking, the other main advantage of BeforeAndAfterEachTestData over BeforeAndAfter
is that BeforeAndAfterEachTestData allows you to make use of test data (such as the test name and config map) in your before
and/or after code, whereas BeforeAndAfter does not.
The main disadvantage of BeforeAndAfterEachTestData compared to BeforeAndAfter and BeforeAndAfterEach is
that BeforeAndAfterEachTestData requires more boilerplate. If you don't need trait stacking or access to the test data, use
BeforeAndAfter instead
of BeforeAndAfterEachTestData.
If you need trait stacking, but not access to the TestData, use
BeforeAndAfterEach instead.
Trait that when mixed into a TestSuite cancels any remaining tests in that
TestSuite instance after a test fails.
Trait that when mixed into a TestSuite cancels any remaining tests in that
TestSuite instance after a test fails.
The intended use case for this trait is if you have a suite of long-running tests that are related such that if one fails, you aren't interested in running the others, you can use this trait to simply cancel any remaining tests, so you need not wait long for them to complete.
Note that this trait only cancels tests in the same TestSuite instance, because
it uses a private volatile instance variable as a flag to indicate whether or not a test has failed.
If you are running each test in its own instance, therefore, it would not cancel the
remaining tests, because they would not see the same flag. For this reason, this trait contains
a final implementation of a method defined in OneInstancePerTest,
to prevent it from being mixed into any class that also mixes in OneInstancePerTest,
including by mixing in ParallelTestExecution or a path trait.
Outcome for a test that was canceled, containing an exception describing the cause of the cancelation.
Trait providing class Checkpoint, which enables multiple assertions
to be performed within a test, with any failures accumulated and reported
together at the end of the test.
Trait providing class Checkpoint, which enables multiple assertions
to be performed within a test, with any failures accumulated and reported
together at the end of the test.
Because ScalaTest uses exceptions to signal failed assertions, normally execution
of a test will stop as soon as the first failed assertion is encountered. Trait
Checkpoints provides an option when you want to continue executing
the remainder of the test body, or part of it, even if an assertion has already failed in that test.
To use a Checkpoint (once you've mixed in or imported the members of trait
Checkpoints), you first need to create one, like this:
val cp = new Checkpoint
Then give the Checkpoint assertions to execute by passing them (via a by-name parameter)
to its apply method, like this:
val (x, y) = (1, 2)
cp { x should be < 0 }
cp { y should be > 9 }
Both of the above assertions will fail, but it won't be reported yet. The Checkpoint will execute them
right away, each time its apply method is invoked. But it will catch the TestFailedExceptions and
save them, only reporting them later when reportAll is invoked. Thus, at the end of the test, you must call
reportAll, like this:
cp.reportAll()
This reportAll invocation will complete abruptly with a TestFailedException whose message
includes the message, source file, and line number of each of the checkpointed assertions that previously failed. For example:
1 was not less than 0 (in Checkpoint) at ExampleSpec.scala:12 2 was not greater than 9 (in Checkpoint) at ExampleSpec.scala:13
Make sure you invoke reportAll before the test completes, otherwise any failures that were detected by the
Checkpoint will not be reported.
Note that a Checkpoint will catch and record for later reporting (via reportAll) exceptions that mix in StackDepth
except for TestCanceledException, TestRegistrationClosedException, NotAllowedException,
and DuplicateTestNameException. If a block of code passed to a Checkpoint's apply method completes
abruptly with any of the StackDepth exceptions in the previous list, or any non-StackDepth exception, that invocation
of the apply method will complete abruptly with the same exception immediately. Unless you put reportAll in a finally
clause and handle this case, such an unexpected exception will cause you to lose any information about assertions that failed earlier in the test and were
recorded by the Checkpoint.
Trait that provides a complete-lastly construct, which ensures
cleanup code in lastly is executed whether the code passed to complete
completes abruptly with an exception or successfully results in a Future,
FutureOutcome, or other type with an
implicit Futuristic instance.
Trait that provides a complete-lastly construct, which ensures
cleanup code in lastly is executed whether the code passed to complete
completes abruptly with an exception or successfully results in a Future,
FutureOutcome, or other type with an
implicit Futuristic instance.
This trait is mixed into ScalaTest's async testing styles, to make it easy to ensure
cleanup code will execute whether code that produces a "futuristic" value (any type F
for which a Futuristic[F] instance is implicitly available). ScalaTest provides
implicit Futuristic instances for Future[T] for any type T
and FutureOutcome.
If the future-producing code passed to complete throws an
exception, the cleanup code passed to lastly will be executed immediately, and the same exception will
be rethrown, unless the code passed to lastly also completes abruptly with an exception. In that case,
complete-lastly will complete abruptly with the exception thrown by the code passed to
lastly (this mimics the behavior of finally).
Otherwise, if the code passed to complete successfully returns a Future (or other "futuristic" type),
complete-lastly
will register the cleanup code to be performed once the future completes and return a new future that will complete
once the original future completes and the subsequent cleanup code has completed execution. The future returned by
complete-lastly will have the same result as the original future passed to complete,
unless the cleanup code throws an exception. If the cleanup code passed to lastly throws
an exception, the future returned by lastly will fail with that exception.
The complete-lastly syntax is intended to be used to ensure cleanup code is executed
in async testing styles like try-finally is used in traditional testing styles.
Here's an example of complete-lastly
used in withFixture in an async testing style:
// Your implementation
override def withFixture(test: NoArgAsyncTest) = {
// Perform setup here
complete {
super.withFixture(test) // Invoke the test function
} lastly {
// Perform cleanup here
}
}
Composite Status that aggregates its completion and failed states of set of other Statuses passed to its constructor.
Composite Status that aggregates its completion and failed states of set of other Statuses passed to its constructor.
A map of configuration data.
A map of configuration data.
A ConfigMap can be populated from the Runner command line via -D
arguments. Runner passes it to many methods where you can use it to configure your
test runs. For example, Runner passed the ConfigMap to:
apply method of Reporters via RunStarting eventsrun method of SuiterunNestedSuites method of SuiterunTests method of SuiterunTest method of SuitewithFixture(NoArgTest) method of SuitewithFixture(OneArgTest) method of fixture.SuitebeforeEach(TestData) method of BeforeAndAfterEachTestDataafterEach(TestData) method of BeforeAndAfterEachTestDataIn addition to accessing the ConfigMap in overriden implementations of the above methods, you can also transform
and pass along a modified ConfigMap.
A ConfigMap maps string keys to values of any type, i.e., it is a Map[String, Any].
To get a configuration value in a variable of the actual type of that value, therefore, you'll need to perform an unsafe cast. If
this cast fails, you'll get an exception, which so long as the ConfigMap is used only in tests, will
result in either a failed or canceled test or aborted suite. To give such exceptions nice stack depths and error messages, and to
eliminate the need for using asInstanceOf in your test code, ConfigMap provides three
methods for accessing values at expected types.
The getRequired method returns the value bound to a key cast to a specified type, or throws TestCanceledException
if either the key is not bound or is bound to an incompatible type. Here's an example:
val tempFileName: String = configMap.getRequired[String]("tempFileName")
The getOptional method returns the value bound to a key cast to a specified type, wrapped in a Some,
returns None if the key is not bound, or throws TestCanceledException if the key exists but is
bound to an incompatible type. Here's an example:
val tempFileName: Option[String] = configMap.getOptional[String]("tempFileName")
The getWithDefault method returns the value bound to a key cast to a specified type,
returns a specified default value if the key is not bound, or throws TestCanceledException if the key exists but is
either not bound or is bound to an incompatible type. Here's an example:
val tempFileName: String = configMap.getWithDefault[String]("tempFileName", "tmp.txt")
Sub-trait of Assertions that override assert and assume methods to include
a diagram showing the values of expression in the error message when the assertion or assumption fails.
Sub-trait of Assertions that override assert and assume methods to include
a diagram showing the values of expression in the error message when the assertion or assumption fails.
Here are some examples:
scala> import DiagrammedAssertions._
import DiagrammedAssertions._
scala> assert(a == b || c >= d)
org.scalatest.exceptions.TestFailedException:
assert(a == b || c >= d)
| | | | | | |
1 | 2 | 3 | 4
| | false
| false
false
at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
...
scala> assert(xs.exists(_ == 4))
org.scalatest.exceptions.TestFailedException:
assert(xs.exists(_ == 4))
| |
| false
List(1, 2, 3)
at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
...
scala> assert("hello".startsWith("h") && "goodbye".endsWith("y"))
org.scalatest.exceptions.TestFailedException:
assert("hello".startsWith("h") && "goodbye".endsWith("y"))
| | | | | | |
"hello" true "h" | "goodbye" false "y"
false
at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
...
scala> assert(num.isInstanceOf[Int])
org.scalatest.exceptions.TestFailedException:
assert(num.isInstanceOf[Int])
| |
1.0 false
at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
...
scala> assert(Some(2).isEmpty)
org.scalatest.exceptions.TestFailedException:
assert(Some(2).isEmpty)
| | |
| 2 false
Some(2)
at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
...
scala> assert(None.isDefined)
org.scalatest.exceptions.TestFailedException:
assert(None.isDefined)
| |
None false
at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
...
scala> assert(xs.exists(i => i > 10))
org.scalatest.exceptions.TestFailedException:
assert(xs.exists(i => i > 10))
| |
| false
List(1, 2, 3)
at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
...
If the expression passed to assert or assume spans more than one line, DiagrammedAssertions falls
back to the default style of error message, since drawing a diagram would be difficult. Here's an example showing how
DiagrammedAssertions will treat a multi-line assertion (i.e., you don't get a diagram):
scala> assert("hello".startsWith("h") &&
| "goodbye".endsWith("y"))
org.scalatest.exceptions.TestFailedException: "hello" started with "h", but "goodbye" did not end with "y"
at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
...
Also, since an expression diagram essentially represents multi-line ascii art, if a clue string is provided, it appears above the diagram, not after it. It will often also show up in the diagram:
scala> assert(None.isDefined, "Don't do this at home")
org.scalatest.exceptions.TestFailedException: Don't do this at home
assert(None.isDefined, "Don't do this at home")
| |
None false
at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
...
scala> assert(None.isDefined,
| "Don't do this at home")
org.scalatest.exceptions.TestFailedException: Don't do this at home
assert(None.isDefined,
| |
None false
at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
...
Trait DiagrammedAssertions was inspired by Peter Niederwieser's work in Spock and Expecty.
A trait that represent an expression recorded by DiagrammedExprMacro, which includes the following members:
A trait that represent an expression recorded by DiagrammedExprMacro, which includes the following members:
DiagrammedExpr is used by code generated from DiagrammedAssertionsMacro, it needs to be public
so that the generated code can be compiled. It is expected that ScalaTest users would ever need to use DiagrammedExpr
directly.
A sorter for the events of a run's distributed suites.
A sorter for the events of a run's distributed suites.
This trait is used, for example, when -PS is passed to Runner, to sort the
events of distributed suites such that each suite's events are propagated together, with a timeout if an event takes too long.
A sorter for the events of a suite's distributed tests.
A sorter for the events of a suite's distributed tests.
This trait is used, for example, by ParallelTestExecution to sort the
events of tests back into sequential order, with a timeout if an event takes too long.
Trait whose instances facilitate parallel execution of Suites.
Trait whose instances facilitate parallel execution of Suites.
An optional Distributor is passed to the run method of Suite. If a
Distributor is indeed passed, trait Suite's implementation of run will
populate that Distributor with its nested Suites (by passing them to the Distributor's
apply method) rather than executing the nested Suites directly. It is then up to another thread or process
to execute those Suites.
If you have a set of nested Suites that must be executed sequentially, you can mix in trait
SequentialNestedSuiteExecution, which overrides runNestedSuites and
calls super's runNestedSuites implementation, passing in None for the
Distributor.
Implementations of this trait must be thread safe.
Trait to which markup text tests can be reported.
Trait to which markup text tests can be reported.
Note: Documenter will be described in more detail in a future 2.0 milestone release. As of this release
you can't see its effects yet.
Trait that contains a markup method, which can be used to send markup to the Reporter.
Trait that contains a markup method, which can be used to send markup to the Reporter.
Dynamic tags for a run.
Dynamic tags for a run.
Instances of this class are passed to the Filter constructor to
support running selected suites and tests via dynamic tagging. For example, dynamic tags can be used
to rerun tests that failed previously, or tests selected via a wildcard from Runner or
the Scala interpreter.
a map from String suite ID to a set of tags for that suite.
a map from String suite ID to a map, whose keys are test names and values the tags for that test.
NullPointerException if either suiteTags or testTags is null
Trait that provides an implicit conversion that adds left.value and right.value methods
to Either, which will return the selected value of the Either if defined,
or throw TestFailedException if not.
Trait that provides an implicit conversion that adds left.value and right.value methods
to Either, which will return the selected value of the Either if defined,
or throw TestFailedException if not.
This construct allows you to express in one statement that an Either should be left or right
and that its value should meet some expectation. Here's are some examples:
either1.right.value should be > 9
either2.left.value should be ("Muchas problemas")
Or, using assertions instead of matcher expressions:
assert(either1.right.value > 9) assert(either2.left.value === "Muchas problemas")
Were you to simply invoke right.get or left.get on the Either,
if the Either wasn't defined as expected (e.g., it was a Left when you expected a Right), it
would throw a NoSuchElementException:
val either: Either[String, Int] = Left("Muchas problemas")
either.right.get should be > 9 // either.right.get throws NoSuchElementException
The NoSuchElementException would cause the test to fail, but without providing a stack depth pointing
to the failing line of test code. This stack depth, provided by TestFailedException (and a
few other ScalaTest exceptions), makes it quicker for
users to navigate to the cause of the failure. Without EitherValues, to get
a stack depth exception you would need to make two statements, like this:
val either: Either[String, Int] = Left("Muchas problemas")
either should be ('right) // throws TestFailedException
either.right.get should be > 9
The EitherValues trait allows you to state that more concisely:
val either: Either[String, Int] = Left("Muchas problemas")
either.right.value should be > 9 // either.right.value throws TestFailedException
A case class implementation of java.util.Map.Entry to make it easier to
test Java Maps with ScalaTest Matchers.
A case class implementation of java.util.Map.Entry to make it easier to
test Java Maps with ScalaTest Matchers.
In Java, java.util.Map is not a subtype of java.util.Collection, and does not
actually define an element type. You can ask a Java Map for an “entry set”
via the entrySet method, which will return the Map's key/value pairs
wrapped in a set of java.util.Map.Entry, but a Map is not actually
a collection of Entry. To make Java Maps easier to work with, however,
ScalaTest matchers allows you to treat a Java Map as a collection of Entry,
and defines this convenience implementation of java.util.Map.Entry.
Here's how you use it:
javaMap should contain (Entry(2, 3)) javaMap should contain oneOf (Entry(2, 3), Entry(3, 4))
the key of this entry
the value of this entry
Superclass for the two outcomes of running a test that contain an exception: Failed and Canceled.
Superclass for the two outcomes of running a test that contain an exception: Failed and Canceled.
This class provides a toOption method that returns a Some wrapping the contained exception, and
an isExceptional field with the value true. It's companion object provides an extractor that
enables patterns that match a test that either failed or canceled, as in:
outcome match {
case Exceptional(ex) => // handle failed or canceled case
case _ => // handle succeeded, pending, or omitted case
}
Outcome for a test that failed, containing an exception describing the cause of the failure.
Outcome for a test that failed, containing an exception describing the cause of the failure.
Note: the difference between this Failed class and the similarly named FailedStatus
object is that an instance of this class indicates one test failed, whereas the FailedStatus object indicates either one or more tests failed
and/or one or more suites aborted during a run. Both are used as the result type of Suite lifecycle methods, but Failed
is a possible result of withFixture, whereas FailedStatus is a possible result of run, runNestedSuites,
runTests, or runTest. In short, Failed is always just about one test, whereas FailedStatus could be
about something larger: multiple tests or an entire suite.
Filter whose apply method determines which of the passed tests to run and ignore based on tags to include and exclude passed as
as class parameters.
Filter whose apply method determines which of the passed tests to run and ignore based on tags to include and exclude passed as
as class parameters.
This class handles the org.scalatest.Ignore tag specially, in that its apply method indicates which
tests should be ignored based on whether they are tagged with org.scalatest.Ignore. If
"org.scalatest.Ignore" is not passed in the tagsToExclude set, it will be implicitly added. However, if the
tagsToInclude option is defined, and the contained set does not include "org.scalatest.Ignore", then only those tests
that are both tagged with org.scalatest.Ignore and at least one of the tags in the tagsToInclude set
will be included in the result of apply and marked as ignored (so long as the test is not also
marked with a tag other than org.scalatest.Ignore that is a member of the tagsToExclude
set. For example, if SlowAsMolasses is a member of the tagsToInclude set and a
test is tagged with both org.scalatest.Ignore and SlowAsMolasses, and
SlowAsMolasses appears in the tagsToExclude set, the
SlowAsMolasses tag will "overpower" the org.scalatest.Ignore tag, and the
test will be filtered out entirely rather than being ignored.
IllegalArgumentException if tagsToInclude is defined, but contains an empty set
NullArgumentException if either tagsToInclude or tagsToExclude are null
Marker trait for fixture-context objects, that enables them
to be used in testing styles that require type Assertion
Marker trait for fixture-context objects, that enables them
to be used in testing styles that require type Assertion
A fixture-context object is a way to share fixtures between different tests that is most useful when different tests need different combinations of fixture objects. The fixture-context object technique is only appropriate if you don't need to clean up the fixtures after using them.
To use this technique, you define instance variables intialized with fixture
objects in traits and/or classes, then in each test instantiate an object that
contains just the fixture objects needed by the test. Traits allow you to mix
together just the fixture objects needed by each test, whereas classes
allow you to pass data in via a constructor to configure the fixture objects.
Here's an example FlatSpec in which fixture objects are partitioned
into two traits and each test just mixes together the traits it needs:
package org.scalatest.examples.flatspec.fixturecontext
import collection.mutable.ListBuffer
import org.scalatest.FlatSpec
import org.scalatest.FixtureContext
class ExampleSpec extends FlatSpec {
trait Builder extends FixtureContext {
val builder = new StringBuilder("ScalaTest is ")
}
trait Buffer extends FixtureContext {
val buffer = ListBuffer("ScalaTest", "is")
}
// This test needs the StringBuilder fixture
"Testing" should "be productive" in new Builder {
builder.append("productive!")
assert(builder.toString === "ScalaTest is productive!")
}
// This test needs the ListBuffer[String] fixture
"Test code" should "be readable" in new Buffer {
buffer += ("readable!")
assert(buffer === List("ScalaTest", "is", "readable!"))
}
// This test needs both the StringBuilder and ListBuffer
it should "be clear and concise" in new Builder with Buffer {
builder.append("clear!")
buffer += ("concise!")
assert(builder.toString === "ScalaTest is clear!")
assert(buffer === List("ScalaTest", "is", "concise!"))
}
}
Extending FixtureContext, which extends trait org.scalatest.compatible.Assertion makes
it more convenient to use fixture-context objects in styles, such as async styles, that require test bodies
to have type Assertion.
Wrapper class for Future[Outcome] that presents a more convenient API for
manipulation in withFixture methods in async styles.
Wrapper class for Future[Outcome] that presents a more convenient API for
manipulation in withFixture methods in async styles.
This type serves as the result type of both test functions and withFixture methods
in ScalaTest's async styles. A Future[Outcome] is not used as this result type
for two reasons. First, Outcome treats exceptions specially, and as a result
methods on Future would usually not yield the desired Future[Outcome] result.
Only run-aborting exceptions should result in a failed Future[Outcome]. Any other thrown exception
other than TestCanceledException or TestPendingException
should result in a successfulFuture containing a org.scalatest.Failed.
A thrown TestCanceledException should result in a successful Future
containing an org.scalatest.Canceled; A thrown TestPendingException should result in
a successful Future containing a org.scalatest.Pending.
If manipulating a Future[Outcome] directly, by contrast, any thrown exception would result in
a failed Future.
Additionally, to be consistent with corresponding transformations in traditional testing styles,
methods registering callbacks should return a new future outcome that doesn't complete until
both the original future outcome has completed and the subsequent callback has completed execution.
Additionally, if the callback itself throws an exception, that exception should determine the result
of the future outcome returned by the callback registration method. This behavior is rather inconvenient
to obtain on the current Future API, so FutureOutcome provides well-named
methods that have this behavior.
Lastly, the FutureOutcome is intended to help prevent confusion by eliminating the need
to work with types like scala.util.Success(org.scalatest.Failed). For this purpose a
org.scalactic.Or is used instead of a scala.util.Try to describe results
of FutureOutcome.
A FutureOutcome represents a computation that can result in an Outcome or an "abort." An abort means
that a run-aborting exception occurred during the computation. Any other, non-run-aborting exception will be represented
as an non-Succeeded Outcome: one of Failed, Canceled, or Pending.
The methods of FutureOutcome include the following callback registration methods:
onSucceededThen - registers a callback to be executed if the future outcome is Succeeded.onFailedThen - registers a callback to be executed if the future outcome is Failed.onCanceledThen - registers a callback to be executed if the future outcome is Canceled.onPendingThen - registers a callback to be executed if the future outcome is Pending.onOutcomeThen - registers a callback to be executed if the future outcome is actually an Outcome
and not an abort.onAbortedThen - registers a callback to be executed if the future outcome aborts.onCompletedThen - registers a callback to be executed upon completion no matter how the future outcome completes.The callback methods listed previously can be used to perform a side effect once a FutureOutcome completes. To change an
Outcome into a different Outcome asynchronously, use the change registration method, which takes a function
from Outcome to Outcome. The other methods on FutureOutcome, isCompleted and
value, allow you to poll a FutureOutcome. None of the methods on FutureOutcome block.
Lastly, because an implicit Futuristic instance is provided for
FutureOutcome, you can use complete-lastly syntax
with FutureOutcome.
Trait that contains methods named given, when, then, and and,
which take a string message and implicit Informer, and forward the message to the informer.
Trait that contains methods named given, when, then, and and,
which take a string message and implicit Informer, and forward the message to the informer.
Here's an example:
package org.scalatest.examples.flatspec.info
import collection.mutable
import org.scalatest._
class SetSpec extends FlatSpec with GivenWhenThen {
"A mutable Set" should "allow an element to be added" in {
Given("an empty mutable Set")
val set = mutable.Set.empty[String]
When("an element is added")
set += "clarity"
Then("the Set should have size 1")
assert(set.size === 1)
And("the Set should contain the added element")
assert(set.contains("clarity"))
info("That's all folks!")
}
}
If you run this SetSpec from the interpreter, you will see the following output:
scala> org.scalatest.run(new SetSpec)
A mutable Set
- should allow an element to be added
+ Given an empty mutable Set
+ When an element is added
+ Then the Set should have size 1
+ And the Set should contain the added element
+ That's all folks!
Trait to which custom information about a running suite of tests can be reported.
Trait to which custom information about a running suite of tests can be reported.
An Informer is essentially
used to wrap a Reporter and provide easy ways to send custom information
to that Reporter via an InfoProvided event.
Informer contains an apply method that takes a string and
an optional payload object of type Any.
The Informer will forward the passed message string to the
Reporter as the message parameter, and the optional
payload object as the payload parameter, of an InfoProvided event.
Here's an example in which the Informer is used both directly via info
method of trait FlatSpec and indirectly via the methods of
trait GivenWhenThen:
package org.scalatest.examples.flatspec.info
import collection.mutable
import org.scalatest._
class SetSpec extends FlatSpec with GivenWhenThen {
"A mutable Set" should "allow an element to be added" in {
given("an empty mutable Set")
val set = mutable.Set.empty[String]
when("an element is added")
set += "clarity"
then("the Set should have size 1")
assert(set.size === 1)
and("the Set should contain the added element")
assert(set.contains("clarity"))
info("That's all folks!")
}
}
If you run this SetSpec from the interpreter, you will see the following output:
scala> org.scalatest.run(new SetSpec)
A mutable Set
- should allow an element to be added
+ Given an empty mutable Set
+ When an element is added
+ Then the Set should have size 1
+ And the Set should contain the added element
+ That's all folks!
Trait that contains the info method, which can be used to send info to the reporter.
Trait that contains the info method, which can be used to send info to the reporter.
Trait containing the inside construct, which allows you to make statements about nested object graphs using pattern matching.
Trait containing the inside construct, which allows you to make statements about nested object graphs using pattern matching.
For example, given the following case classes:
case class Address(street: String, city: String, state: String, zip: String) case class Name(first: String, middle: String, last: String) case class Record(name: Name, address: Address, age: Int)
You could write:
inside (rec) { case Record(name, address, age) =>
inside (name) { case Name(first, middle, last) =>
first should be ("Sally")
middle should be ("Ann")
last should be ("Jones")
}
inside (address) { case Address(street, city, state, zip) =>
street should startWith ("25")
city should endWith ("Angeles")
state should equal ("CA")
zip should be ("12345")
}
age should be < 99
}
If an assertion fails, the error message will include the toString of each value passed
to inside clauses enclosing the failed assertion. For example, if rec in
the previous expression was defined like this:
val rec = Record(
Name("Sally", "Anna", "Jones"),
Address("25 Main St", "Los Angeles", "CA", "12345"),
38
)
The error message will read:
"Ann[a]" was not equal to "Ann[]", inside Name(Sally,Anna,Jones), inside Record(Name(Sally,Anna,Jones),Address(25 Main St,Los Angeles,CA,12345),38)
Provides nestable inspector methods (or just inspectors) that enable assertions to be made about collections.
Provides nestable inspector methods (or just inspectors) that enable assertions to be made about collections.
For example, the forAll method enables you to state that something should be true about all elements of a collection, such
as that all elements should be positive:
scala> import org.scalatest._
import org.scalatest._
scala> import Assertions._
import Assertions._
scala> import Inspectors._
import Inspectors._
scala> val xs = List(1, 2, 3, 4, 5)
xs: List[Int] = List(1, 2, 3, 4, 5)
scala> forAll (xs) { x => assert(x > 0) }
Or, with matchers:
scala> import Matchers._
import Matchers._
scala> forAll (xs) { x => x should be > 0 }
To make assertions about nested collections, you can nest the inspector method invocations.
For example, given the following list of lists of Int:
scala> val yss =
| List(
| List(1, 2, 3),
| List(1, 2, 3),
| List(1, 2, 3)
| )
yss: List[List[Int]] = List(List(1, 2, 3), List(1, 2, 3), List(1, 2, 3))
You can assert that all Int elements in all nested lists are positive by nesting two forAll method invocations, like this:
scala> forAll (yss) { ys =>
| forAll (ys) { y => y should be > 0 }
| }
The full list of inspector methods are:
forAll - succeeds if the assertion holds true for every elementforAtLeast - succeeds if the assertion holds true for at least the specified number of elementsforAtMost - succeeds if the assertion holds true for at most the specified number of elementsforBetween - succeeds if the assertion holds true for between the specified minimum and maximum number of elements, inclusiveforEvery - same as forAll, but lists all failing elements if it fails (whereas forAll just reports the first failing element)forExactly - succeeds if the assertion holds true for exactly the specified number of elementsThe error messages produced by inspector methods are designed to make sense no matter how deeply you nest the method invocations. Here's an example of a nested inspection that fails and the resulting error message:
scala> forAll (yss) { ys =>
| forAll (ys) { y => y should be < 2 }
| }
org.scalatest.exceptions.TestFailedException: forAll failed, because:
at index 0, forAll failed, because:
at index 1, 2 was not less than 2 (<console>:20)
in List(1, 2, 3) (<console>:20)
in List(List(1, 2, 3), List(1, 2, 3), List(1, 2, 3))
at org.scalatest.InspectorsHelper$.forAll(Inspectors.scala:146)
...
One way the error message is designed to help you understand the error is by using indentation that mimics the indentation of the
source code (optimistically assuming the source will be nicely indented). The error message above indicates the outer forAll failed
because its initial List (i.e., at index 0) failed
the assertion, which was that all elements of that initial List[Int] at index 0 should be less than 2. This assertion failed because index 1 of
that inner list contained the value 2, which was indeed “not less than 2.” The error message for the inner list is an indented line inside the error message
for the outer list. The actual contents of each list are displayed at the end in inspector error messages, also indented appropriately. The actual contents
are placed at the end so that for very large collections, the contents will not drown out and make it difficult to find the messages that describe
actual causes of the failure.
The forAll and forEvery methods are similar in that both succeed only if the assertion holds for all elements of the collection.
They differ in that forAll will only report the first element encountered that failed the assertion, but forEvery will report all
elements that fail the assertion. The tradeoff is that while forEvery gives more information, it may take longer to run because it must inspect every element
of the collection. The forAll method can simply stop inspecting once it encounters the first failing element. Here's an example that
shows the difference in the forAll and forEvery error messages:
scala> forAll (xs) { x => x should be < 3 }
org.scalatest.exceptions.TestFailedException: forAll failed, because:
at index 2, 3 was not less than 3 (<console>:18)
in List(1, 2, 3, 4, 5)
at org.scalatest.InspectorsHelper$.forAll(Inspectors.scala:146)
...
scala> forEvery (xs) { x => x should be < 3 }
org.scalatest.exceptions.TestFailedException: forEvery failed, because:
at index 2, 3 was not less than 3 (<console>:18),
at index 3, 4 was not less than 3 (<console>:18),
at index 4, 5 was not less than 3 (<console>:18)
in List(1, 2, 3, 4, 5)
at org.scalatest.InspectorsHelper$.forEvery(Inspectors.scala:226)
...
Note that if you're using matchers, you can alternatively use inspector shorthands for writing non-nested inspections. Here's an example:
scala> all (xs) should be > 3
org.scalatest.exceptions.TestFailedException: 'all' inspection failed, because:
at index 0, 1 was not greater than 3
in List(1, 2, 3, 4, 5)
at org.scalatest.InspectorsHelper$.forAll(Inspectors.scala:146)
You can use Inspectors on any scala.collection.GenTraversable, java.util.Collection,
java.util.Map (with Entry), Array, or String.
Here are some examples:
scala> import org.scalatest._
import org.scalatest._
scala> import Inspectors._
import Inspectors._
scala> import Matchers._
import Matchers._
scala> forAll (Array(1, 2, 3)) { e => e should be < 5 }
scala> import collection.JavaConverters._
import collection.JavaConverters._
scala> val js = List(1, 2, 3).asJava
js: java.util.List[Int] = [1, 2, 3]
scala> forAll (js) { j => j should be < 5 }
scala> val jmap = Map("a" -> 1, "b" -> 2).asJava
jmap: java.util.Map[String,Int] = {a=1, b=2}
scala> forAtLeast(1, jmap) { e => e shouldBe Entry("b", 2) }
scala> forAtLeast(2, "hello, world!") { c => c shouldBe 'o' }
Trait that provides an implicit conversion that adds to collection types a loneElement method, which
will return the value of the lone element if the collection does
indeed contain one and only one element, or throw TestFailedException if not.
Trait that provides an implicit conversion that adds to collection types a loneElement method, which
will return the value of the lone element if the collection does
indeed contain one and only one element, or throw TestFailedException if not.
This construct allows you to express in one statement that a collection should contain one and only one element and that the element value should meet some expectation. Here's an example:
set.loneElement should be > 9
Or, using an assertion instead of a matcher expression:
assert(set.loneElement > 9)
The loneElement syntax can be used with any collection type C for which an
implicit Collecting[C] is available. ScalaTest provides
implicit Collecting instances for scala.collection.GenTraversable, Array,
and java.util.Collection. You can enable the loneElement
syntax on other collection types by defining an implicit Collecting instances for those types.
If you want to use loneElement with a java.util.Map, first transform it to a
set of entries with entrySet, and if helpful, use ScalaTest's Entry class:
scala> import org.scalatest._
import org.scalatest._
scala> import LoneElement._
import LoneElement._
scala> import Matchers._
import Matchers._
scala> val jmap = new java.util.HashMap[String, Int]
jmap: java.util.HashMap[String,Int] = {}
scala> jmap.put("one", 1)
res0: Int = 0
scala> jmap.entrySet.loneElement should be (Entry("one", 1))
Trait that provides a domain specific language (DSL) for expressing assertions in tests
using the word should.
Trait that provides a domain specific language (DSL) for expressing assertions in tests
using the word should.
For example, if you mix Matchers into
a suite class, you can write an equality assertion in that suite like this:
result should equal (3)
Here result is a variable, and can be of any type. If the object is an
Int with the value 3, execution will continue (i.e., the expression will result
in the unit value, ()). Otherwise, a TestFailedException
will be thrown with a detail message that explains the problem, such as "7 did not equal 3".
This TestFailedException will cause the test to fail.
Here is a table of contents for this documentation:
Boolean properties with beBeMatchersStrings and Arrays as collectionsand and orOptionshavelength and size with HavePropertyMatchersTrait MustMatchers is an alternative to Matchers that provides the exact same
meaning, syntax, and behavior as Matchers, but uses the verb must instead of should.
The two traits differ only in the English semantics of the verb: should
is informal, making the code feel like conversation between the writer and the reader; must is more formal, making the code feel more like
a written specification.
ScalaTest matchers provides five different ways to check equality, each designed to address a different need. They are:
result should equal (3) // can customize equality result should === (3) // can customize equality and enforce type constraints result should be (3) // cannot customize equality, so fastest to compile result shouldEqual 3 // can customize equality, no parentheses required result shouldBe 3 // cannot customize equality, so fastest to compile, no parentheses required
The “left should equal (right)” syntax requires an
org.scalactic.Equality[L] to be provided (either implicitly or explicitly), where
L is the left-hand type on which should is invoked. In the "left should equal (right)" case,
for example, L is the type of left. Thus if left is type Int, the "left should
equal (right)"
statement would require an Equality[Int].
By default, an implicit Equality[T] instance is available for any type T, in which equality is implemented
by simply invoking == on the left
value, passing in the right value, with special treatment for arrays. If either left or right is an array, deep
will be invoked on it before comparing with ==. Thus, the following expression
will yield false, because Array's equals method compares object identity:
Array(1, 2) == Array(1, 2) // yields false
The next expression will by default not result in a TestFailedException, because default Equality[Array[Int]] compares
the two arrays structurally, taking into consideration the equality of the array's contents:
Array(1, 2) should equal (Array(1, 2)) // succeeds (i.e., does not throw TestFailedException)
If you ever do want to verify that two arrays are actually the same object (have the same identity), you can use the
be theSameInstanceAs syntax, described below.
You can customize the meaning of equality for a type when using "should equal," "should ===,"
or shouldEqual syntax by defining implicit Equality instances that will be used instead of default Equality.
You might do this to normalize types before comparing them with ==, for instance, or to avoid calling the == method entirely,
such as if you want to compare Doubles with a tolerance.
For an example, see the main documentation of trait Equality.
You can always supply implicit parameters explicitly, but in the case of implicit parameters of type Equality[T], Scalactic provides a
simple "explictly" DSL. For example, here's how you could explicitly supply an Equality[String] instance that normalizes both left and right
sides (which must be strings), by transforming them to lowercase:
scala> import org.scalatest.Matchers._
import org.scalatest.Matchers._
scala> import org.scalactic.Explicitly._
import org.scalactic.Explicitly._
scala> import org.scalactic.StringNormalizations._
import org.scalactic.StringNormalizations._
scala> "Hi" should equal ("hi") (after being lowerCased)
The after being lowerCased expression results in an Equality[String], which is then passed
explicitly as the second curried parameter to equal. For more information on the explictly DSL, see the main documentation
for trait Explicitly.
The "should be" and shouldBe syntax do not take an Equality[T] and can therefore not be customized.
They always use the default approach to equality described above. As a result, "should be" and shouldBe will
likely be the fastest-compiling matcher syntax for equality comparisons, since the compiler need not search for
an implicit Equality[T] each time.
The should === syntax (and its complement, should !==) can be used to enforce type
constraints at compile-time between the left and right sides of the equality comparison. Here's an example:
scala> import org.scalatest.Matchers._
import org.scalatest.Matchers._
scala> import org.scalactic.TypeCheckedTripleEquals._
import org.scalactic.TypeCheckedTripleEquals._
scala> Some(2) should === (2)
<console>:17: error: types Some[Int] and Int do not adhere to the equality constraint
selected for the === and !== operators; the missing implicit parameter is of
type org.scalactic.CanEqual[Some[Int],Int]
Some(2) should === (2)
^
By default, the "Some(2) should === (2)" statement would fail at runtime. By mixing in
the equality constraints provided by TypeCheckedTripleEquals, however, the statement fails to compile. For more information
and examples, see the main documentation for trait TypeCheckedTripleEquals.
You can check the size or length of any type of object for which it makes sense. Here's how checking for length looks:
result should have length 3
Size is similar:
result should have size 10
The length syntax can be used with String, Array, any scala.collection.GenSeq,
any java.util.List, and any type T for which an implicit Length[T] type class is
available in scope.
Similarly, the size syntax can be used with Array, any scala.collection.GenTraversable,
any java.util.Collection, any java.util.Map, and any type T for which an implicit Size[T] type class is
available in scope. You can enable the length or size syntax for your own arbitrary types, therefore,
by defining Length or Size type
classes for those types.
In addition, the length syntax can be used with any object that has a field or method named length
or a method named getLength. Similarly, the size syntax can be used with any
object that has a field or method named size or a method named getSize.
The type of a length or size field, or return type of a method, must be either Int
or Long. Any such method must take no parameters. (The Scala compiler will ensure at compile time that
the object on which should is being invoked has the appropriate structure.)
You can check for whether a string starts with, ends with, or includes a substring like this:
string should startWith ("Hello")
string should endWith ("world")
string should include ("seven")
You can check for whether a string starts with, ends with, or includes a regular expression, like this:
string should startWith regex "Hel*o" string should endWith regex "wo.ld" string should include regex "wo.ld"
And you can check whether a string fully matches a regular expression, like this:
string should fullyMatch regex """(-)?(\d+)(\.\d*)?"""
The regular expression passed following the regex token can be either a String
or a scala.util.matching.Regex.
With the startWith, endWith, include, and fullyMatch
tokens can also be used with an optional specification of required groups, like this:
"abbccxxx" should startWith regex ("a(b*)(c*)" withGroups ("bb", "cc"))
"xxxabbcc" should endWith regex ("a(b*)(c*)" withGroups ("bb", "cc"))
"xxxabbccxxx" should include regex ("a(b*)(c*)" withGroups ("bb", "cc"))
"abbcc" should fullyMatch regex ("a(b*)(c*)" withGroups ("bb", "cc"))
You can check whether a string is empty with empty:
s shouldBe empty
You can also use most of ScalaTest's matcher syntax for collections on String by
treating the Strings as collections of characters. For examples, see the
Strings and Arrays as collections section below.
You can check whether any type for which an implicit Ordering[T] is available
is greater than, less than, greater than or equal, or less
than or equal to a value of type T. The syntax is:
one should be < 7 one should be > 0 one should be <= 7 one should be >= 0
Boolean properties with be If an object has a method that takes no parameters and returns boolean, you can check
it by placing a Symbol (after be) that specifies the name
of the method (excluding an optional prefix of "is"). A symbol literal
in Scala begins with a tick mark and ends at the first non-identifier character. Thus,
'traversableAgain results in a Symbol object at runtime, as does
'completed and 'file. Here's an example:
iter shouldBe 'traversableAgain
Given this code, ScalaTest will use reflection to look on the object referenced from
emptySet for a method that takes no parameters and results in Boolean,
with either the name empty or isEmpty. If found, it will invoke
that method. If the method returns true, execution will continue. But if it returns
false, a TestFailedException will be thrown that will contain a detail message, such as:
non-empty iterator was not traversableAgain
This be syntax can be used with any reference (AnyRef) type. If the object does
not have an appropriately named predicate method, you'll get a TestFailedException
at runtime with a detailed message that explains the problem.
(For the details on how a field or method is selected during this
process, see the documentation for BeWord.)
If you think it reads better, you can optionally put a or an after
be. For example, java.io.File has two predicate methods,
isFile and isDirectory. Thus with a File object
named temp, you could write:
temp should be a 'file
Or, given java.awt.event.KeyEvent has a method isActionKey that takes
no arguments and returns Boolean, you could assert that a KeyEvent is
an action key with:
keyEvent should be an 'actionKey
If you prefer to check Boolean properties in a type-safe manner, you can use a BePropertyMatcher.
This would allow you to write expressions such as:
xs shouldBe traversableAgain temp should be a file keyEvent should be an actionKey
These expressions would fail to compile if should is used on an inappropriate type, as determined
by the type parameter of the BePropertyMatcher being used. (For example, file in this example
would likely be of type BePropertyMatcher[java.io.File]. If used with an appropriate type, such an expression will compile
and at run time the Boolean property method or field will be accessed directly; i.e., no reflection will be used.
See the documentation for BePropertyMatcher for more information.
BeMatchers If you want to create a new way of using be, which doesn't map to an actual property on the
type you care about, you can create a BeMatcher. You could use this, for example, to create BeMatcher[Int]
called odd, which would match any odd Int, and even, which would match
any even Int.
Given this pair of BeMatchers, you could check whether an Int was odd or even with expressions like:
num shouldBe odd num should not be even
For more information, see the documentation for BeMatcher.
If you need to check that two references refer to the exact same object, you can write:
ref1 should be theSameInstanceAs ref2
If you need to check that an object is an instance of a particular class or trait, you can supply the type to
“be a” or “be an”:
result1 shouldBe a [Tiger] result1 should not be an [Orangutan]
Because type parameters are erased on the JVM, we recommend you insert an underscore for any type parameters
when using this syntax. Both of the following test only that the result is an instance of List[_], because at
runtime the type parameter has been erased:
result shouldBe a [List[_]] // recommended result shouldBe a [List[Fruit]] // discouraged
Often you may want to check whether a number is within a
range. You can do that using the +- operator, like this:
sevenDotOh should equal (6.9 +- 0.2) sevenDotOh should === (6.9 +- 0.2) sevenDotOh should be (6.9 +- 0.2) sevenDotOh shouldEqual 6.9 +- 0.2 sevenDotOh shouldBe 6.9 +- 0.2
Any of these expressions will cause a TestFailedException to be thrown if the floating point
value, sevenDotOh is outside the range 6.7 to 7.1.
You can use +- with any type T for which an implicit Numeric[T] exists, such as integral types:
seven should equal (6 +- 2) seven should === (6 +- 2) seven should be (6 +- 2) seven shouldEqual 6 +- 2 seven shouldBe 6 +- 2
You can check whether an object is "empty", like this:
traversable shouldBe empty javaMap should not be empty
The empty token can be used with any type L for which an implicit Emptiness[L] exists.
The Emptiness companion object provides implicits for GenTraversable[E], java.util.Collection[E],
java.util.Map[K, V], String, Array[E], and Option[E]. In addition, the
Emptiness companion object provides structural implicits for types that declare an isEmpty method that
returns a Boolean. Here are some examples:
scala> import org.scalatest.Matchers._
import org.scalatest.Matchers._
scala> List.empty shouldBe empty
scala> None shouldBe empty
scala> Some(1) should not be empty
scala> "" shouldBe empty
scala> new java.util.HashMap[Int, Int] shouldBe empty
scala> new { def isEmpty = true} shouldBe empty
scala> Array(1, 2, 3) should not be empty
You can check whether a collection contains a particular element like this:
traversable should contain ("five")
The contain syntax shown above can be used with any type C that has a "containing" nature, evidenced by
an implicit org.scalatest.enablers.Containing[L], where L is left-hand type on
which should is invoked. In the Containing
companion object, implicits are provided for types GenTraversable[E], java.util.Collection[E],
java.util.Map[K, V], String, Array[E], and Option[E].
Here are some examples:
scala> import org.scalatest.Matchers._
import org.scalatest.Matchers._
scala> List(1, 2, 3) should contain (2)
scala> Map('a' -> 1, 'b' -> 2, 'c' -> 3) should contain ('b' -> 2)
scala> Set(1, 2, 3) should contain (2)
scala> Array(1, 2, 3) should contain (2)
scala> "123" should contain ('2')
scala> Some(2) should contain (2)
ScalaTest's implicit methods that provide the Containing[L] type classes require an Equality[E], where
E is an element type. For example, to obtain a Containing[Array[Int]] you must supply an Equality[Int],
either implicitly or explicitly. The contain syntax uses this Equality[E] to determine containership.
Thus if you want to change how containership is determined for an element type E, place an implicit Equality[E]
in scope or use the explicitly DSL. Although the implicit parameter required for the contain syntax is of type Containing[L],
implicit conversions are provided in the Containing companion object from Equality[E] to the various
types of containers of E. Here's an example:
scala> import org.scalatest.Matchers._
import org.scalatest.Matchers._
scala> List("Hi", "Di", "Ho") should contain ("ho")
org.scalatest.exceptions.TestFailedException: List(Hi, Di, Ho) did not contain element "ho"
at ...
scala> import org.scalactic.Explicitly._
import org.scalactic.Explicitly._
scala> import org.scalactic.StringNormalizations._
import org.scalactic.StringNormalizations._
scala> (List("Hi", "Di", "Ho") should contain ("ho")) (after being lowerCased)
Note that when you use the explicitly DSL with contain you need to wrap the entire
contain expression in parentheses, as shown here.
(List("Hi", "Di", "Ho") should contain ("ho")) (after being lowerCased)
^ ^
In addition to determining whether an object contains another object, you can use contain to
make other determinations.
For example, the contain oneOf syntax ensures that one and only one of the specified elements are
contained in the containing object:
List(1, 2, 3, 4, 5) should contain oneOf (5, 7, 9)
Some(7) should contain oneOf (5, 7, 9)
"howdy" should contain oneOf ('a', 'b', 'c', 'd')
Note that if multiple specified elements appear in the containing object, oneOf will fail:
scala> List(1, 2, 3) should contain oneOf (2, 3, 4)
org.scalatest.exceptions.TestFailedException: List(1, 2, 3) did not contain one (and only one) of (2, 3, 4)
at ...
If you really want to ensure one or more of the specified elements are contained in the containing object,
use atLeastOneOf, described below, instead of oneOf. Keep in mind, oneOf
means "exactly one of."
Note also that with any contain syntax, you can place custom implicit Equality[E] instances in scope
to customize how containership is determined, or use the explicitly DSL. Here's an example:
(Array("Doe", "Ray", "Me") should contain oneOf ("X", "RAY", "BEAM")) (after being lowerCased)
If you have a collection of elements that you'd like to use in a "one of" comparison, you can use "oneElementOf," like this:
List(1, 2, 3, 4, 5) should contain oneElementOf List(5, 7, 9)
Some(7) should contain oneElementOf Vector(5, 7, 9)
"howdy" should contain oneElementOf Set('a', 'b', 'c', 'd')
(Array("Doe", "Ray", "Me") should contain oneElementOf List("X", "RAY", "BEAM")) (after being lowerCased)
The contain noneOf syntax does the opposite of oneOf: it ensures none of the specified elements
are contained in the containing object:
List(1, 2, 3, 4, 5) should contain noneOf (7, 8, 9)
Some(0) should contain noneOf (7, 8, 9)
"12345" should contain noneOf ('7', '8', '9')
If you have a collection of elements that you'd like to use in a "none of" comparison, you can use "noElementsOf," like this:
List(1, 2, 3, 4, 5) should contain noElementsOf List(7, 8, 9)
Some(0) should contain noElementsOf Vector(7, 8, 9)
"12345" should contain noElementsOf Set('7', '8', '9')
As mentioned, the "contain," "contain oneOf," and "contain noneOf" syntax requires a
Containing[L] be provided, where L is the left-hand type. Other contain syntax, which
will be described in this section, requires an Aggregating[L] be provided, where again L is the left-hand type.
(An Aggregating[L] instance defines the "aggregating nature" of a type L.)
The reason, essentially, is that contain syntax that makes sense for Option is enabled by
Containing[L], whereas syntax that does not make sense for Option is enabled
by Aggregating[L]. For example, it doesn't make sense to assert that an Option[Int] contains all of a set of integers, as it
could only ever contain one of them. But this does make sense for a type such as List[Int] that can aggregate zero to many integers.
The Aggregating companion object provides implicit instances of Aggregating[L]
for types GenTraversable[E], java.util.Collection[E],
java.util.Map[K, V], String, Array[E]. Note that these are the same types as are supported with
Containing, but with Option[E] missing.
Here are some examples:
The contain atLeastOneOf syntax, for example, works for any type L for which an Aggregating[L] exists. It ensures
that at least one of (i.e., one or more of) the specified objects are contained in the containing object:
List(1, 2, 3) should contain atLeastOneOf (2, 3, 4)
Array(1, 2, 3) should contain atLeastOneOf (3, 4, 5)
"abc" should contain atLeastOneOf ('c', 'a', 't')
Similar to Containing[L], the implicit methods that provide the Aggregating[L] instances require an Equality[E], where
E is an element type. For example, to obtain a Aggregating[Vector[String]] you must supply an Equality[String],
either implicitly or explicitly. The contain syntax uses this Equality[E] to determine containership.
Thus if you want to change how containership is determined for an element type E, place an implicit Equality[E]
in scope or use the explicitly DSL. Although the implicit parameter required for the contain syntax is of type Aggregating[L],
implicit conversions are provided in the Aggregating companion object from Equality[E] to the various
types of aggregations of E. Here's an example:
(Vector(" A", "B ") should contain atLeastOneOf ("a ", "b", "c")) (after being lowerCased and trimmed)
If you have a collection of elements that you'd like to use in an "at least one of" comparison, you can use "atLeastOneElementOf," like this:
List(1, 2, 3) should contain atLeastOneElementOf List(2, 3, 4)
Array(1, 2, 3) should contain atLeastOneElementOf Vector(3, 4, 5)
"abc" should contain atLeastOneElementOf Set('c', 'a', 't')
(Vector(" A", "B ") should contain atLeastOneElementOf List("a ", "b", "c")) (after being lowerCased and trimmed)
The "contain atMostOneOf" syntax lets you specify a set of objects at most one of which should be contained in the containing object:
List(1, 2, 3, 4, 5) should contain atMostOneOf (5, 6, 7)
If you have a collection of elements that you'd like to use in a "at most one of" comparison, you can use "atMostOneElementOf," like this:
List(1, 2, 3, 4, 5) should contain atMostOneElementOf Vector(5, 6, 7)
The "contain allOf" syntax lets you specify a set of objects that should all be contained in the containing object:
List(1, 2, 3, 4, 5) should contain allOf (2, 3, 5)
If you have a collection of elements that you'd like to use in a "all of" comparison, you can use "allElementsOf," like this:
List(1, 2, 3, 4, 5) should contain allElementsOf Array(2, 3, 5)
The "contain only" syntax lets you assert that the containing object contains only the specified objects, though it may
contain more than one of each:
List(1, 2, 3, 2, 1) should contain only (1, 2, 3)
The "contain theSameElementsAs" and "contain theSameElementsInOrderAs syntax differ from the others
in that the right hand side is a GenTraversable[_] rather than a varargs of Any. (Note: in a future 2.0 milestone release, possibly
2.0.M6, these will likely be widened to accept any type R for which an Aggregating[R] exists.)
The "contain theSameElementsAs" syntax lets you assert that two aggregations contain the same objects:
List(1, 2, 2, 3, 3, 3) should contain theSameElementsAs Vector(3, 2, 3, 1, 2, 3)
The number of times any family of equal objects appears must also be the same in both the left and right aggregations. The specified objects may appear multiple times, but must appear in the order they appear in the right-hand list. For example, if the last 3 element is left out of the right-hand list in the previous example, the expression would fail because the left side has three 3's and the right hand side has only two:
List(1, 2, 2, 3, 3, 3) should contain theSameElementsAs Vector(3, 2, 3, 1, 2)
org.scalatest.exceptions.TestFailedException: List(1, 2, 2, 3, 3, 3) did not contain the same elements as Vector(3, 2, 3, 1, 2)
at ...
Note that no onlyElementsOf matcher is provided, because it would have the same
behavior as theSameElementsAs. (I.e., if you were looking for onlyElementsOf, please use theSameElementsAs
instead.)
The rest of the contain syntax, which
will be described in this section, requires a Sequencing[L] be provided, where again L is the left-hand type.
(A Sequencing[L] instance defines the "sequencing nature" of a type L.)
The reason, essentially, is that contain syntax that implies an "order" of elements makes sense only for types that place elements in a sequence.
For example, it doesn't make sense to assert that a Map[String, Int] or Set[Int] contains all of a set of integers in a particular
order, as these types don't necessarily define an order for their elements. But this does make sense for a type such as Seq[Int] that does define
an order for its elements.
The Sequencing companion object provides implicit instances of Sequencing[L]
for types GenSeq[E], java.util.List[E],
String, and Array[E].
Here are some examples:
Similar to Containing[L], the implicit methods that provide the Aggregating[L] instances require an Equality[E], where
E is an element type. For example, to obtain a Aggregating[Vector[String]] you must supply an Equality[String],
either implicitly or explicitly. The contain syntax uses this Equality[E] to determine containership.
Thus if you want to change how containership is determined for an element type E, place an implicit Equality[E]
in scope or use the explicitly DSL. Although the implicit parameter required for the contain syntax is of type Aggregating[L],
implicit conversions are provided in the Aggregating companion object from Equality[E] to the various
types of aggregations of E. Here's an example:
The "contain inOrderOnly" syntax lets you assert that the containing object contains only the specified objects, in order.
The specified objects may appear multiple times, but must appear in the order they appear in the right-hand list. Here's an example:
List(1, 2, 2, 3, 3, 3) should contain inOrderOnly (1, 2, 3)
The "contain inOrder" syntax lets you assert that the containing object contains only the specified objects in order, like
inOrderOnly, but allows other objects to appear in the left-hand aggregation as well:
contain more than one of each:
List(0, 1, 2, 2, 99, 3, 3, 3, 5) should contain inOrder (1, 2, 3)
If you have a collection of elements that you'd like to use in a "in order" comparison, you can use "inOrderElementsOf," like this:
List(0, 1, 2, 2, 99, 3, 3, 3, 5) should contain inOrderElementsOf Array(1, 2, 3)
Note that "order" in inOrder, inOrderOnly, and theSameElementsInOrderAs (described below)
in the Aggregation[L] instances built-in to ScalaTest is defined as "iteration order".
Lastly, the "contain theSameElementsInOrderAs" syntax lets you assert that two aggregations contain
the same exact elements in the same (iteration) order:
List(1, 2, 3) should contain theSameElementsInOrderAs collection.mutable.TreeSet(3, 2, 1)
The previous assertion succeeds because the iteration order of aTreeSet is the natural
ordering of its elements, which in this case is 1, 2, 3. An iterator obtained from the left-hand List will produce the same elements
in the same order.
Note that no inOrderOnlyElementsOf matcher is provided, because it would have the same
behavior as theSameElementsInOrderAs. (I.e., if you were looking for inOrderOnlyElementsOf, please use theSameElementsInOrderAs
instead.)
You can also ask whether the elements of "sortable" objects (such as Arrays, Java Lists, and GenSeqs)
are in sorted order, like this:
List(1, 2, 3) shouldBe sorted
Althought it seems desireable to provide similar matcher syntax for Scala and Java iterators to that provided for sequences like
Seqs, Array, and java.util.List, the
ephemeral nature of iterators makes this problematic. Some syntax (such as should contain) is relatively straightforward to
support on iterators, but other syntax (such
as, for example, Inspector expressions on nested iterators) is not. Rather
than allowing inconsistencies between sequences and iterators in the API, we chose to not support any such syntax directly on iterators:
scala> val it = List(1, 2, 3).iterator
it: Iterator[Int] = non-empty iterator
scala> it should contain (2)
<console>:15: error: could not find implicit value for parameter typeClass1: org.scalatest.enablers.Containing[Iterator[Int]]
it should contain (2)
^
Instead, you will need to convert your iterators to a sequence explicitly before using them in matcher expressions:
scala> it.toStream should contain (2)
We recommend you convert (Scala or Java) iterators to Streams, as shown in the previous example, so that you can
continue to reap any potential benefits provided by the laziness of the underlying iterator.
You can use the Inspectors syntax with matchers as well as assertions. If you have a multi-dimensional collection, such as a
list of lists, using Inspectors is your best option:
val yss =
List(
List(1, 2, 3),
List(1, 2, 3),
List(1, 2, 3)
)
forAll (yss) { ys =>
forAll (ys) { y => y should be > 0 }
}
For assertions on one-dimensional collections, however, matchers provides "inspector shorthands." Instead of writing:
val xs = List(1, 2, 3)
forAll (xs) { x => x should be < 10 }
You can write:
all (xs) should be < 10
The previous statement asserts that all elements of the xs list should be less than 10.
All of the inspectors have shorthands in matchers. Here is the full list:
all - succeeds if the assertion holds true for every elementatLeast - succeeds if the assertion holds true for at least the specified number of elementsatMost - succeeds if the assertion holds true for at most the specified number of elementsbetween - succeeds if the assertion holds true for between the specified minimum and maximum number of elements, inclusiveevery - same as all, but lists all failing elements if it fails (whereas all just reports the first failing element)exactly - succeeds if the assertion holds true for exactly the specified number of elementsHere are some examples:
scala> import org.scalatest.Matchers._
import org.scalatest.Matchers._
scala> val xs = List(1, 2, 3, 4, 5)
xs: List[Int] = List(1, 2, 3, 4, 5)
scala> all (xs) should be > 0
scala> atMost (2, xs) should be >= 4
scala> atLeast (3, xs) should be < 5
scala> between (2, 3, xs) should (be > 1 and be < 5)
scala> exactly (2, xs) should be <= 2
scala> every (xs) should be < 10
scala> // And one that fails...
scala> exactly (2, xs) shouldEqual 2
org.scalatest.exceptions.TestFailedException: 'exactly(2)' inspection failed, because only 1 element
satisfied the assertion block at index 1:
at index 0, 1 did not equal 2,
at index 2, 3 did not equal 2,
at index 3, 4 did not equal 2,
at index 4, 5 did not equal 2
in List(1, 2, 3, 4, 5)
at ...
Like Inspectors, objects used with inspector shorthands can be any type T for which a Collecting[T, E]
is availabe, which by default includes GenTraversable,
Java Collection, Java Map, Arrays, and Strings.
Here are some examples:
scala> import org.scalatest._
import org.scalatest._
scala> import Matchers._
import Matchers._
scala> all (Array(1, 2, 3)) should be < 5
scala> import collection.JavaConverters._
import collection.JavaConverters._
scala> val js = List(1, 2, 3).asJava
js: java.util.List[Int] = [1, 2, 3]
scala> all (js) should be < 5
scala> val jmap = Map("a" -> 1, "b" -> 2).asJava
jmap: java.util.Map[String,Int] = {a=1, b=2}
scala> atLeast(1, jmap) shouldBe Entry("b", 2)
scala> atLeast(2, "hello, world!") shouldBe 'o'
To assert both that a collection contains just one "lone" element as well as something else about that element, you can use
the loneElement syntax provided by trait LoneElement. For example, if a
Set[Int] should contain just one element, an Int
less than or equal to 10, you could write:
import LoneElement._ set.loneElement should be <= 10
You can invoke loneElement on any type T for which an implicit Collecting[E, T]
is available, where E is the element type returned by the loneElement invocation. By default, you can use loneElement
on GenTraversable, Java Collection, Java Map, Array, and String.
You can use similar syntax on Java collections (java.util.Collection) and maps (java.util.Map).
For example, you can check whether a Java Collection or Map is empty,
like this:
javaCollection should be ('empty)
javaMap should be ('empty)
Even though Java's List type doesn't actually have a length or getLength method,
you can nevertheless check the length of a Java List (java.util.List) like this:
javaList should have length 9
You can check the size of any Java Collection or Map, like this:
javaMap should have size 20 javaSet should have size 90
In addition, you can check whether a Java Collection contains a particular
element, like this:
javaCollection should contain ("five")
One difference to note between the syntax supported on Java and Scala collections is that
in Java, Map is not a subtype of Collection, and does not
actually define an element type. You can ask a Java Map for an "entry set"
via the entrySet method, which will return the Map's key/value pairs
wrapped in a set of java.util.Map.Entry, but a Map is not actually
a collection of Entry. To make Java Maps easier to work with, however,
ScalaTest matchers allows you to treat a Java Map as a collection of Entry,
and defines a convenience implementation of java.util.Map.Entry in
org.scalatest.Entry. Here's how you use it:
javaMap should contain (Entry(2, 3)) javaMap should contain oneOf (Entry(2, 3), Entry(3, 4))
You can you alse just check whether a Java Map contains a particular key, or value, like this:
javaMap should contain key 1 javaMap should contain value "Howdy"
Strings and Arrays as collections You can also use all the syntax described above for Scala and Java collections on Arrays and
Strings. Here are some examples:
scala> import org.scalatest._
import org.scalatest._
scala> import Matchers._
import Matchers._
scala> atLeast (2, Array(1, 2, 3)) should be > 1
scala> atMost (2, "halloo") shouldBe 'o'
scala> Array(1, 2, 3) shouldBe sorted
scala> "abcdefg" shouldBe sorted
scala> Array(1, 2, 3) should contain atMostOneOf (3, 4, 5)
scala> "abc" should contain atMostOneOf ('c', 'd', 'e')
be as an equality comparison All uses of be other than those shown previously perform an equality comparison. They work
the same as equal when it is used with default equality. This redundancy between be and equals exists in part
because it enables syntax that sometimes sounds more natural. For example, instead of writing:
result should equal (null)
You can write:
result should be (null)
(Hopefully you won't write that too much given null is error prone, and Option
is usually a better, well, option.)
As mentioned previously, the other difference between equal
and be is that equal delegates the equality check to an Equality typeclass, whereas
be always uses default equality.
Here are some other examples of be used for equality comparison:
sum should be (7.0) boring should be (false) fun should be (true) list should be (Nil) option should be (None) option should be (Some(1))
As with equal used with default equality, using be on arrays results in deep being called on both arrays prior to
calling equal. As a result,
the following expression would not throw a TestFailedException:
Array(1, 2) should be (Array(1, 2)) // succeeds (i.e., does not throw TestFailedException)
Because be is used in several ways in ScalaTest matcher syntax, just as it is used in many ways in English, one
potential point of confusion in the event of a failure is determining whether be was being used as an equality comparison or
in some other way, such as a property assertion. To make it more obvious when be is being used for equality, the failure
messages generated for those equality checks will include the word equal in them. For example, if this expression fails with a
TestFailedException:
option should be (Some(1))
The detail message in that TestFailedException will include the words "equal to" to signify be
was in this case being used for equality comparison:
Some(2) was not equal to Some(1)
If you wish to check the opposite of some condition, you can simply insert not in the expression.
Here are a few examples:
result should not be (null)
sum should not be <= (10)
mylist should not equal (yourList)
string should not startWith ("Hello")
Often when creating libraries you may wish to ensure that certain arrangements of code that
represent potential “user errors” do not compile, so that your library is more error resistant.
ScalaTest Matchers trait includes the following syntax for that purpose:
"val a: String = 1" shouldNot compile
If you want to ensure that a snippet of code does not compile because of a type error (as opposed to a syntax error), use:
"val a: String = 1" shouldNot typeCheck
Note that the shouldNot typeCheck syntax will only succeed if the given snippet of code does not
compile because of a type error. A syntax error will still result on a thrown TestFailedException.
If you want to state that a snippet of code does compile, you can make that more obvious with:
"val a: Int = 1" should compile
Although the previous three constructs are implemented with macros that determine at compile time whether the snippet of code represented by the string does or does not compile, errors are reported as test failures at runtime.
and and or You can also combine matcher expressions with and and/or or, however,
you must place parentheses or curly braces around the and or or expression. For example,
this and-expression would not compile, because the parentheses are missing:
map should contain key ("two") and not contain value (7) // ERROR, parentheses missing!
Instead, you need to write:
map should (contain key ("two") and not contain value (7))
Here are some more examples:
number should (be > (0) and be <= (10))
option should (equal (Some(List(1, 2, 3))) or be (None))
string should (
equal ("fee") or
equal ("fie") or
equal ("foe") or
equal ("fum")
)
Two differences exist between expressions composed of these and and or operators and the expressions you can write
on regular Booleans using its && and || operators. First, expressions with and
and or do not short-circuit. The following contrived expression, for example, would print "hello, world!":
"yellow" should (equal ("blue") and equal { println("hello, world!"); "green" })
In other words, the entire and or or expression is always evaluated, so you'll see any side effects
of the right-hand side even if evaluating
only the left-hand side is enough to determine the ultimate result of the larger expression. Failure messages produced by these
expressions will "short-circuit," however,
mentioning only the left-hand side if that's enough to determine the result of the entire expression. This "short-circuiting" behavior
of failure messages is intended
to make it easier and quicker for you to ascertain which part of the expression caused the failure. The failure message for the previous
expression, for example, would be:
"yellow" did not equal "blue"
Most likely this lack of short-circuiting would rarely be noticeable, because evaluating the right hand side will usually not
involve a side effect. One situation where it might show up, however, is if you attempt to and a null check on a variable with an expression
that uses the variable, like this:
map should (not be (null) and contain key ("ouch"))
If map is null, the test will indeed fail, but with a NullArgumentException, not a
TestFailedException. Here, the NullArgumentException is the visible right-hand side effect. To get a
TestFailedException, you would need to check each assertion separately:
map should not be (null)
map should contain key ("ouch")
If map is null in this case, the null check in the first expression will fail with
a TestFailedException, and the second expression will never be executed.
The other difference with Boolean operators is that although && has a higher precedence than ||,
and and or
have the same precedence. Thus although the Boolean expression (a || b && c) will evaluate the && expression
before the || expression, like (a || (b && c)), the following expression:
traversable should (contain (7) or contain (8) and have size (9))
Will evaluate left to right, as:
traversable should ((contain (7) or contain (8)) and have size (9))
If you really want the and part to be evaluated first, you'll need to put in parentheses, like this:
traversable should (contain (7) or (contain (8) and have size (9)))
Options You can work with options using ScalaTest's equality, empty,
defined, and contain syntax.
For example, if you wish to check whether an option is None, you can write any of:
option shouldEqual None option shouldBe None option should === (None) option shouldBe empty
If you wish to check an option is defined, and holds a specific value, you can write any of:
option shouldEqual Some("hi")
option shouldBe Some("hi")
option should === (Some("hi"))
If you only wish to check that an option is defined, but don't care what it's value is, you can write:
option shouldBe defined
If you mix in (or import the members of) OptionValues,
you can write one statement that indicates you believe an option should be defined and then say something else about its value. Here's an example:
import org.scalatest.OptionValues._ option.value should be < 7
As mentioned previously, you can use also use ScalaTest's contain, contain oneOf, and
contain noneOf syntax with options:
Some(2) should contain (2) Some(7) should contain oneOf (5, 7, 9) Some(0) should contain noneOf (7, 8, 9)
have Using have, you can check properties of any type, where a property is an attribute of any
object that can be retrieved either by a public field, method, or JavaBean-style get
or is method, like this:
book should have (
'title ("Programming in Scala"),
'author (List("Odersky", "Spoon", "Venners")),
'pubYear (2008)
)
This expression will use reflection to ensure the title, author, and pubYear properties of object book
are equal to the specified values. For example, it will ensure that book has either a public Java field or method
named title, or a public method named getTitle, that when invoked (or accessed in the field case) results
in a the string "Programming in Scala". If all specified properties exist and have their expected values, respectively,
execution will continue. If one or more of the properties either does not exist, or exists but results in an unexpected value,
a TestFailedException will be thrown that explains the problem. (For the details on how a field or method is selected during this
process, see the documentation for HavePropertyMatcherGenerator.)
When you use this syntax, you must place one or more property values in parentheses after have, seperated by commas, where a property
value is a symbol indicating the name of the property followed by the expected value in parentheses. The only exceptions to this rule is the syntax
for checking size and length shown previously, which does not require parentheses. If you forget and put parentheses in, however, everything will
still work as you'd expect. Thus instead of writing:
array should have length (3) set should have size (90)
You can alternatively, write:
array should have (length (3)) set should have (size (90))
If a property has a value different from the specified expected value, a TestFailedError will be thrown
with a detailed message that explains the problem. For example, if you assert the following on
a book whose title is Moby Dick:
book should have ('title ("A Tale of Two Cities"))
You'll get a TestFailedException with this detail message:
The title property had value "Moby Dick", instead of its expected value "A Tale of Two Cities",
on object Book("Moby Dick", "Melville", 1851)
If you prefer to check properties in a type-safe manner, you can use a HavePropertyMatcher.
This would allow you to write expressions such as:
book should have (
title ("Programming in Scala"),
author (List("Odersky", "Spoon", "Venners")),
pubYear (2008)
)
These expressions would fail to compile if should is used on an inappropriate type, as determined
by the type parameter of the HavePropertyMatcher being used. (For example, title in this example
might be of type HavePropertyMatcher[org.publiclibrary.Book]. If used with an appropriate type, such an expression will compile
and at run time the property method or field will be accessed directly; i.e., no reflection will be used.
See the documentation for HavePropertyMatcher for more information.
length and size with HavePropertyMatchers If you want to use length or size syntax with your own custom HavePropertyMatchers, you
can do so, but you must write (of [“the type”]) afterwords. For example, you could write:
book should have (
title ("A Tale of Two Cities"),
length (220) (of [Book]),
author ("Dickens")
)
Prior to ScalaTest 2.0, “length (22)” yielded a HavePropertyMatcher[Any, Int] that used reflection to dynamically look
for a length field or getLength method. In ScalaTest 2.0, “length (22)” yields a
MatcherFactory1[Any, Length], so it is no longer a HavePropertyMatcher. The (of [<type>]) syntax converts the
the MatcherFactory1[Any, Length] to a HavePropertyMatcher[<type>, Int].
ScalaTest's Inside trait allows you to make assertions after a pattern match.
Here's an example:
case class Name(first: String, middle: String, last: String)
val name = Name("Jane", "Q", "Programmer")
inside(name) { case Name(first, _, _) =>
first should startWith ("S")
}
You can use inside to just ensure a pattern is matched, without making any further assertions, but a better
alternative for that kind of assertion is matchPattern. The matchPattern syntax allows you
to express that you expect a value to match a particular pattern, no more and no less:
name should matchPattern { case Name("Sarah", _, _) => }
If none of the built-in matcher syntax (or options shown so far for extending the syntax) satisfy a particular need you have, you can create
custom Matchers that allow
you to place your own syntax directly after should. For example, class java.io.File has a method isHidden, which
indicates whether a file of a certain path and name is hidden. Because the isHidden method takes no parameters and returns Boolean,
you can call it using be with a symbol or BePropertyMatcher, yielding assertions like:
file should be ('hidden) // using a symbol
file should be (hidden) // using a BePropertyMatcher
If it doesn't make sense to have your custom syntax follow be, you might want to create a custom Matcher
instead, so your syntax can follow should directly. For example, you might want to be able to check whether
a java.io.File's name ends with a particular extension, like this:
// using a plain-old Matcher
file should endWithExtension ("txt")
ScalaTest provides several mechanism to make it easy to create custom matchers, including ways to compose new matchers
out of existing ones complete with new error messages. For more information about how to create custom
Matchers, please see the documentation for the Matcher trait.
Sometimes you need to test whether a method throws an expected exception under certain circumstances, such
as when invalid arguments are passed to the method. With Matchers mixed in, you can
check for an expected exception like this:
an [IndexOutOfBoundsException] should be thrownBy s.charAt(-1)
If charAt throws an instance of StringIndexOutOfBoundsException,
this expression will result in that exception. But if charAt completes normally, or throws a different
exception, this expression will complete abruptly with a TestFailedException.
If you need to further isnpect an expected exception, you can capture it using this syntax:
val thrown = the [IndexOutOfBoundsException] thrownBy s.charAt(-1)
This expression returns the caught exception so that you can inspect it further if you wish, for example, to ensure that data contained inside the exception has the expected values. Here's an example:
thrown.getMessage should equal ("String index out of range: -1")
If you prefer you can also capture and inspect an expected exception in one statement, like this:
the [ArithmeticException] thrownBy 1 / 0 should have message "/ by zero"
the [IndexOutOfBoundsException] thrownBy {
s.charAt(-1)
} should have message "String index out of range: -1"
You can also state that no exception should be thrown by some code, like this:
noException should be thrownBy 0 / 1
Perhaps the most tricky part of writing assertions using ScalaTest matchers is remembering when you need or don't need parentheses, but bearing in mind a few simple rules should help. It is also reassuring to know that if you ever leave off a set of parentheses when they are required, your code will not compile. Thus the compiler will help you remember when you need the parens. That said, the rules are:
1. Although you don't always need them, you may choose to always put parentheses
around right-hand values, such as the 7 in num should equal (7):
result should equal (4) array should have length (3) book should have ( 'title ("Programming in Scala"), 'author (List("Odersky", "Spoon", "Venners")), 'pubYear (2008) ) option should be ('defined) catMap should (contain key (9) and contain value ("lives")) keyEvent should be an ('actionKey) javaSet should have size (90)
2. Except for length, size and message, you must always put parentheses around
the list of one or more property values following a have:
file should (exist and have ('name ("temp.txt"))) book should have ( title ("Programming in Scala"), author (List("Odersky", "Spoon", "Venners")), pubYear (2008) ) javaList should have length (9) // parens optional for length and size
3. You must always put parentheses around and and or expressions, as in:
catMap should (contain key (9) and contain value ("lives")) number should (equal (2) or equal (4) or equal (8))
4. Although you don't always need them, you may choose to always put parentheses
around custom Matchers when they appear directly after not:
file should exist file should not (exist) file should (exist and have ('name ("temp.txt"))) file should (not (exist) and have ('name ("temp.txt")) file should (have ('name ("temp.txt") or exist) file should (have ('name ("temp.txt") or not (exist))
That's it. With a bit of practice it should become natural to you, and the compiler will always be there to tell you if you forget a set of needed parentheses.
Note: ScalaTest's matchers are in part inspired by the matchers of RSpec,
Hamcrest, and
specs2, and its “shouldNot compile” syntax
by the illTyped macro of shapeless.
Trait that provides a domain specific language (DSL) for expressing assertions in tests
using the word must.
Trait that provides a domain specific language (DSL) for expressing assertions in tests
using the word must.
For example, if you mix Matchers into
a suite class, you can write an equality assertion in that suite like this:
result must equal (3)
Here result is a variable, and can be of any type. If the object is an
Int with the value 3, execution will continue (i.e., the expression will result
in the unit value, ()). Otherwise, a TestFailedException
will be thrown with a detail message that explains the problem, such as "7 did not equal 3".
This TestFailedException will cause the test to fail.
Here is a table of contents for this documentation:
Boolean properties with beBeMatchersStrings and Arrays as collectionsand and orOptionshavelength and size with HavePropertyMatchersTrait MustMatchers is an alternative to Matchers that provides the exact same
meaning, syntax, and behavior as Matchers, but uses the verb must instead of should.
The two traits differ only in the English semantics of the verb: should
is informal, making the code feel like conversation between the writer and the reader; must is more formal, making the code feel more like
a written specification.
ScalaTest matchers provides five different ways to check equality, each designed to address a different need. They are:
result must equal (3) // can customize equality result must === (3) // can customize equality and enforce type constraints result must be (3) // cannot customize equality, so fastest to compile result mustEqual 3 // can customize equality, no parentheses required result mustBe 3 // cannot customize equality, so fastest to compile, no parentheses required
The “left must equal (right)” syntax requires an
org.scalactic.Equality[L] to be provided (either implicitly or explicitly), where
L is the left-hand type on which must is invoked. In the "left must equal (right)" case,
for example, L is the type of left. Thus if left is type Int, the "left must
equal (right)"
statement would require an Equality[Int].
By default, an implicit Equality[T] instance is available for any type T, in which equality is implemented
by simply invoking == on the left
value, passing in the right value, with special treatment for arrays. If either left or right is an array, deep
will be invoked on it before comparing with ==. Thus, the following expression
will yield false, because Array's equals method compares object identity:
Array(1, 2) == Array(1, 2) // yields false
The next expression will by default not result in a TestFailedException, because default Equality[Array[Int]] compares
the two arrays structurally, taking into consideration the equality of the array's contents:
Array(1, 2) must equal (Array(1, 2)) // succeeds (i.e., does not throw TestFailedException)
If you ever do want to verify that two arrays are actually the same object (have the same identity), you can use the
be theSameInstanceAs syntax, described below.
You can customize the meaning of equality for a type when using "must equal," "must ===,"
or mustEqual syntax by defining implicit Equality instances that will be used instead of default Equality.
You might do this to normalize types before comparing them with ==, for instance, or to avoid calling the == method entirely,
such as if you want to compare Doubles with a tolerance.
For an example, see the main documentation of trait Equality.
You can always supply implicit parameters explicitly, but in the case of implicit parameters of type Equality[T], Scalactic provides a
simple "explictly" DSL. For example, here's how you could explicitly supply an Equality[String] instance that normalizes both left and right
sides (which must be strings), by transforming them to lowercase:
scala> import org.scalatest.MustMatchers._
import org.scalatest.MustMatchers._
scala> import org.scalactic.Explicitly._
import org.scalactic.Explicitly._
scala> import org.scalactic.StringNormalizations._
import org.scalactic.StringNormalizations._
scala> "Hi" must equal ("hi") (after being lowerCased)
The after being lowerCased expression results in an Equality[String], which is then passed
explicitly as the second curried parameter to equal. For more information on the explictly DSL, see the main documentation
for trait Explicitly.
The "must be" and mustBe syntax do not take an Equality[T] and can therefore not be customized.
They always use the default approach to equality described above. As a result, "must be" and mustBe will
likely be the fastest-compiling matcher syntax for equality comparisons, since the compiler need not search for
an implicit Equality[T] each time.
The must === syntax (and its complement, must !==) can be used to enforce type
constraints at compile-time between the left and right sides of the equality comparison. Here's an example:
scala> import org.scalatest.MustMatchers._
import org.scalatest.MustMatchers._
scala> import org.scalactic.TypeCheckedTripleEquals._
import org.scalactic.TypeCheckedTripleEquals._
scala> Some(2) must === (2)
<console>:17: error: types Some[Int] and Int do not adhere to the equality constraint
selected for the === and !== operators; the missing implicit parameter is of
type org.scalactic.CanEqual[Some[Int],Int]
Some(2) must === (2)
^
By default, the "Some(2) must === (2)" statement would fail at runtime. By mixing in
the equality constraints provided by TypeCheckedTripleEquals, however, the statement fails to compile. For more information
and examples, see the main documentation for trait TypeCheckedTripleEquals.
You can check the size or length of any type of object for which it makes sense. Here's how checking for length looks:
result must have length 3
Size is similar:
result must have size 10
The length syntax can be used with String, Array, any scala.collection.GenSeq,
any java.util.List, and any type T for which an implicit Length[T] type class is
available in scope.
Similarly, the size syntax can be used with Array, any scala.collection.GenTraversable,
any java.util.Collection, any java.util.Map, and any type T for which an implicit Size[T] type class is
available in scope. You can enable the length or size syntax for your own arbitrary types, therefore,
by defining Length or Size type
classes for those types.
In addition, the length syntax can be used with any object that has a field or method named length
or a method named getLength. Similarly, the size syntax can be used with any
object that has a field or method named size or a method named getSize.
The type of a length or size field, or return type of a method, must be either Int
or Long. Any such method must take no parameters. (The Scala compiler will ensure at compile time that
the object on which must is being invoked has the appropriate structure.)
You can check for whether a string starts with, ends with, or includes a substring like this:
string must startWith ("Hello")
string must endWith ("world")
string must include ("seven")
You can check for whether a string starts with, ends with, or includes a regular expression, like this:
string must startWith regex "Hel*o" string must endWith regex "wo.ld" string must include regex "wo.ld"
And you can check whether a string fully matches a regular expression, like this:
string must fullyMatch regex """(-)?(\d+)(\.\d*)?"""
The regular expression passed following the regex token can be either a String
or a scala.util.matching.Regex.
With the startWith, endWith, include, and fullyMatch
tokens can also be used with an optional specification of required groups, like this:
"abbccxxx" must startWith regex ("a(b*)(c*)" withGroups ("bb", "cc"))
"xxxabbcc" must endWith regex ("a(b*)(c*)" withGroups ("bb", "cc"))
"xxxabbccxxx" must include regex ("a(b*)(c*)" withGroups ("bb", "cc"))
"abbcc" must fullyMatch regex ("a(b*)(c*)" withGroups ("bb", "cc"))
You can check whether a string is empty with empty:
s mustBe empty
You can also use most of ScalaTest's matcher syntax for collections on String by
treating the Strings as collections of characters. For examples, see the
Strings and Arrays as collections section below.
You can check whether any type for which an implicit Ordering[T] is available
is greater than, less than, greater than or equal, or less
than or equal to a value of type T. The syntax is:
one must be < 7 one must be > 0 one must be <= 7 one must be >= 0
Boolean properties with be If an object has a method that takes no parameters and returns boolean, you can check
it by placing a Symbol (after be) that specifies the name
of the method (excluding an optional prefix of "is"). A symbol literal
in Scala begins with a tick mark and ends at the first non-identifier character. Thus,
'traversableAgain results in a Symbol object at runtime, as does
'completed and 'file. Here's an example:
iter mustBe 'traversableAgain
Given this code, ScalaTest will use reflection to look on the object referenced from
emptySet for a method that takes no parameters and results in Boolean,
with either the name empty or isEmpty. If found, it will invoke
that method. If the method returns true, execution will continue. But if it returns
false, a TestFailedException will be thrown that will contain a detail message, such as:
non-empty iterator was not traversableAgain
This be syntax can be used with any reference (AnyRef) type. If the object does
not have an appropriately named predicate method, you'll get a TestFailedException
at runtime with a detailed message that explains the problem.
(For the details on how a field or method is selected during this
process, see the documentation for BeWord.)
If you think it reads better, you can optionally put a or an after
be. For example, java.io.File has two predicate methods,
isFile and isDirectory. Thus with a File object
named temp, you could write:
temp must be a 'file
Or, given java.awt.event.KeyEvent has a method isActionKey that takes
no arguments and returns Boolean, you could assert that a KeyEvent is
an action key with:
keyEvent must be an 'actionKey
If you prefer to check Boolean properties in a type-safe manner, you can use a BePropertyMatcher.
This would allow you to write expressions such as:
xs mustBe traversableAgain temp must be a file keyEvent must be an actionKey
These expressions would fail to compile if must is used on an inappropriate type, as determined
by the type parameter of the BePropertyMatcher being used. (For example, file in this example
would likely be of type BePropertyMatcher[java.io.File]. If used with an appropriate type, such an expression will compile
and at run time the Boolean property method or field will be accessed directly; i.e., no reflection will be used.
See the documentation for BePropertyMatcher for more information.
BeMatchers If you want to create a new way of using be, which doesn't map to an actual property on the
type you care about, you can create a BeMatcher. You could use this, for example, to create BeMatcher[Int]
called odd, which would match any odd Int, and even, which would match
any even Int.
Given this pair of BeMatchers, you could check whether an Int was odd or even with expressions like:
num mustBe odd num must not be even
For more information, see the documentation for BeMatcher.
If you need to check that two references refer to the exact same object, you can write:
ref1 must be theSameInstanceAs ref2
If you need to check that an object is an instance of a particular class or trait, you can supply the type to
“be a” or “be an”:
result1 mustBe a [Tiger] result1 must not be an [Orangutan]
Because type parameters are erased on the JVM, we recommend you insert an underscore for any type parameters
when using this syntax. Both of the following test only that the result is an instance of List[_], because at
runtime the type parameter has been erased:
result mustBe a [List[_]] // recommended result mustBe a [List[Fruit]] // discouraged
Often you may want to check whether a number is within a
range. You can do that using the +- operator, like this:
sevenDotOh must equal (6.9 +- 0.2) sevenDotOh must === (6.9 +- 0.2) sevenDotOh must be (6.9 +- 0.2) sevenDotOh mustEqual 6.9 +- 0.2 sevenDotOh mustBe 6.9 +- 0.2
Any of these expressions will cause a TestFailedException to be thrown if the floating point
value, sevenDotOh is outside the range 6.7 to 7.1.
You can use +- with any type T for which an implicit Numeric[T] exists, such as integral types:
seven must equal (6 +- 2) seven must === (6 +- 2) seven must be (6 +- 2) seven mustEqual 6 +- 2 seven mustBe 6 +- 2
You can check whether an object is "empty", like this:
traversable mustBe empty javaMap must not be empty
The empty token can be used with any type L for which an implicit Emptiness[L] exists.
The Emptiness companion object provides implicits for GenTraversable[E], java.util.Collection[E],
java.util.Map[K, V], String, Array[E], and Option[E]. In addition, the
Emptiness companion object provides structural implicits for types that declare an isEmpty method that
returns a Boolean. Here are some examples:
scala> import org.scalatest.MustMatchers._
import org.scalatest.MustMatchers._
scala> List.empty mustBe empty
scala> None mustBe empty
scala> Some(1) must not be empty
scala> "" mustBe empty
scala> new java.util.HashMap[Int, Int] mustBe empty
scala> new { def isEmpty = true} mustBe empty
scala> Array(1, 2, 3) must not be empty
You can check whether a collection contains a particular element like this:
traversable must contain ("five")
The contain syntax shown above can be used with any type C that has a "containing" nature, evidenced by
an implicit org.scalatest.enablers.Containing[L], where L is left-hand type on
which must is invoked. In the Containing
companion object, implicits are provided for types GenTraversable[E], java.util.Collection[E],
java.util.Map[K, V], String, Array[E], and Option[E].
Here are some examples:
scala> import org.scalatest.MustMatchers._
import org.scalatest.MustMatchers._
scala> List(1, 2, 3) must contain (2)
scala> Map('a' -> 1, 'b' -> 2, 'c' -> 3) must contain ('b' -> 2)
scala> Set(1, 2, 3) must contain (2)
scala> Array(1, 2, 3) must contain (2)
scala> "123" must contain ('2')
scala> Some(2) must contain (2)
ScalaTest's implicit methods that provide the Containing[L] type classes require an Equality[E], where
E is an element type. For example, to obtain a Containing[Array[Int]] you must supply an Equality[Int],
either implicitly or explicitly. The contain syntax uses this Equality[E] to determine containership.
Thus if you want to change how containership is determined for an element type E, place an implicit Equality[E]
in scope or use the explicitly DSL. Although the implicit parameter required for the contain syntax is of type Containing[L],
implicit conversions are provided in the Containing companion object from Equality[E] to the various
types of containers of E. Here's an example:
scala> import org.scalatest.MustMatchers._
import org.scalatest.MustMatchers._
scala> List("Hi", "Di", "Ho") must contain ("ho")
org.scalatest.exceptions.TestFailedException: List(Hi, Di, Ho) did not contain element "ho"
at ...
scala> import org.scalactic.Explicitly._
import org.scalactic.Explicitly._
scala> import org.scalactic.StringNormalizations._
import org.scalactic.StringNormalizations._
scala> (List("Hi", "Di", "Ho") must contain ("ho")) (after being lowerCased)
Note that when you use the explicitly DSL with contain you need to wrap the entire
contain expression in parentheses, as shown here.
(List("Hi", "Di", "Ho") must contain ("ho")) (after being lowerCased)
^ ^
In addition to determining whether an object contains another object, you can use contain to
make other determinations.
For example, the contain oneOf syntax ensures that one and only one of the specified elements are
contained in the containing object:
List(1, 2, 3, 4, 5) must contain oneOf (5, 7, 9)
Some(7) must contain oneOf (5, 7, 9)
"howdy" must contain oneOf ('a', 'b', 'c', 'd')
Note that if multiple specified elements appear in the containing object, oneOf will fail:
scala> List(1, 2, 3) must contain oneOf (2, 3, 4)
org.scalatest.exceptions.TestFailedException: List(1, 2, 3) did not contain one (and only one) of (2, 3, 4)
at ...
If you really want to ensure one or more of the specified elements are contained in the containing object,
use atLeastOneOf, described below, instead of oneOf. Keep in mind, oneOf
means "exactly one of."
Note also that with any contain syntax, you can place custom implicit Equality[E] instances in scope
to customize how containership is determined, or use the explicitly DSL. Here's an example:
(Array("Doe", "Ray", "Me") must contain oneOf ("X", "RAY", "BEAM")) (after being lowerCased)
If you have a collection of elements that you'd like to use in a "one of" comparison, you can use "oneElementOf," like this:
List(1, 2, 3, 4, 5) must contain oneElementOf List(5, 7, 9)
Some(7) must contain oneElementOf Vector(5, 7, 9)
"howdy" must contain oneElementOf Set('a', 'b', 'c', 'd')
(Array("Doe", "Ray", "Me") must contain oneElementOf List("X", "RAY", "BEAM")) (after being lowerCased)
The contain noneOf syntax does the opposite of oneOf: it ensures none of the specified elements
are contained in the containing object:
List(1, 2, 3, 4, 5) must contain noneOf (7, 8, 9)
Some(0) must contain noneOf (7, 8, 9)
"12345" must contain noneOf ('7', '8', '9')
If you have a collection of elements that you'd like to use in a "none of" comparison, you can use "noElementsOf," like this:
List(1, 2, 3, 4, 5) must contain noElementsOf List(7, 8, 9)
Some(0) must contain noElementsOf Vector(7, 8, 9)
"12345" must contain noElementsOf Set('7', '8', '9')
As mentioned, the "contain," "contain oneOf," and "contain noneOf" syntax requires a
Containing[L] be provided, where L is the left-hand type. Other contain syntax, which
will be described in this section, requires an Aggregating[L] be provided, where again L is the left-hand type.
(An Aggregating[L] instance defines the "aggregating nature" of a type L.)
The reason, essentially, is that contain syntax that makes sense for Option is enabled by
Containing[L], whereas syntax that does not make sense for Option is enabled
by Aggregating[L]. For example, it doesn't make sense to assert that an Option[Int] contains all of a set of integers, as it
could only ever contain one of them. But this does make sense for a type such as List[Int] that can aggregate zero to many integers.
The Aggregating companion object provides implicit instances of Aggregating[L]
for types GenTraversable[E], java.util.Collection[E],
java.util.Map[K, V], String, Array[E]. Note that these are the same types as are supported with
Containing, but with Option[E] missing.
Here are some examples:
The contain atLeastOneOf syntax, for example, works for any type L for which an Aggregating[L] exists. It ensures
that at least one of (i.e., one or more of) the specified objects are contained in the containing object:
List(1, 2, 3) must contain atLeastOneOf (2, 3, 4)
Array(1, 2, 3) must contain atLeastOneOf (3, 4, 5)
"abc" must contain atLeastOneOf ('c', 'a', 't')
Similar to Containing[L], the implicit methods that provide the Aggregating[L] instances require an Equality[E], where
E is an element type. For example, to obtain a Aggregating[Vector[String]] you must supply an Equality[String],
either implicitly or explicitly. The contain syntax uses this Equality[E] to determine containership.
Thus if you want to change how containership is determined for an element type E, place an implicit Equality[E]
in scope or use the explicitly DSL. Although the implicit parameter required for the contain syntax is of type Aggregating[L],
implicit conversions are provided in the Aggregating companion object from Equality[E] to the various
types of aggregations of E. Here's an example:
(Vector(" A", "B ") must contain atLeastOneOf ("a ", "b", "c")) (after being lowerCased and trimmed)
If you have a collection of elements that you'd like to use in an "at least one of" comparison, you can use "atLeastOneElementOf," like this:
List(1, 2, 3) must contain atLeastOneElementOf List(2, 3, 4)
Array(1, 2, 3) must contain atLeastOneElementOf Vector(3, 4, 5)
"abc" must contain atLeastOneElementOf Set('c', 'a', 't')
(Vector(" A", "B ") must contain atLeastOneElementOf List("a ", "b", "c")) (after being lowerCased and trimmed)
The "contain atMostOneOf" syntax lets you specify a set of objects at most one of which must be contained in the containing object:
List(1, 2, 3, 4, 5) must contain atMostOneOf (5, 6, 7)
If you have a collection of elements that you'd like to use in a "at most one of" comparison, you can use "atMostOneElementOf," like this:
List(1, 2, 3, 4, 5) must contain atMostOneElementOf Vector(5, 6, 7)
The "contain allOf" syntax lets you specify a set of objects that must all be contained in the containing object:
List(1, 2, 3, 4, 5) must contain allOf (2, 3, 5)
If you have a collection of elements that you'd like to use in a "all of" comparison, you can use "allElementsOf," like this:
List(1, 2, 3, 4, 5) must contain allElementsOf Array(2, 3, 5)
The "contain only" syntax lets you assert that the containing object contains only the specified objects, though it may
contain more than one of each:
List(1, 2, 3, 2, 1) must contain only (1, 2, 3)
The "contain theSameElementsAs" and "contain theSameElementsInOrderAs syntax differ from the others
in that the right hand side is a GenTraversable[_] rather than a varargs of Any. (Note: in a future 2.0 milestone release, possibly
2.0.M6, these will likely be widened to accept any type R for which an Aggregating[R] exists.)
The "contain theSameElementsAs" syntax lets you assert that two aggregations contain the same objects:
List(1, 2, 2, 3, 3, 3) must contain theSameElementsAs Vector(3, 2, 3, 1, 2, 3)
The number of times any family of equal objects appears must also be the same in both the left and right aggregations. The specified objects may appear multiple times, but must appear in the order they appear in the right-hand list. For example, if the last 3 element is left out of the right-hand list in the previous example, the expression would fail because the left side has three 3's and the right hand side has only two:
List(1, 2, 2, 3, 3, 3) must contain theSameElementsAs Vector(3, 2, 3, 1, 2)
org.scalatest.exceptions.TestFailedException: List(1, 2, 2, 3, 3, 3) did not contain the same elements as Vector(3, 2, 3, 1, 2)
at ...
Note that no onlyElementsOf matcher is provided, because it would have the same
behavior as theSameElementsAs. (I.e., if you were looking for onlyElementsOf, please use theSameElementsAs
instead.)
The rest of the contain syntax, which
will be described in this section, requires a Sequencing[L] be provided, where again L is the left-hand type.
(A Sequencing[L] instance defines the "sequencing nature" of a type L.)
The reason, essentially, is that contain syntax that implies an "order" of elements makes sense only for types that place elements in a sequence.
For example, it doesn't make sense to assert that a Map[String, Int] or Set[Int] contains all of a set of integers in a particular
order, as these types don't necessarily define an order for their elements. But this does make sense for a type such as Seq[Int] that does define
an order for its elements.
The Sequencing companion object provides implicit instances of Sequencing[L]
for types GenSeq[E], java.util.List[E],
String, and Array[E].
Here are some examples:
Similar to Containing[L], the implicit methods that provide the Aggregating[L] instances require an Equality[E], where
E is an element type. For example, to obtain a Aggregating[Vector[String]] you must supply an Equality[String],
either implicitly or explicitly. The contain syntax uses this Equality[E] to determine containership.
Thus if you want to change how containership is determined for an element type E, place an implicit Equality[E]
in scope or use the explicitly DSL. Although the implicit parameter required for the contain syntax is of type Aggregating[L],
implicit conversions are provided in the Aggregating companion object from Equality[E] to the various
types of aggregations of E. Here's an example:
The "contain inOrderOnly" syntax lets you assert that the containing object contains only the specified objects, in order.
The specified objects may appear multiple times, but must appear in the order they appear in the right-hand list. Here's an example:
List(1, 2, 2, 3, 3, 3) must contain inOrderOnly (1, 2, 3)
The "contain inOrder" syntax lets you assert that the containing object contains only the specified objects in order, like
inOrderOnly, but allows other objects to appear in the left-hand aggregation as well:
contain more than one of each:
List(0, 1, 2, 2, 99, 3, 3, 3, 5) must contain inOrder (1, 2, 3)
If you have a collection of elements that you'd like to use in a "in order" comparison, you can use "inOrderElementsOf," like this:
List(0, 1, 2, 2, 99, 3, 3, 3, 5) must contain inOrderElementsOf Array(1, 2, 3)
Note that "order" in inOrder, inOrderOnly, and theSameElementsInOrderAs (described below)
in the Aggregation[L] instances built-in to ScalaTest is defined as "iteration order".
Lastly, the "contain theSameElementsInOrderAs" syntax lets you assert that two aggregations contain
the same exact elements in the same (iteration) order:
List(1, 2, 3) must contain theSameElementsInOrderAs collection.mutable.TreeSet(3, 2, 1)
The previous assertion succeeds because the iteration order of aTreeSet is the natural
ordering of its elements, which in this case is 1, 2, 3. An iterator obtained from the left-hand List will produce the same elements
in the same order.
Note that no inOrderOnlyElementsOf matcher is provided, because it would have the same
behavior as theSameElementsInOrderAs. (I.e., if you were looking for inOrderOnlyElementsOf, please use theSameElementsInOrderAs
instead.)
You can also ask whether the elements of "sortable" objects (such as Arrays, Java Lists, and GenSeqs)
are in sorted order, like this:
List(1, 2, 3) mustBe sorted
Althought it seems desireable to provide similar matcher syntax for Scala and Java iterators to that provided for sequences like
Seqs, Array, and java.util.List, the
ephemeral nature of iterators makes this problematic. Some syntax (such as must contain) is relatively straightforward to
support on iterators, but other syntax (such
as, for example, Inspector expressions on nested iterators) is not. Rather
than allowing inconsistencies between sequences and iterators in the API, we chose to not support any such syntax directly on iterators:
scala> val it = List(1, 2, 3).iterator
it: Iterator[Int] = non-empty iterator
scala> it must contain (2)
<console>:15: error: could not find implicit value for parameter typeClass1: org.scalatest.enablers.Containing[Iterator[Int]]
it must contain (2)
^
Instead, you will need to convert your iterators to a sequence explicitly before using them in matcher expressions:
scala> it.toStream must contain (2)
We recommend you convert (Scala or Java) iterators to Streams, as shown in the previous example, so that you can
continue to reap any potential benefits provided by the laziness of the underlying iterator.
You can use the Inspectors syntax with matchers as well as assertions. If you have a multi-dimensional collection, such as a
list of lists, using Inspectors is your best option:
val yss =
List(
List(1, 2, 3),
List(1, 2, 3),
List(1, 2, 3)
)
forAll (yss) { ys =>
forAll (ys) { y => y must be > 0 }
}
For assertions on one-dimensional collections, however, matchers provides "inspector shorthands." Instead of writing:
val xs = List(1, 2, 3)
forAll (xs) { x => x must be < 10 }
You can write:
all (xs) must be < 10
The previous statement asserts that all elements of the xs list must be less than 10.
All of the inspectors have shorthands in matchers. Here is the full list:
all - succeeds if the assertion holds true for every elementatLeast - succeeds if the assertion holds true for at least the specified number of elementsatMost - succeeds if the assertion holds true for at most the specified number of elementsbetween - succeeds if the assertion holds true for between the specified minimum and maximum number of elements, inclusiveevery - same as all, but lists all failing elements if it fails (whereas all just reports the first failing element)exactly - succeeds if the assertion holds true for exactly the specified number of elementsHere are some examples:
scala> import org.scalatest.MustMatchers._
import org.scalatest.MustMatchers._
scala> val xs = List(1, 2, 3, 4, 5)
xs: List[Int] = List(1, 2, 3, 4, 5)
scala> all (xs) must be > 0
scala> atMost (2, xs) must be >= 4
scala> atLeast (3, xs) must be < 5
scala> between (2, 3, xs) must (be > 1 and be < 5)
scala> exactly (2, xs) must be <= 2
scala> every (xs) must be < 10
scala> // And one that fails...
scala> exactly (2, xs) mustEqual 2
org.scalatest.exceptions.TestFailedException: 'exactly(2)' inspection failed, because only 1 element
satisfied the assertion block at index 1:
at index 0, 1 did not equal 2,
at index 2, 3 did not equal 2,
at index 3, 4 did not equal 2,
at index 4, 5 did not equal 2
in List(1, 2, 3, 4, 5)
at ...
Like Inspectors, objects used with inspector shorthands can be any type T for which a Collecting[T, E]
is availabe, which by default includes GenTraversable,
Java Collection, Java Map, Arrays, and Strings.
Here are some examples:
scala> import org.scalatest._
import org.scalatest._
scala> import MustMatchers._
import MustMatchers._
scala> all (Array(1, 2, 3)) must be < 5
scala> import collection.JavaConverters._
import collection.JavaConverters._
scala> val js = List(1, 2, 3).asJava
js: java.util.List[Int] = [1, 2, 3]
scala> all (js) must be < 5
scala> val jmap = Map("a" -> 1, "b" -> 2).asJava
jmap: java.util.Map[String,Int] = {a=1, b=2}
scala> atLeast(1, jmap) mustBe Entry("b", 2)
scala> atLeast(2, "hello, world!") mustBe 'o'
To assert both that a collection contains just one "lone" element as well as something else about that element, you can use
the loneElement syntax provided by trait LoneElement. For example, if a
Set[Int] must contain just one element, an Int
less than or equal to 10, you could write:
import LoneElement._ set.loneElement must be <= 10
You can invoke loneElement on any type T for which an implicit Collecting[E, T]
is available, where E is the element type returned by the loneElement invocation. By default, you can use loneElement
on GenTraversable, Java Collection, Java Map, Array, and String.
You can use similar syntax on Java collections (java.util.Collection) and maps (java.util.Map).
For example, you can check whether a Java Collection or Map is empty,
like this:
javaCollection must be ('empty)
javaMap must be ('empty)
Even though Java's List type doesn't actually have a length or getLength method,
you can nevertheless check the length of a Java List (java.util.List) like this:
javaList must have length 9
You can check the size of any Java Collection or Map, like this:
javaMap must have size 20 javaSet must have size 90
In addition, you can check whether a Java Collection contains a particular
element, like this:
javaCollection must contain ("five")
One difference to note between the syntax supported on Java and Scala collections is that
in Java, Map is not a subtype of Collection, and does not
actually define an element type. You can ask a Java Map for an "entry set"
via the entrySet method, which will return the Map's key/value pairs
wrapped in a set of java.util.Map.Entry, but a Map is not actually
a collection of Entry. To make Java Maps easier to work with, however,
ScalaTest matchers allows you to treat a Java Map as a collection of Entry,
and defines a convenience implementation of java.util.Map.Entry in
org.scalatest.Entry. Here's how you use it:
javaMap must contain (Entry(2, 3)) javaMap must contain oneOf (Entry(2, 3), Entry(3, 4))
You can you alse just check whether a Java Map contains a particular key, or value, like this:
javaMap must contain key 1 javaMap must contain value "Howdy"
Strings and Arrays as collections You can also use all the syntax described above for Scala and Java collections on Arrays and
Strings. Here are some examples:
scala> import org.scalatest._
import org.scalatest._
scala> import MustMatchers._
import MustMatchers._
scala> atLeast (2, Array(1, 2, 3)) must be > 1
scala> atMost (2, "halloo") mustBe 'o'
scala> Array(1, 2, 3) mustBe sorted
scala> "abcdefg" mustBe sorted
scala> Array(1, 2, 3) must contain atMostOneOf (3, 4, 5)
scala> "abc" must contain atMostOneOf ('c', 'd', 'e')
be as an equality comparison All uses of be other than those shown previously perform an equality comparison. They work
the same as equal when it is used with default equality. This redundancy between be and equals exists in part
because it enables syntax that sometimes sounds more natural. For example, instead of writing:
result must equal (null)
You can write:
result must be (null)
(Hopefully you won't write that too much given null is error prone, and Option
is usually a better, well, option.)
As mentioned previously, the other difference between equal
and be is that equal delegates the equality check to an Equality typeclass, whereas
be always uses default equality.
Here are some other examples of be used for equality comparison:
sum must be (7.0) boring must be (false) fun must be (true) list must be (Nil) option must be (None) option must be (Some(1))
As with equal used with default equality, using be on arrays results in deep being called on both arrays prior to
calling equal. As a result,
the following expression would not throw a TestFailedException:
Array(1, 2) must be (Array(1, 2)) // succeeds (i.e., does not throw TestFailedException)
Because be is used in several ways in ScalaTest matcher syntax, just as it is used in many ways in English, one
potential point of confusion in the event of a failure is determining whether be was being used as an equality comparison or
in some other way, such as a property assertion. To make it more obvious when be is being used for equality, the failure
messages generated for those equality checks will include the word equal in them. For example, if this expression fails with a
TestFailedException:
option must be (Some(1))
The detail message in that TestFailedException will include the words "equal to" to signify be
was in this case being used for equality comparison:
Some(2) was not equal to Some(1)
If you wish to check the opposite of some condition, you can simply insert not in the expression.
Here are a few examples:
result must not be (null)
sum must not be <= (10)
mylist must not equal (yourList)
string must not startWith ("Hello")
Often when creating libraries you may wish to ensure that certain arrangements of code that
represent potential “user errors” do not compile, so that your library is more error resistant.
ScalaTest Matchers trait includes the following syntax for that purpose:
"val a: String = 1" mustNot compile
If you want to ensure that a snippet of code does not compile because of a type error (as opposed to a syntax error), use:
"val a: String = 1" mustNot typeCheck
Note that the mustNot typeCheck syntax will only succeed if the given snippet of code does not
compile because of a type error. A syntax error will still result on a thrown TestFailedException.
If you want to state that a snippet of code does compile, you can make that more obvious with:
"val a: Int = 1" must compile
Although the previous three constructs are implemented with macros that determine at compile time whether the snippet of code represented by the string does or does not compile, errors are reported as test failures at runtime.
and and or You can also combine matcher expressions with and and/or or, however,
you must place parentheses or curly braces around the and or or expression. For example,
this and-expression would not compile, because the parentheses are missing:
map must contain key ("two") and not contain value (7) // ERROR, parentheses missing!
Instead, you need to write:
map must (contain key ("two") and not contain value (7))
Here are some more examples:
number must (be > (0) and be <= (10))
option must (equal (Some(List(1, 2, 3))) or be (None))
string must (
equal ("fee") or
equal ("fie") or
equal ("foe") or
equal ("fum")
)
Two differences exist between expressions composed of these and and or operators and the expressions you can write
on regular Booleans using its && and || operators. First, expressions with and
and or do not short-circuit. The following contrived expression, for example, would print "hello, world!":
"yellow" must (equal ("blue") and equal { println("hello, world!"); "green" })
In other words, the entire and or or expression is always evaluated, so you'll see any side effects
of the right-hand side even if evaluating
only the left-hand side is enough to determine the ultimate result of the larger expression. Failure messages produced by these
expressions will "short-circuit," however,
mentioning only the left-hand side if that's enough to determine the result of the entire expression. This "short-circuiting" behavior
of failure messages is intended
to make it easier and quicker for you to ascertain which part of the expression caused the failure. The failure message for the previous
expression, for example, would be:
"yellow" did not equal "blue"
Most likely this lack of short-circuiting would rarely be noticeable, because evaluating the right hand side will usually not
involve a side effect. One situation where it might show up, however, is if you attempt to and a null check on a variable with an expression
that uses the variable, like this:
map must (not be (null) and contain key ("ouch"))
If map is null, the test will indeed fail, but with a NullArgumentException, not a
TestFailedException. Here, the NullArgumentException is the visible right-hand side effect. To get a
TestFailedException, you would need to check each assertion separately:
map must not be (null)
map must contain key ("ouch")
If map is null in this case, the null check in the first expression will fail with
a TestFailedException, and the second expression will never be executed.
The other difference with Boolean operators is that although && has a higher precedence than ||,
and and or
have the same precedence. Thus although the Boolean expression (a || b && c) will evaluate the && expression
before the || expression, like (a || (b && c)), the following expression:
traversable must (contain (7) or contain (8) and have size (9))
Will evaluate left to right, as:
traversable must ((contain (7) or contain (8)) and have size (9))
If you really want the and part to be evaluated first, you'll need to put in parentheses, like this:
traversable must (contain (7) or (contain (8) and have size (9)))
Options You can work with options using ScalaTest's equality, empty,
defined, and contain syntax.
For example, if you wish to check whether an option is None, you can write any of:
option mustEqual None option mustBe None option must === (None) option mustBe empty
If you wish to check an option is defined, and holds a specific value, you can write any of:
option mustEqual Some("hi")
option mustBe Some("hi")
option must === (Some("hi"))
If you only wish to check that an option is defined, but don't care what it's value is, you can write:
option mustBe defined
If you mix in (or import the members of) OptionValues,
you can write one statement that indicates you believe an option must be defined and then say something else about its value. Here's an example:
import org.scalatest.OptionValues._ option.value must be < 7
As mentioned previously, you can use also use ScalaTest's contain, contain oneOf, and
contain noneOf syntax with options:
Some(2) must contain (2) Some(7) must contain oneOf (5, 7, 9) Some(0) must contain noneOf (7, 8, 9)
have Using have, you can check properties of any type, where a property is an attribute of any
object that can be retrieved either by a public field, method, or JavaBean-style get
or is method, like this:
book must have (
'title ("Programming in Scala"),
'author (List("Odersky", "Spoon", "Venners")),
'pubYear (2008)
)
This expression will use reflection to ensure the title, author, and pubYear properties of object book
are equal to the specified values. For example, it will ensure that book has either a public Java field or method
named title, or a public method named getTitle, that when invoked (or accessed in the field case) results
in a the string "Programming in Scala". If all specified properties exist and have their expected values, respectively,
execution will continue. If one or more of the properties either does not exist, or exists but results in an unexpected value,
a TestFailedException will be thrown that explains the problem. (For the details on how a field or method is selected during this
process, see the documentation for HavePropertyMatcherGenerator.)
When you use this syntax, you must place one or more property values in parentheses after have, seperated by commas, where a property
value is a symbol indicating the name of the property followed by the expected value in parentheses. The only exceptions to this rule is the syntax
for checking size and length shown previously, which does not require parentheses. If you forget and put parentheses in, however, everything will
still work as you'd expect. Thus instead of writing:
array must have length (3) set must have size (90)
You can alternatively, write:
array must have (length (3)) set must have (size (90))
If a property has a value different from the specified expected value, a TestFailedError will be thrown
with a detailed message that explains the problem. For example, if you assert the following on
a book whose title is Moby Dick:
book must have ('title ("A Tale of Two Cities"))
You'll get a TestFailedException with this detail message:
The title property had value "Moby Dick", instead of its expected value "A Tale of Two Cities",
on object Book("Moby Dick", "Melville", 1851)
If you prefer to check properties in a type-safe manner, you can use a HavePropertyMatcher.
This would allow you to write expressions such as:
book must have (
title ("Programming in Scala"),
author (List("Odersky", "Spoon", "Venners")),
pubYear (2008)
)
These expressions would fail to compile if must is used on an inappropriate type, as determined
by the type parameter of the HavePropertyMatcher being used. (For example, title in this example
might be of type HavePropertyMatcher[org.publiclibrary.Book]. If used with an appropriate type, such an expression will compile
and at run time the property method or field will be accessed directly; i.e., no reflection will be used.
See the documentation for HavePropertyMatcher for more information.
length and size with HavePropertyMatchers If you want to use length or size syntax with your own custom HavePropertyMatchers, you
can do so, but you must write (of [“the type”]) afterwords. For example, you could write:
book must have (
title ("A Tale of Two Cities"),
length (220) (of [Book]),
author ("Dickens")
)
Prior to ScalaTest 2.0, “length (22)” yielded a HavePropertyMatcher[Any, Int] that used reflection to dynamically look
for a length field or getLength method. In ScalaTest 2.0, “length (22)” yields a
MatcherFactory1[Any, Length], so it is no longer a HavePropertyMatcher. The (of [<type>]) syntax converts the
the MatcherFactory1[Any, Length] to a HavePropertyMatcher[<type>, Int].
ScalaTest's Inside trait allows you to make assertions after a pattern match.
Here's an example:
case class Name(first: String, middle: String, last: String)
val name = Name("Jane", "Q", "Programmer")
inside(name) { case Name(first, _, _) =>
first must startWith ("S")
}
You can use inside to just ensure a pattern is matched, without making any further assertions, but a better
alternative for that kind of assertion is matchPattern. The matchPattern syntax allows you
to express that you expect a value to match a particular pattern, no more and no less:
name must matchPattern { case Name("Sarah", _, _) => }
If none of the built-in matcher syntax (or options shown so far for extending the syntax) satisfy a particular need you have, you can create
custom Matchers that allow
you to place your own syntax directly after must. For example, class java.io.File has a method isHidden, which
indicates whether a file of a certain path and name is hidden. Because the isHidden method takes no parameters and returns Boolean,
you can call it using be with a symbol or BePropertyMatcher, yielding assertions like:
file must be ('hidden) // using a symbol
file must be (hidden) // using a BePropertyMatcher
If it doesn't make sense to have your custom syntax follow be, you might want to create a custom Matcher
instead, so your syntax can follow must directly. For example, you might want to be able to check whether
a java.io.File's name ends with a particular extension, like this:
// using a plain-old Matcher
file must endWithExtension ("txt")
ScalaTest provides several mechanism to make it easy to create custom matchers, including ways to compose new matchers
out of existing ones complete with new error messages. For more information about how to create custom
Matchers, please see the documentation for the Matcher trait.
Sometimes you need to test whether a method throws an expected exception under certain circumstances, such
as when invalid arguments are passed to the method. With Matchers mixed in, you can
check for an expected exception like this:
an [IndexOutOfBoundsException] must be thrownBy s.charAt(-1)
If charAt throws an instance of StringIndexOutOfBoundsException,
this expression will result in that exception. But if charAt completes normally, or throws a different
exception, this expression will complete abruptly with a TestFailedException.
If you need to further isnpect an expected exception, you can capture it using this syntax:
val thrown = the [IndexOutOfBoundsException] thrownBy s.charAt(-1)
This expression returns the caught exception so that you can inspect it further if you wish, for example, to ensure that data contained inside the exception has the expected values. Here's an example:
thrown.getMessage must equal ("String index out of range: -1")
If you prefer you can also capture and inspect an expected exception in one statement, like this:
the [ArithmeticException] thrownBy 1 / 0 must have message "/ by zero"
the [IndexOutOfBoundsException] thrownBy {
s.charAt(-1)
} must have message "String index out of range: -1"
You can also state that no exception must be thrown by some code, like this:
noException must be thrownBy 0 / 1
Perhaps the most tricky part of writing assertions using ScalaTest matchers is remembering when you need or don't need parentheses, but bearing in mind a few simple rules should help. It is also reassuring to know that if you ever leave off a set of parentheses when they are required, your code will not compile. Thus the compiler will help you remember when you need the parens. That said, the rules are:
1. Although you don't always need them, you may choose to always put parentheses
around right-hand values, such as the 7 in num must equal (7):
result must equal (4) array must have length (3) book must have ( 'title ("Programming in Scala"), 'author (List("Odersky", "Spoon", "Venners")), 'pubYear (2008) ) option must be ('defined) catMap must (contain key (9) and contain value ("lives")) keyEvent must be an ('actionKey) javaSet must have size (90)
2. Except for length, size and message, you must always put parentheses around
the list of one or more property values following a have:
file must (exist and have ('name ("temp.txt"))) book must have ( title ("Programming in Scala"), author (List("Odersky", "Spoon", "Venners")), pubYear (2008) ) javaList must have length (9) // parens optional for length and size
3. You must always put parentheses around and and or expressions, as in:
catMap must (contain key (9) and contain value ("lives")) number must (equal (2) or equal (4) or equal (8))
4. Although you don't always need them, you may choose to always put parentheses
around custom Matchers when they appear directly after not:
file must exist file must not (exist) file must (exist and have ('name ("temp.txt"))) file must (not (exist) and have ('name ("temp.txt")) file must (have ('name ("temp.txt") or exist) file must (have ('name ("temp.txt") or not (exist))
That's it. With a bit of practice it should become natural to you, and the compiler will always be there to tell you if you forget a set of needed parentheses.
Note: ScalaTest's matchers are in part inspired by the matchers of RSpec,
Hamcrest, and
specs2, and its “mustNot compile” syntax
by the illTyped macro of shapeless.
Trait that can be mixed into a Suite to disable the implicit conversions provided by default in trait
Assertions, which trait Suite extends.
Trait that can be mixed into a Suite to disable the implicit conversions provided by default in trait
Assertions, which trait Suite extends.
This trait can be used to quickly solve a problem in which ScalaTest's default implicit conversion is clashing with those of some other library you need to use in your tests. After mixing in this trait, like this:
class MySuite extends FunSuite with NonImplicitAssertions {
// ...
}
You can write tests using assert (without triple equals), assertResult, assertThrows,
intercept, assertCompiles, assertDoesNotCompile, and assertTypeError:
assert(a < 7)
assertResult(2) { 1 + 1 }
assertThrows[IndexOutOfBoundsException] {
"hi".charAt(-1)
}
val caught =
intercept[IndexOutOfBoundsException] {
"hi".charAt(-1)
}
assertDoesNotCompile("val a: String = 1")
assertTypeError("val a: String = 1")
assertCompiles("val a: Int = 1")
Trait providing an apply method to which status updates about a running suite of tests can be reported.
Trait providing an apply method to which status updates about a running suite of tests can be reported.
An Notifier is essentially
used to wrap a Reporter and provide easy ways to send status updates
to that Reporter via an NoteProvided event.
Notifier contains an apply method that takes a string and
an optional payload object of type Any.
The Notifier will forward the passed alert message string to the
Reporter as the message parameter, and the optional
payload object as the payload parameter, of an NoteProvided event.
For insight into the differences between Notifier, Alerter, and Informer, see the
main documentation for trait Notifying.
Trait that contains the note method, which can be used to send a status notification to the reporter.
Trait that contains the note method, which can be used to send a status notification to the reporter.
The difference between note and the info method of Informer is that
info messages provided during a test are recorded and sent as part of test completion event, whereas
note messages are sent right away as NoteProvided messages. For long-running tests,
note allows you to send "status notifications" to the reporter right away, so users can track the
progress of the long-running tests. By contrast, info messages will only be seen by the user after the
test has completed, and are more geared towards specification (such as Given/When/Then messages) than notification.
The difference between note and the alert method of Alerting is
that alert is intended to be used
for warnings or notifications of potential problems, whereas note is just for status notifications.
In string reporters for which ANSI color is enabled, note notifications are shown in green and alert notifications
in yellow.
Trait that facilitates a style of testing in which each test is run in its own instance of the suite class to isolate each test from the side effects of the other tests in the suite.
Trait that facilitates a style of testing in which each test is run in its own instance of the suite class to isolate each test from the side effects of the other tests in the suite.
Recommended Usage: Trait OneInstancePerTest is intended primarily to serve as a supertrait for
ParallelTestExecution and the path traits, to
facilitate porting JUnit tests to ScalaTest, and to make it easy for users who prefer JUnit's approach to isolation to obtain similar
behavior in ScalaTest.
|
If you mix this trait into a Suite, you can initialize shared reassignable
fixture variables as well as shared mutable fixture objects in the constructor of the
class. Because each test will run in its own instance of the class, each test will
get a fresh copy of the instance variables. This is the approach to test isolation taken,
for example, by the JUnit framework. OneInstancePerTest can, therefore,
be handy when porting JUnit tests to ScalaTest.
Here's an example of OneInstancePerTest being used in a FunSuite:
import org.scalatest.FunSuite
import org.scalatest.OneInstancePerTest
import collection.mutable.ListBuffer
class MySuite extends FunSuite with OneInstancePerTest {
val builder = new StringBuilder("ScalaTest is ")
val buffer = new ListBuffer[String]
test("easy") {
builder.append("easy!")
assert(builder.toString === "ScalaTest is easy!")
assert(buffer.isEmpty)
buffer += "sweet"
}
test("fun") {
builder.append("fun!")
assert(builder.toString === "ScalaTest is fun!")
assert(buffer.isEmpty)
}
}
OneInstancePerTest is supertrait to ParallelTestExecution, in which
running each test in its own instance is intended to make it easier to write suites of tests that run in parallel (by reducing the likelihood
of concurrency bugs in those suites.) OneInstancePerTest is also supertrait to the path traits,
path.FunSpec and path.FreeSpec, to make it obvious
these traits run each test in a new, isolated instance.
For the details on how OneInstancePerTest works, see the documentation for methods runTests and runTest,
which this trait overrides.
Trait that provides an implicit conversion that adds a value method
to Option, which will return the value of the option if it is defined,
or throw TestFailedException if not.
Trait that provides an implicit conversion that adds a value method
to Option, which will return the value of the option if it is defined,
or throw TestFailedException if not.
This construct allows you to express in one statement that an option should be defined and that its value should meet some expectation. Here's an example:
opt.value should be > 9
Or, using an assertion instead of a matcher expression:
assert(opt.value > 9)
Were you to simply invoke get on the Option,
if the option wasn't defined, it would throw a NoSuchElementException:
val opt: Option[Int] = None opt.get should be > 9 // opt.get throws NoSuchElementException
The NoSuchElementException would cause the test to fail, but without providing a stack depth pointing
to the failing line of test code. This stack depth, provided by TestFailedException (and a
few other ScalaTest exceptions), makes it quicker for
users to navigate to the cause of the failure. Without OptionValues, to get
a stack depth exception you would need to make two statements, like this:
val opt: Option[Int] = None
opt should be ('defined) // throws TestFailedException
opt.get should be > 9
The OptionValues trait allows you to state that more concisely:
val opt: Option[Int] = None opt.value should be > 9 // opt.value throws TestFailedException
Superclass for the possible outcomes of running a test.
Superclass for the possible outcomes of running a test.
Outcome is the result type of the withFixture methods of traits
Suite and fixture.Suite, as well as their
NoArgTest and OneArgTest function types.
The four possible outcomes are:
Succeeded - indicates a test succeededFailed - indicates a test failed and contains an exception describing the failureCanceled - indicates a test was canceled and contains an exception describing the cancelationPending - indicates a test was pendingNote that "ignored" does not appear as a type of Outcome, because tests are
marked as ignored on the outside and skipped over as the suite executes. So an ignored test never runs, and therefore
never has an outcome. By contrast, a test is determined to be pending by running the test
and observing the actual outcome. If the test body completes abruptly with a TestPendingException,
then the outcome was that the test was pending.
Trait that contains the outcomeOf method, which executes a passed code block and
transforms the outcome into an Outcome, using the
same mechanism used by ScalaTest to produce an Outcome when executing
a test.
Trait that causes that the tests of any suite it is mixed into to be run in parallel if
a Distributor is passed to runTests.
Trait that causes that the tests of any suite it is mixed into to be run in parallel if
a Distributor is passed to runTests.
ScalaTest's normal approach for running suites of tests in parallel is to run different suites in parallel, but the tests of any one suite sequentially. This approach should provide sufficient distribution of the work load in most cases, but some suites may encapsulate multiple long-running tests. Such suites may dominate the execution time of the run. If so, mixing in this trait into just those suites will allow their long-running tests to run in parallel with each other, thereby helping to reduce the total time required to run an entire run.
To make it easier for users to write tests that run in parallel, this trait runs each test in its own instance of the class.
Running each test in its own instance enables tests to use the same instance vars and mutable objects referenced from
instance variables without needing to synchronize. Although ScalaTest provides functional approaches to
factoring out common test code that can help avoid such issues, running each test in its own instance is an insurance policy that makes
running tests in parallel easier and less error prone.
For the details on how ParallelTestExecution works, see the documentation for methods run, runTests, and runTest,
which this trait overrides.
Note: This trait's implementation of runTest is final, to ensure that behavior
related to individual tests are executed by the same thread that executes the actual test. This means,
for example, that you won't be allowed to write ...with ParallelTestExecution with BeforeAndAfter.
Instead, you'd need to put ParallelTestExecution last, as
in: with BeforeAndAfter with ParallelTestExecution. For more details, see the documentation
for the runTest method.
Trait that provides an implicit conversion that adds a valueAt method
to PartialFunction, which will return the value (result) of the function applied to the argument passed to valueAt,
or throw TestFailedException if the partial function is not defined at the argument.
Trait that provides an implicit conversion that adds a valueAt method
to PartialFunction, which will return the value (result) of the function applied to the argument passed to valueAt,
or throw TestFailedException if the partial function is not defined at the argument.
This construct allows you to express in one statement that a partial function should be defined for a particular input, and that its result value should meet some expectation. Here's an example:
pf.valueAt("IV") should equal (4)
Or, using an assertion instead of a matcher expression:
assert(pf.valueAt("IV") === 4)
Were you to simply invoke apply on the PartialFunction, passing in an input value,
if the partial function wasn't defined at that input, it would throw some exception, but likely not one
that provides a stack depth:
// Note: a Map[K, V] is a PartialFunction[K, V]
val pf: PartialFunction[String, Int] = Map("I" -> 1, "II" -> 2, "III" -> 3, "IV" -> 4)
pf("V") should equal (5) // pf("V") throws NoSuchElementException
The NoSuchElementException thrown in this situation would cause the test to fail, but without providing a stack depth pointing
to the failing line of test code. This stack depth, provided by TestFailedException (and a
few other ScalaTest exceptions), makes it quicker for
users to navigate to the cause of the failure. Without PartialFunctionValues, to get
a stack depth exception you would need to make two statements, like this:
val pf: PartialFunction[String, Int] = Map("I" -> 1, "II" -> 2, "III" -> 3, "IV" -> 4)
pf.isDefinedAt("V") should be (true) // throws TestFailedException
pf("V") should equal (5)
The PartialFunctionValues trait allows you to state that more concisely:
val pf: PartialFunction[String, Int] = Map("I" -> 1, "II" -> 2, "III" -> 3, "IV" -> 4)
pf.valueAt("V") should equal (5) // pf.valueAt("V") throws TestFailedException
Trait facilitating the inclusion of a payload in a thrown ScalaTest exception.
Trait facilitating the inclusion of a payload in a thrown ScalaTest exception.
This trait includes a withPayload construct
that enables a payload object (or modified
payload object) to be included as the payload of a thrown exception.
Many ScalaTest events include an optional "payload" field that can be used
to pass information to a custom reporter. This trait facilitates such customization,
by making it easy to insert a payload into a thrown exception, such as a TestFailedException.
The thrown exception must mix in Payload.
ScalaTest looks for trait Payload and fires any payloads it finds in the relevant ScalaTest event
stimulated by the exception, such as a TestFailed event stimulated by a TestFailedException.
Here's an example in which a GUI snapshot is included as a payload when a test fails:
withPayload(generateGUISnapshot()) {
1 + 1 should === (3)
}
Trait mixed into the result type of the pending statement of trait Assertions, which always throws TestPendingException.
Trait mixed into the result type of the pending statement of trait Assertions, which always throws TestPendingException.
This type is used primarily to facilitate the is (pending) syntax of
traits FlatSpec, WordSpec, and
FLatSpec as well the
is (pending) or (pending) syntax of sibling traits
in the org.scalatest.fixture package. Because the pending
method in Assertions always completes abruptly with an exception, its
type would be inferred to be Nothing, which is a relatively common
type. To make sure syntax like is (pending) only works with
method pending, it is helpful to have a specially named
"Nothing" type.
Trait that facilitates the testing of private methods.
Trait that facilitates the testing of private methods.
To test a private method, mix in trait PrivateMethodTester and
create a PrivateMethod object, like this:
val decorateToStringValue = PrivateMethod[String]('decorateToStringValue)
The type parameter on PrivateMethod, in this case String, is the result type of the private method
you wish to invoke. The symbol passed to the PrivateMethod.apply factory method, in this
case 'decorateToStringValue, is the name of the private method to invoke. To test
the private method, use the invokePrivate operator, like this:
targetObject invokePrivate decorateToStringValue(1)
Here, targetObject is a variable or singleton object name referring to the object whose
private method you want to test. You pass the arguments to the private method in the parentheses after
the PrivateMethod object.
The result type of an invokePrivate operation will be the type parameter of the PrivateMethod
object, thus you need not cast the result to use it. In other words, after creating a PrivateMethod object, the
syntax to invoke the private method
looks like a regular method invocation, but with the dot (.) replaced by invokePrivate.
The private method is invoked dynamically via reflection, so if you have a typo in the method name symbol, specify the wrong result type,
or pass invalid parameters, the invokePrivate operation will compile, but throw an exception at runtime.
One limitation to be aware of is that you can't use PrivateMethodTester to test a private method
declared in a trait, because the class the trait gets mixed into will not declare that private method. Only the
class generated to hold method implementations for the trait will have that private method. If you want to
test a private method declared in a trait, and that method does not use any state of that trait, you can move
the private method to a companion object for the trait and test it using PrivateMethodTester that
way. If the private trait method you want to test uses the trait's state, your best options are to test it
indirectly via a non-private trait method that calls the private method, or make the private method package access
and test it directly via regular static method invocations.
Also, if you want to use PrivateMethodTester to invoke a parameterless private method, you'll need to use
empty parens. Instead of:
targetObject invokePrivate privateParameterlessMethod
You'll need to write:
targetObject invokePrivate privateParameterlessMethod()
Trait that causes tests to be run in pseudo-random order.
Trait that causes tests to be run in pseudo-random order.
Although the tests are run in pseudo-random order, events will be fired in the “normal” order for the Suite
that mixes in this trait, as determined by runTests.
The purpose of this trait is to reduce the likelihood of unintentional order dependencies between tests in the same test class.
Offers two methods for transforming futures when exceptions are expected.
Offers two methods for transforming futures when exceptions are expected.
This trait offers two methods for testing for expected exceptions in the context of
futures: recoverToSucceededIf and recoverToExceptionIf.
Because this trait is mixed into trait AsyncTestSuite, both of its methods are
available by default in any async-style suite.
If you just want to ensure that a future fails with a particular exception type, and do
not need to inspect the exception further, use recoverToSucceededIf:
recoverToSucceededIf[IllegalStateException] { // Result type: Future[Assertion]
emptyStackActor ? Peek
}
The recoverToSucceededIf method performs a job similar to
assertThrows, except
in the context of a future. It transforms a Future of any type into a
Future[Assertion] that succeeds only if the original future fails with the specified
exception. Here's an example in the REPL:
scala> import org.scalatest.RecoverMethods._
import org.scalatest.RecoverMethods._
scala> import scala.concurrent.Future
import scala.concurrent.Future
scala> import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.ExecutionContext.Implicits.global
scala> recoverToSucceededIf[IllegalStateException] {
| Future { throw new IllegalStateException }
| }
res0: scala.concurrent.Future[org.scalatest.Assertion] = ...
scala> res0.value
res1: Option[scala.util.Try[org.scalatest.Assertion]] = Some(Success(Succeeded))
Otherwise it fails with an error message similar to those given by assertThrows:
scala> recoverToSucceededIf[IllegalStateException] {
| Future { throw new RuntimeException }
| }
res2: scala.concurrent.Future[org.scalatest.Assertion] = ...
scala> res2.value
res3: Option[scala.util.Try[org.scalatest.Assertion]] =
Some(Failure(org.scalatest.exceptions.TestFailedException: Expected exception
java.lang.IllegalStateException to be thrown, but java.lang.RuntimeException
was thrown))
scala> recoverToSucceededIf[IllegalStateException] {
| Future { 42 }
| }
res4: scala.concurrent.Future[org.scalatest.Assertion] = ...
scala> res4.value
res5: Option[scala.util.Try[org.scalatest.Assertion]] =
Some(Failure(org.scalatest.exceptions.TestFailedException: Expected exception
java.lang.IllegalStateException to be thrown, but no exception was thrown))
The recoverToExceptionIf method differs from the recoverToSucceededIf in
its behavior when the assertion succeeds: recoverToSucceededIf yields a Future[Assertion],
whereas recoverToExceptionIf yields a Future[T], where T is the
expected exception type.
recoverToExceptionIf[IllegalStateException] { // Result type: Future[IllegalStateException]
emptyStackActor ? Peek
}
In other words, recoverToExpectionIf is to
intercept as
recovertToSucceededIf is to assertThrows. The first one allows you to perform further
assertions on the expected exception. The second one gives you a result type that will satisfy the type checker
at the end of the test body. Here's an example showing recoverToExceptionIf in the REPL:
scala> val futureEx =
| recoverToExceptionIf[IllegalStateException] {
| Future { throw new IllegalStateException("hello") }
| }
futureEx: scala.concurrent.Future[IllegalStateException] = ...
scala> futureEx.value
res6: Option[scala.util.Try[IllegalStateException]] =
Some(Success(java.lang.IllegalStateException: hello))
scala> futureEx map { ex => assert(ex.getMessage == "world") }
res7: scala.concurrent.Future[org.scalatest.Assertion] = ...
scala> res7.value
res8: Option[scala.util.Try[org.scalatest.Assertion]] =
Some(Failure(org.scalatest.exceptions.TestFailedException: "[hello]" did not equal "[world]"))
Trait whose instances collect the results of a running suite of tests and presents those results in some way to the user.
Trait whose instances collect the results of a running suite of tests and presents those results in some way to the user. Instances of this trait can be called "report functions" or "reporters."
Reporters receive test results via fifteen events. Each event is fired to pass a particular kind of information to the reporter. The events are:
DiscoveryStartingDiscoveryCompletedRunStartingRunStoppedRunAbortedRunCompletedScopeOpenedScopeClosedScopePendingTestStartingTestSucceededTestFailedTestCanceledTestIgnoredTestPendingSuiteStartingSuiteCompletedSuiteAbortedInfoProvidedMarkupProvidedAlertProvidedNoteProvidedReporters may be implemented such that they only present some of the reported events to the user. For example, you could
define a reporter class that does nothing in response to SuiteStarting events.
Such a class would always ignore SuiteStarting events.
The term test as used in the TestStarting, TestSucceeded,
and TestFailed event names
is defined abstractly to enable a wide range of test implementations.
ScalaTest's style traits (subclasse of trait Suite) fire
TestStarting to indicate they are about to invoke one
of their tests, TestSucceeded to indicate a test returned normally,
and TestFailed to indicate a test completed abruptly with an exception.
Although the execution of a Suite subclass's tests will likely be a common event
reported via the
TestStarting, TestSucceeded, and TestFailed events, because
of the abstract definition of “test” used by the
the event classes, these events are not limited to this use. Information about any conceptual test
may be reported via the TestStarting, TestSucceeded, and
TestFailed events.
Likewise, the term suite as used in the SuiteStarting, SuiteAborted,
and SuiteCompleted event names
is defined abstractly to enable a wide range of suite implementations.
Object Runner fires SuiteStarting to indicate it is about to invoke
run on a
Suite, SuiteCompleted to indicate a Suite's
run method returned normally,
and SuiteAborted to indicate a Suite's run
method completed abruptly with an exception.
Similarly, class Suite fires SuiteStarting to indicate it is about to invoke
run on a
nested Suite, SuiteCompleted to indicate a nested Suite's
run method returned normally,
and SuiteAborted to indicate a nested Suite's run
method completed abruptly with an exception.
Although the execution of a Suite's run method will likely be a
common event reported via the
SuiteStarting, SuiteAborted, and SuiteCompleted events, because
of the abstract definition of "suite" used by the
event classes, these events are not limited to this use. Information about any conceptual suite
may be reported via the SuiteStarting, SuiteAborted, and
SuiteCompleted events.
You can create classes that extend Reporter to report test results in custom ways, and to
report custom information passed as an event "payload."
Reporter classes can handle events in any manner, including doing nothing.
Subtrait of Reporter that contains a dispose method for
releasing any finite, non-memory resources, such as file handles, held by the
Reporter.
Subtrait of Reporter that contains a dispose method for
releasing any finite, non-memory resources, such as file handles, held by the
Reporter. Runner will invoke dispose on
any ResourcefulReporter when it no longer needs the Reporter.
Provides methods that can be used in withFixture implementations to retry tests in various scenarios.
Provides methods that can be used in withFixture implementations to retry tests in various scenarios.
Trait Retries is intended to help you deal with “flickers”—tests that usually pass, but
occasionally fail. The best way to deal with such tests is to fix them so they always pass. Sometimes, however, this is
not practical. In such cases, flickers can waste your time by forcing you to investigate test failures that turn
out to be flickers. Or worse, like the boy who cried wolf, the flickers may train you an your colleagues to not pay attention
to failures such that you don't notice real problems, at least not in a timely manner.
Trait Retries offers methods that will retry a failed and/or canceled test once, on the same thread,
with or without a delay. These methods take a block that results in Outcome,
and are intended to be used in withFixture methods. You should be very selective about which tests you
retry, retrying those for which you have good evidence to conclude they are flickers. Thus it is recommended you
only retry tests that are tagged with Retryable, and only tag tests as such once they have flickered
consistently for a while, and only after you invested a reasonable effort into fixing them properly.
Here's an example showing how you might use Retries:
package org.scalatest.examples.tagobjects.retryable
import org.scalatest._
import tagobjects.Retryable
class SetSpec extends FlatSpec with Retries {
override def withFixture(test: NoArgTest) = {
if (isRetryable(test))
withRetry { super.withFixture(test) }
else
super.withFixture(test)
}
"An empty Set" should "have size 0" taggedAs(Retryable) in {
assert(Set.empty.size === 0)
}
}
A Suite class mixing in SequentialNestedSuiteExecution that takes
zero to many Suites, which will be returned from its nestedSuites method.
A Suite class mixing in SequentialNestedSuiteExecution that takes
zero to many Suites, which will be returned from its nestedSuites method.
For example, you can define a suite that always executes a list of nested suites like this:
class StepsSuite extends Sequential( new Step1Suite, new Step2Suite, new Step3Suite, new Step4Suite, new Step5Suite )
When StepsSuite is executed, it will execute its
nested suites in the passed order: Step1Suite, Step2Suite,
Step3Suite, Step4Suite, and Step5Suite.
Because Sequential extends SequentialNestedSuiteExecution,
the distributor passed to runNestedSuites will always be None.
So not only will the suites passed to the constructor be executed sequentially, any
tests and nested suites of the passed suites will also be executed sequentually.
The difference between Sequential and Stepwise
is that although Stepwise executes its own nested suites sequentially, it passes
whatever distributor was passed to it to those nested suites. Thus the nested suites could run their own nested
suites and tests in parallel if that distributor is defined. By contrast, Sequential always
passes None for the distributor to the nested suites, so any and every test and nested suite
contained within the nested suites passed to the Sequential construtor will be executed sequentially.
NullArgumentException if suitesToNest, or any suite
it contains, is null.
Trait that causes the nested suites of any suite it is mixed into to be run sequentially even if
a Distributor is passed to runNestedSuites.
Trait that causes the nested suites of any suite it is mixed into to be run sequentially even if
a Distributor is passed to runNestedSuites. This trait overrides the
runNestedSuites method and fowards every parameter passed to it to a superclass invocation
of runNestedSuites, except it always passes None for the Distributor.
Mix in this trait into any suite whose nested suites need to be run sequentially even with the rest of the
run is being executed concurrently.
Status implementation that can change its state over time.
Status implementation that can change its state over time.
A StatefulStatus begins its life in a successful state, and will remain successful unless setFailed is called.
Once setFailed is called, the status will remain at failed. The setFailed method can be called multiple times (even
though invoking it once is sufficient to permanently set the status to failed), but only up until setCompleted has been called.
After setCompleted has been called, any invocation of setFailed will be greeted with an IllegalStateException.
Instances of this class are thread safe.
The result status of running a test or a suite, which is used to support parallel and asynchronous execution of tests.
The result status of running a test or a suite, which is used to support parallel and asynchronous execution of tests.
This trait is the result type of the "run" lifecycle methods of trait Suite:
run, runNestedSuites, runTests, and runTest. It can be used to determine whether
a test or suite has completed, and if so, whether it succeeded, and if not, whether an exception was thrown that was
not yet reported via a ScalaTest event. A Status is like a domain-specific Future[Boolean], where:
Success(true)TestFailedException)
is represented by Success(false)Failure(unreportedException)Note that pending and canceled tests will not cause a Status to fail. Only failed tests
and aborted suites will cause a Status to fail.
One use case of Status is to ensure that "after" code (such as an afterEach or afterAll method)
does not execute until after the relevant entity (one test, one suite, or all of a suite's tests or nested suites) has completed.
Another use case is to implement the default behavior of asynchronous styles, in which subsequent each test does not begin
execution until after the previous test has completed.
A Suite class that takes zero to many Suites,
which will be returned from its nestedSuites method and
executed in “stepwise” fashion by its runNestedSuites method.
A Suite class that takes zero to many Suites,
which will be returned from its nestedSuites method and
executed in “stepwise” fashion by its runNestedSuites method.
For example, you can define a suite that always executes a list of nested suites like this:
class StepsSuite extends Stepwise( new Step1Suite, new Step2Suite, new Step3Suite, new Step4Suite, new Step5Suite )
When StepsSuite is executed, regardless of whether a Distributor
is passed, it will execute its
nested suites sequentially in the passed order: Step1Suite, Step2Suite,
Step3Suite, Step4Suite, and Step5Suite.
The difference between Stepwise and Sequential
is that although Stepwise executes its own nested suites sequentially, it passes
whatever distributor was passed to it to those nested suites. Thus the nested suites could run their own nested
suites and tests in parallel if that distributor is defined. By contrast, Sequential always
passes None for the distributor to the nested suites, so any and every test and nested suite
contained within the nested suites passed to the Sequential construtor will be executed sequentially.
NullArgumentException if suitesToNest, or any suite
it contains, is null.
Trait that causes the nested suites of any suite it is mixed into to be run sequentially even if
a Distributor is passed to runNestedSuites.
Trait that causes the nested suites of any suite it is mixed into to be run sequentially even if
a Distributor is passed to runNestedSuites. This trait overrides the
runNestedSuites method and fowards every parameter passed to it to a superclass invocation
of runNestedSuites, except it always passes None for the Distributor.
Mix in this trait into any suite whose nested suites need to be run sequentially even with the rest of the
run is being executed concurrently.
Trait whose instances can accept a stop request and indicate whether a stop has already been requested.
Trait whose instances can accept a stop request and indicate whether a stop has already been requested.
This is passed in
to the run method of Suite, so that running suites of tests can be
requested to stop early.
A suite of tests.
A suite of tests. A Suite instance encapsulates a conceptual
suite (i.e., a collection) of tests.
This trait provides an interface composed of "lifecycle methods" that allow suites of tests to be run.
Its implementation enables a default way of writing and executing tests. Subtraits and subclasses can
override Suite's lifecycle methods to enable other ways of writing and executing tests.
A Suite can refer to a collection of other Suites,
which are called nested Suites. Those nested Suites can in turn have
their own nested Suites, and so on. Large test suites can be organized, therefore, as a tree of
nested Suites.
This trait's run method, in addition to invoking its
test methods, invokes run on each of its nested Suites.
A List of a Suite's nested Suites can be obtained by invoking its
nestedSuites method. If you wish to create a Suite that serves as a
container for nested Suites, whether or not it has test methods of its own, simply override nestedSuites
to return a List of the nested Suites. Because this is a common use case, ScalaTest provides
a convenience Suites class, which takes a variable number of nested Suites as constructor
parameters. Here's an example:
package org.scalatest.examples.suite.nested
import org.scalatest._
class ASuite extends FunSuite {
test("A should have ASCII value 41 hex") {
assert('A' === 0x41)
}
test("a should have ASCII value 61 hex") {
assert('a' === 0x61)
}
}
class BSuite extends FunSuite {
test("B should have ASCII value 42 hex") {
assert('B' === 0x42)
}
test("b should have ASCII value 62 hex") {
assert('b' === 0x62)
}
}
class CSuite extends FunSuite {
test("C should have ASCII value 43 hex") {
assert('C' === 0x43)
}
test("c should have ASCII value 63 hex") {
assert('c' === 0x63)
}
}
class ASCIISuite extends Suites(
new ASuite,
new BSuite,
new CSuite
)
If you now run ASCIISuite:
scala> org.scalatest.run(new ASCIISuite)
You will see reports printed to the standard output that indicate the nested
suites—ASuite, BSuite, and
CSuite—were run:
ASCIISuite:
ASuite:
- A should have ASCII value 41 hex
- a should have ASCII value 61 hex
BSuite:
- B should have ASCII value 42 hex
- b should have ASCII value 62 hex
CSuite:
- C should have ASCII value 43 hex
- c should have ASCII value 63 hex
Note that Runner can discover Suites automatically, so you need not
necessarily define nested Suites explicitly. See the documentation
for Runner for more information.
In some cases you may need to pass information to a suite of tests.
For example, perhaps a suite of tests needs to grab information from a file, and you want
to be able to specify a different filename during different runs. You can accomplish this in ScalaTest by passing
the filename in a config map of key-value pairs, which is passed to run as a ConfigMap.
The values in the config map are called "config objects," because they can be used to configure
suites, reporters, and tests.
You can specify a string config object is via the ScalaTest Runner, either via the command line
or ScalaTest's ant task.
(See the documentation for Runner for information on how to specify
config objects on the command line.)
The config map is passed to run, runNestedSuites, runTests, and runTest,
so one way to access it in your suite is to override one of those methods. If you need to use the config map inside your tests, you
can access it from the NoArgTest passed to withFixture, or the OneArgTest passed to
withFixture in the traits in the org.scalatest.fixture package. (See the
documentation for fixture.Suite
for instructions on how to access the config map in tests.)
The run method takes as one of its parameters an optional Distributor. If
a Distributor is passed in, this trait's implementation of run puts its nested
Suites into the distributor rather than executing them directly. The caller of run
is responsible for ensuring that some entity runs the Suites placed into the
distributor. The -P command line parameter to Runner, for example, will cause
Suites put into the Distributor to be run in parallel via a pool of threads.
If you wish to execute the tests themselves in parallel, mix in ParallelTestExecution.
The Javadoc documentation for java.lang.Error states:
AnErroris a subclass ofThrowablethat indicates serious problems that a reasonable application should not try to catch. Most such errors are abnormal conditions.
Because Errors are used to denote serious errors, trait Suite and its subtypes in the ScalaTest API
do not always treat a test that completes abruptly with an Error as a test failure, but sometimes as an indication
that serious problems have arisen that should cause the run to abort. For example, if a test completes abruptly with an
OutOfMemoryError, it will not be reported as a test failure, but will instead cause the run to abort. Because not
everyone uses Errors only to represent serious
problems, however, ScalaTest only behaves this way for the following run-aborting exception types (and their subclasses):
java.lang.annotation.AnnotationFormatErrorjava.awt.AWTErrorjava.nio.charset.CoderMalfunctionErrorjavax.xml.parsers.FactoryConfigurationErrorjava.lang.LinkageErrorjava.lang.ThreadDeathjavax.xml.transform.TransformerFactoryConfigurationErrorjava.lang.VirtualMachineErrorThe previous list includes all Errors that exist as part of Java 1.5 API, excluding java.lang.AssertionError.
ScalaTest does treat a thrown AssertionError as an indication of a test failure. In addition, any other
Error that is not an instance of a type mentioned in the previous list will be caught by the Suite traits
in the ScalaTest API and reported as the cause of a test failure.
Although trait Suite and all its subtypes in the ScalaTest API consistently behave this way with regard to Errors,
this behavior is not required by the contract of Suite. Subclasses and subtraits that you define, for example, may treat all
Errors as test failures, or indicate errors in some other way that has nothing to do with exceptions.
Trait Suite provides default implementations of its methods that should
be sufficient for most applications, but many methods can be overridden when desired. Here's
a summary of the methods that are intended to be overridden:
run - override this method to define custom ways to run suites of
tests.runNestedSuites - override this method to define custom ways to run nested suites.runTests - override this method to define custom ways to run a suite's tests.runTest - override this method to define custom ways to run a single named test.testNames - override this method to specify the Suite's test names in a custom way.tags - override this method to specify the Suite's test tags in a custom way.nestedSuites - override this method to specify the Suite's nested Suites in a custom way.suiteName - override this method to specify the Suite's name in a custom way.expectedTestCount - override this method to count this Suite's expected tests in a custom way.For example, this trait's implementation of testNames performs reflection to discover methods starting with test,
and places these in a Set whose iterator returns the names in alphabetical order. If you wish to run tests in a different
order in a particular Suite, perhaps because a test named testAlpha can only succeed after a test named
testBeta has run, you can override testNames so that it returns a Set whose iterator returns
testBeta before testAlpha. (This trait's implementation of run will invoke tests
in the order they come out of the testNames Set iterator.)
Alternatively, you may not like starting your test methods with test, and prefer using @Test annotations in
the style of Java's JUnit 4 or TestNG. If so, you can override testNames to discover tests using either of these two APIs
@Test annotations, or one of your own invention. (This is in fact
how org.scalatest.junit.JUnitSuite and org.scalatest.testng.TestNGSuite work.)
Moreover, test in ScalaTest does not necessarily mean test method. A test can be anything that can be given a name,
that starts and either succeeds or fails, and can be ignored. In org.scalatest.FunSuite, for example, tests are represented
as function values. This
approach might look foreign to JUnit users, but may feel more natural to programmers with a functional programming background.
To facilitate this style of writing tests, FunSuite overrides testNames, runTest, and run such that you can
define tests as function values.
You can also model existing JUnit 3, JUnit 4, or TestNG tests as suites of tests, thereby incorporating tests written in Java into a ScalaTest suite.
The "wrapper" classes in packages org.scalatest.junit and org.scalatest.testng exist to make this easy.
No matter what legacy tests you may have, it is likely you can create or use an existing Suite subclass that allows you to model those tests
as ScalaTest suites and tests and incorporate them into a ScalaTest suite. You can then write new tests in Scala and continue supporting
older tests in Java.
Trait defining abstract "lifecycle" methods that are implemented in Suite and can
be overridden in stackable modification traits.
Trait defining abstract "lifecycle" methods that are implemented in Suite and can
be overridden in stackable modification traits.
The main purpose of SuiteMixin is to differentiate core Suite
style traits, such as Spec, FunSuite, and FunSpec from stackable
modification traits for Suites such as BeforeAndAfterEach, OneInstancePerTest,
and SequentialNestedSuiteExecution. Because these stackable traits extend SuiteMixin
instead of Suite, you can't define a suite by simply extending one of the stackable traits:
class MySuite extends BeforeAndAfterEach // Won't compile
Instead, you need to extend a core Suite trait and mix the stackable BeforeAndAfterEach trait
into that, like this:
class MySuite extends FunSuite with BeforeAndAfterEach // Compiles fine
A Suite class that takes zero to many Suites in its constructor,
which will be returned from its nestedSuites method.
A Suite class that takes zero to many Suites in its constructor,
which will be returned from its nestedSuites method.
For example, you can define a suite that always executes a list of nested suites like this:
class StepsSuite extends Suites( new Step1Suite, new Step2Suite, new Step3Suite, new Step4Suite, new Step5Suite )
If StepsSuite is executed sequentially, it will execute its
nested suites in the passed order: Step1Suite, Step2Suite,
Step3Suite, Step4Suite, and Step5Suite.
If StepsSuite is executed in parallel, the nested suites will
be executed concurrently.
NullPointerException if suitesToNest, or any suite
it contains, is null.
Class whose subclasses can be used to tag tests in style traits in which tests are defined as functions.
Class whose subclasses can be used to tag tests in style traits in which tests are defined as functions.
ScalaTest has two ways to tag tests: annotations and instances of this Tag class.
To tag a test method or an entire test class, you use a tag annotation, whereas to tag a test function,
you use a Tag object. Though not required, it is usually a good idea to define both an annotation
and a corresponding Tag object for each conceptual tag you want, so you can tag anything: test functions, test classes,
and test methods. The name of the conceptual tag is the fully qualified name of the annotation interface, so you must
pass this name to the Tag constructor.
For example, imagine you want to tag integration tests that use the actual database, and are, therefore, generally slower. You could
create a tag annotation and object called DbTest. To give them both the same simple name, you can declare them in different packages.
The tag annotation must be written in Java, not Scala, because annotations written
in Scala are not accessible at runtime. Here's an example:
package com.mycompany.myproject.testing.tags;
import java.lang.annotation.*;
import org.scalatest.TagAnnotation
@TagAnnotation
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD, ElementType.TYPE})
public @interface DbTest {}
Given this annotation's fully qualified name is com.mycompany.myproject.testing.tags.DbTest the corresponding Tag
object decaration must have that name passed to its constructor, like this:
package com.mycompany.myproject.testing.tagobjects
object DbTest extends Tag("com.mycompany.myproject.testing.tags.DbTest")
Given these definitions, you could tag a test function as a DbTest in, for
example, a FlatSpec like this:
import org.scalatest.FlatSpec
import com.mycompany.myproject.testing.tagobjects.DbTest
class ExampleSpec extends FlatSpec {
"Integration tests" can "sometimes be slow" taggedAs(DbTest) in {
Thread.sleep(1000)
}
}
You could tag a test method as a DbTest in, for
example, a Suite like this:
import org.scalatest.Suite
import com.mycompany.myproject.testing.tags.DbTest
class ExampleSuite extends Suite {
@DbTest
def `integration tests can sometimes be slow` {
Thread.sleep(1000)
}
}
And you could tag all the tests in an entire test class by annotating the class, like this:
import org.scalatest.FlatSpec
import com.mycompany.myproject.testing.tags.DbTest
@DBTest
class ExampleSpec extends FlatSpec {
"Integration tests" can "sometimes be slow" in {
Thread.sleep(1000)
}
they should "likely sometimes be excluded " in {
Thread.sleep(1000)
}
}
In the previous example, both tests will be tagged as DBTests even though the
tests are not tagged as such individually.
When you run ScalaTest and want to either include or exclude DbTests, you'd give the fully qualified
name of the tag annotation (which is also the name passed to the corresponding Tag constructor) to Runner. For
example, here's how you'd exclude DbTests on the Runner command line:
-l com.mycompany.myproject.testing.tags.DbTest
For examples of tagging in other style traits, see the "Tagging tests" section in the documentation for the trait:
A bundle of information about the current test.
A bundle of information about the current test.
A TestData object is passed to the withFixture methods of traits Suite and fixture.Suite
(both NoArgTest and OneArgTest
extend TestData) and to the beforeEach and afterEach
methods of trait BeforeAndAfterEach. This enables fixtures and tests to make use
of the test name and configuration objects in the config map.
In ScalaTest's event model, a test may be surrounded by “scopes.” Each test and scope is associated with string of text. A test's name is a concatenation of the text of any surrounding scopes followed by the text provided with the test itself, after each text element has been trimmed and one space inserted between each component. Here's an example:
package org.scalatest.examples.freespec
import org.scalatest.FreeSpec
class SetSpec extends FreeSpec {
"A Set" - {
"when empty" - {
"should have size 0" in {
assert(Set.empty.size === 0)
}
"should produce NoSuchElementException when head is invoked" in {
assertThrows[NoSuchElementException] {
Set.empty.head
}
}
}
}
}
The above FreeSpec contains two tests, both nested inside the same two scopes. The outermost scope names
the subject, A Set. The nested scope qualifies the subject with when empty. Inside that
scope are the two tests. The text of the tests are:
should have size 0should produce NoSuchElementException when head is invokedTherefore, the names of these two tests are:
A Stack when empty should have size 0A Stack when empty should produce NoSuchElementException when head is invokedThe TestData instance for the first test would contain:
name: "A Stack when empty should have size 0"scopes: collection.immutable.IndexedSeq("A Stack", "when empty")text: "should have size 0"
Trait declaring methods that can be used to register by-name test functions that have any result type.
The base trait of ScalaTest's synchronous testing styles, which defines a
withFixture lifecycle method that accepts as its parameter a test function
that returns an Outcome.
The base trait of ScalaTest's synchronous testing styles, which defines a
withFixture lifecycle method that accepts as its parameter a test function
that returns an Outcome.
The withFixture method add by this trait has the
following signature and implementation:
def withFixture(test: NoArgTest): Outcome = {
test()
}
The apply method of test function interface,
NoArgTest, also returns Outcome:
// In trait NoArgTest: def apply(): Outcome
Because the result of a test is an Outcome, when the test function returns, the test body must have determined an outcome already. It
will already be one of Succeeded, Failed, Canceled, or Pending. This is
also true when withFixture(NoArgTest) returns: because the result type of withFixture(NoArgTest) is Outcome,
the test has by definition already finished execution.
The recommended way to ensure cleanup is performed after a test body finishes execution is
to use a try-finally clause.
Using try-finally will ensure that cleanup will occur whether
the test function completes abruptly by throwing a suite-aborting exception, or returns
normally yielding an Outcome. Note that the only situation in which a test function
will complete abruptly with an exception is if the test body throws a suite-aborting exception.
Any other exception will be caught and reported as either a Failed, Canceled,
or Pending.
The withFixture method is designed to be stacked, and to enable this, you should always call the super implementation
of withFixture, and let it invoke the test function rather than invoking the test function directly. In other words, instead of writing
“test()”, you should write “super.withFixture(test)”. Thus, the recommended
structure of a withFixture implementation that performs cleanup looks like this:
// Your implementation
override def withFixture(test: NoArgTest) = {
// Perform setup here
try {
super.withFixture(test) // Invoke the test function
} finally {
// Perform cleanup here
}
}
If you have no cleanup to perform, you can write withFixture like this instead:
// Your implementation
override def withFixture(test: NoArgTest) = {
// Perform setup here
super.withFixture(test) // Invoke the test function
}
If you want to perform an action only for certain outcomes, you can use
a pattern match.
For example, if you want to perform an action if a test fails, you'd
match on Failed, like this:
// Your implementation
override def withFixture(test: NoArgTest) = {
// Perform setup here
val outcome = super.withFixture(test) // Invoke the test function
outcome match {
case failed: Failed =>
// perform action that you want to occur
// only if a test fails here
failed
case other => other
}
}
If you want to change the outcome in some way in withFixture, you can also
use a pattern match.
For example, if a particular exception intermittently causes a test to fail, and can
transform those failures into cancelations, like this:
// Your implementation
override def withFixture(test: NoArgTest) = {
super.withFixture(test) match {
case Failed(ex: ParticularException) =>
Canceled("Muting flicker", ex)
case other => other
}
}
Class that tracks the progress of a series of Ordinals produced by invoking
next and nextNewOldPair on the current Ordinal.
Class that tracks the progress of a series of Ordinals produced by invoking
next and nextNewOldPair on the current Ordinal.
Instances of this class are thread safe. Multiple threads can invoke nextOrdinal
and nextTracker concurrently. This facilitates multi-threaded tests that send
infoProvided reports concurrently. When using a Dispatcher to execute
suites in parallel, the intention is that each Tracker will only be used by one
thread. For example, if the optional Dispatcher passed to Suite's implementation
of runNestedSuites is defined, that method will obtain a new Tracker by invoking
nextTracker for each nested suite it passes to the Dispatcher.
Trait that provides an implicit conversion that adds success and failure methods
to scala.util.Try, enabling you to make assertions about the value of a Success or
the exception of a Failure.
Trait that provides an implicit conversion that adds success and failure methods
to scala.util.Try, enabling you to make assertions about the value of a Success or
the exception of a Failure.
The success method will return the Try on which it is invoked as a Success if the Try
actually is a Success, or throw TestFailedException if not.
The failure method will return the Try on which it is invoked as a Failure if the Try
actually is a Failure, or throw TestFailedException if not.
This construct allows you to express in one statement that an Try should be either a Success
or a Failure and that its value or exception, respectively,should meet some expectation. Here's an example:
try1.success.value should be > 9 try2.failure.exception should have message "/ by zero"
Or, using assertions instead of a matchers:
assert(try1.success.value > 9) assert(try2.failure.exception.getMessage == "/ by zero")
Were you to simply invoke get on the Try,
if the Try wasn't a Success, it would throw the exception contained in the Failure:
val try2 = Try { 1 / 0 }
try2.get should be < 9 // try2.get throws ArithmeticException
The ArithmeticException would cause the test to fail, but without providing a stack depth pointing
to the failing line of test code. This stack depth, provided by TestFailedException (and a
few other ScalaTest exceptions), makes it quicker for
users to navigate to the cause of the failure. Without TryValues, to get
a stack depth exception you would need to make two statements, like this:
try2 should be a 'success // throws TestFailedException try2.get should be < 9
The TryValues trait allows you to state that more concisely:
try2.success.value should be < 9 // throws TestFailedException
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.featurespec.AsyncFeatureSpec instead.
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.featurespec.AsyncFeatureSpec instead.
Please use org.scalatest.featurespec.AsyncFeatureSpec instead
Please use org.scalatest.featurespec.AsyncFeatureSpecLike instead
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.flatspec.AsyncFlatSpec instead.
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.flatspec.AsyncFlatSpec instead.
Please use org.scalatest.flatspec.AsyncFlatSpec instead
Please use org.scalatest.flatspec.AsyncFlatSpecLike instead
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.freespec.AsyncFreeSpec instead.
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.freespec.AsyncFreeSpec instead.
Please use org.scalatest.freespec.AsyncFreeSpec instead
Please use org.scalatest.freespec.AsyncFreeSpecLike instead
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.funspec.AsyncFunSpec instead.
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.funspec.AsyncFunSpec instead.
Please use org.scalatest.funspec.AsyncFunSpec instead
Please use org.scalatest.funspec.AsyncFunSpecLike instead
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.funsuite.AsyncFunSuite instead.
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.funsuite.AsyncFunSuite instead.
Please use org.scalatest.funsuite.AsyncFunSuite instead
Please use org.scalatest.funsuite.AsyncFunSuiteLike instead
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.wordspec.AsyncWordSpec instead.
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.wordspec.AsyncWordSpec instead.
Please use org.scalatest.wordspec.AsyncWordSpec instead
Please use org.scalatest.wordspec.AsyncWordSpecLike instead
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.featurespec.AnyFeatureSpec instead.
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.featurespec.AnyFeatureSpec instead.
Please use org.scalatest.featurespec.AnyFeatureSpec instead
Please use org.scalatest.funsuite.AnyFeatureSpecLike instead
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.flatspec.AnyFlatSpec instead.
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.flatspec.AnyFlatSpec instead.
Please use org.scalatest.flatspec.AnyFlatSpec instead
Please use org.scalatest.flatspec.AnyFlatSpecLike instead
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.funspec.AnyFunSpec instead.
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.funspec.AnyFunSpec instead.
Please use org.scalatest.freespec.AnyFreeSpec instead
Please use org.scalatest.freespec.AnyFreeSpecLike instead
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.funspec.AnyFunSpec instead.
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.funspec.AnyFunSpec instead.
Please use org.scalatest.funspec.AnyFunSpec instead
Please use org.scalatest.funsuite.AnyFunSpecLike instead
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.funsuite.AnyFunSuite instead.
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.funsuite.AnyFunSuite instead.
Please use org.scalatest.funsuite.AnyFunSuite instead
Please use org.scalatest.funsuite.AnyFunSuiteLike instead
Please use PendingStatement instead
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.funspec.AnyFunSpec instead.
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.funspec.AnyFunSpec instead.
Please use org.scalatest.propspec.AnyPropSpec instead
Please use org.scalatest.propspec.AnyPropSpecLike instead
Trait whose instances can rerun tests or other entities (such as suites).
Trait whose instances can rerun tests or other entities (such as suites). An object extending
this trait can be passed to a Reporter as part of a Report. The
test or other entity about which the report is made can then be rerun by invoking the
rerun method on the Rerunnable.
We are considering removing Rerunner in ScalaTest 2.0 and would like to know if anyone is using it. If you are, please email [email protected] or and describe your use case. Thanks!
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.funspec.AnyFunSpec instead.
This class is deprecated and will be removed in future version of ScalaTest, please use org.scalatest.funspec.AnyFunSpec instead.
Please use org.scalatest.wordspec.AnyWordSpec instead
Please use org.scalatest.wordspec.AnyWordSpecLike instead
Companion object that facilitates the importing of AppendedClues members as
an alternative to mixing it in.
Companion object that facilitates the importing of AppendedClues members as
an alternative to mixing it in. One use case is to import AppendedClues
members so you can use them in the Scala interpreter.
Companion object that facilitates the importing of Assertions members as
an alternative to mixing it in.
Companion object that facilitates the importing of Assertions members as
an alternative to mixing it in. One use case is to import Assertions members so you can use
them in the Scala interpreter:
$scala -classpath scalatest.jar
Welcome to Scala version 2.7.3.final (Java HotSpot(TM) Client VM, Java 1.5.0_16).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import org.scalatest.Assertions._
import org.scalatest.Assertions._
scala> assert(1 === 2)
org.scalatest.TestFailedException: 1 did not equal 2
at org.scalatest.Assertions$class.assert(Assertions.scala:211)
at org.scalatest.Assertions$.assert(Assertions.scala:511)
at .<init>(<console>:7)
at .<clinit>(<console>)
at RequestResult$.<init>(<console>:3)
at RequestResult$.<clinit>(<console>)
at RequestResult$result(<console>)
at sun.reflect.NativeMethodAccessorImpl.invoke...
scala> assertResult(3) { 1 + 3 }
org.scalatest.TestFailedException: Expected 3, but got 4
at org.scalatest.Assertions$class.expect(Assertions.scala:447)
at org.scalatest.Assertions$.expect(Assertions.scala:511)
at .<init>(<console>:7)
at .<clinit>(<console>)
at RequestResult$.<init>(<console>:3)
at RequestResult$.<clinit>(<console>)
at RequestResult$result(<console>)
at sun.reflect.NativeMethodAccessorImpl.in...
scala> val caught = intercept[StringIndexOutOfBoundsException] { "hi".charAt(-1) }
caught: StringIndexOutOfBoundsException = java.lang.StringIndexOutOfBoundsException: String index out of range: -1
Companion object to class Canceled that provides, in addition to the extractor and factory method
provided by the compiler given its companion is a case class, a second factory method
that produces a Canceled outcome given a string message.
Companion object to class Canceled that provides, in addition to the extractor and factory method
provided by the compiler given its companion is a case class, a second factory method
that produces a Canceled outcome given a string message.
Companion object that facilitates the importing the members of trait Checkpoints as
an alternative to mixing it in.
Companion object that facilitates the importing the members of trait Checkpoints as
an alternative to mixing it in. One use case is to import Checkpoints so you can use
it in the Scala interpreter.
Companion object that facilitates the importing of CompleteLastly members as
an alternative to mixing it in.
Companion object that facilitates the importing of CompleteLastly members as
an alternative to mixing it in.
Companion object to class ConfigMap containing factory methods.
Companion object to class ConfigMap containing factory methods.
Companion object that facilitates the importing of DiagrammedAssertions members as
an alternative to mixing it in.
Companion object that facilitates the importing of DiagrammedAssertions members as
an alternative to mixing it in. One use case is to import DiagrammedAssertions members so you can use
them in the Scala interpreter:
$scala -classpath scalatest.jar
Welcome to Scala version 2.10.4.final (Java HotSpot(TM) Client VM, Java 1.6.0_45).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import org.scalatest.Assertions._
import org.scalatest.Assertions._
scala> assert(1 === 2)
org.scalatest.exceptions.TestFailedException:
assert(1 === 2)
| | |
1 | 2
false
at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
at org.scalatest.DiagrammedAssertions$.newAssertionFailedException(DiagrammedAssertions.scala:249)
at org.scalatest.DiagrammedAssertions$DiagrammedAssertionsHelper.macroAssert(DiagrammedAssertions.scala:111)
at .<init>(<console>:20)
at .<clinit>(<console>)
at .<init>(<console>:7)
at .<clinit>(<console>)
at $print(<console>)
at sun.reflect.NativeMethodAccessorImpl.invoke...
DiagrammedExpr companion object that provides factory methods to create different sub types of DiagrammedExpr
DiagrammedExpr companion object that provides factory methods to create different sub types of DiagrammedExpr
DiagrammedExpr is used by code generated from DiagrammedAssertionsMacro, it needs to be public
so that the generated code can be compiled. It is expected that ScalaTest users would ever need to use DiagrammedExpr
directly.
Companion object that facilitates the importing of ValueEither members as
an alternative to mixing it in.
Companion object that facilitates the importing of ValueEither members as
an alternative to mixing it in. One use case is to import EitherValues's members so you can use
left.value and right.value on Either in the Scala interpreter:
$ scala -cp scalatest-1.7.jar
Welcome to Scala version 2.9.1.final (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_29).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import org.scalatest._
import org.scalatest._
scala> import matchers.Matchers._
import matchers.Matchers._
scala> import EitherValues._
import EitherValues._
scala> val e: Either[String, Int] = Left("Muchas problemas")
e: Either[String,Int] = Left(Muchas problemas)
scala> e.left.value should be ("Muchas problemas")
scala> e.right.value should be < 9
org.scalatest.TestFailedException: The Either on which rightValue was invoked was not defined.
at org.scalatest.EitherValues$RightValuable.value(EitherValues.scala:148)
at .<init>(<console>:18)
...
Companion object to class Exceptional that provides a factory method and an extractor that enables
patterns that match both Failed and Canceled outcomes and
extracts the contained exception and a factory method.
Companion object to class Exceptional that provides a factory method and an extractor that enables
patterns that match both Failed and Canceled outcomes and
extracts the contained exception and a factory method.
Singleton status that represents an already completed run with at least one failed test or aborted suite.
Singleton status that represents an already completed run with at least one failed test or aborted suite.
Note: the difference between this FailedStatus object and the similarly named Failed
class is that a Failed instance indicates one test failed, whereas this FailedStatus object indicates either one or more tests failed
and/or one or more suites aborted during a run. Both are used as the result type of Suite lifecycle methods, but Failed
is a possible result of withFixture, whereas FailedStatus is a possible result of run, runNestedSuites,
runTests, or runTest. In short, Failed is always just about one test, whereas FailedStatus could be
about something larger: multiple tests or an entire suite.
Companion object to FutureOutcomes that contains factory methods for creating already-completed
FutureOutcomes.
Companion object to FutureOutcomes that contains factory methods for creating already-completed
FutureOutcomes.
Companion object that facilitates the importing of the inside construct as
an alternative to mixing it in.
Companion object that facilitates the importing of the inside construct as
an alternative to mixing it in. One use case is to import the inside construct so you can use
it in the Scala interpreter:
$ scala -cp scalatest-1.8.jar
Welcome to Scala version 2.9.1.final (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_29).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import org.scalatest._
import org.scalatest._
scala> import matchers.Matchers._
import matchers.Matchers._
scala> import Inside._
import Inside._
scala> inside (List(1, 2, 3)) { case List(x, y, z) =>
| y should equal (2)
| }
scala> inside (List(1, 2, 3)) { case List(x, y, z) =>
| x should equal (2)
| }
org.scalatest.TestFailedException: 1 did not equal 2, inside List(1, 2, 3)
at org.scalatest.matchers.Matchers$class.newTestFailedException(Matchers.scala:150)
at org.scalatest.matchers.Matchers$.newTestFailedException(Matchers.scala:2331)
at org.scalatest.matchers.Matchers$ShouldMethodHelper$.shouldMatcher(Matchers.scala:873)
...
Companion object that facilitates the importing of Inspectors members as
an alternative to mixing it in.
Companion object that facilitates the importing of Inspectors members as
an alternative to mixing it in. One use case is to import Inspectors's members so you can use
them in the Scala interpreter.
Companion object that facilitates the importing of LoneElement members as
an alternative to mixing it in.
Companion object that facilitates the importing of LoneElement members as
an alternative to mixing it in. One use case is to import LoneElement's members so you can use
loneElement in the Scala interpreter.
Companion object that facilitates the importing of Matchers members as
an alternative to mixing it the trait.
Companion object that facilitates the importing of Matchers members as
an alternative to mixing it the trait. One use case is to import Matchers members so you can use
them in the Scala interpreter.
Companion object that facilitates the importing of Matchers members as
an alternative to mixing it the trait.
Companion object that facilitates the importing of Matchers members as
an alternative to mixing it the trait. One use case is to import Matchers members so you can use
them in the Scala interpreter.
Companion object that facilitates the importing of the members of trait Assertions without importing the implicit conversions
it provides by default.
Companion object that facilitates the importing of the members of trait Assertions without importing the implicit conversions
it provides by default. One use case for this object is to import the non-implicit Assertions members so you can use
them in the Scala interpreter along with another library whose implicits conflict with those provided by Assertions:
$ scala -cp scalatest-1.7.jar
Welcome to Scala version 2.9.1.final (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_29).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import org.scalatest._
import org.scalatest._
scala> import NonImplicitAssertions._
import NonImplicitAssertions._
scala> assert(1 + 1 === 2)
<console>:14: error: value === is not a member of Int
assert(1 + 1 === 2)
^
scala> assert(1 + 1 == 2)
scala> expect(2) { 1 + 1 }
scala> expect(2) { 1 + 1 + 1 }
org.scalatest.TestFailedException: Expected 2, but got 3
at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:318)
at org.scalatest.NonImplicitAssertions$.newAssertionFailedException(NonImplicitAssertions.scala:73)
...
scala> intercept[IndexOutOfBoundsException] { "hi".charAt(-1) }
res3: IndexOutOfBoundsException = java.lang.StringIndexOutOfBoundsException: String index out of range: -1
Companion object that facilitates the importing of OptionValues members as
an alternative to mixing it in.
Companion object that facilitates the importing of OptionValues members as
an alternative to mixing it in. One use case is to import OptionValues's members so you can use
value on option in the Scala interpreter:
$ scala -cp scalatest-1.7.jar Welcome to Scala version 2.9.1.final (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_29). Type in expressions to have them evaluated. Type :help for more information. scala> import org.scalatest._ import org.scalatest._ scala> import matchers.Matchers._ import matchers.Matchers._ scala> import OptionValues._ import OptionValues._ scala> val opt1: Option[Int] = Some(1) opt1: Option[Int] = Some(1) scala> val opt2: Option[Int] = None opt2: Option[Int] = None scala> opt1.value should be < 10 scala> opt2.value should be < 10 org.scalatest.TestFailedException: The Option on which value was invoked was not defined. at org.scalatest.OptionValues$Valuable.value(OptionValues.scala:68) at .<init>(<console>:18) ...
Companion object for trait Outcome that contains an implicit method that enables
collections of Outcomes to be flattened into a collections of contained exceptions.
Companion object for trait Outcome that contains an implicit method that enables
collections of Outcomes to be flattened into a collections of contained exceptions.
Companion object that facilitates the importing of OutcomeOf's method as
an alternative to mixing it in.
Companion object that facilitates the importing of OutcomeOf's method as
an alternative to mixing it in. One use case is to import OutcomeOf's method so you can use
it in the Scala interpreter.
Companion object that facilitates the importing of PartialFunctionValues members as
an alternative to mixing it in.
Companion object that facilitates the importing of PartialFunctionValues members as
an alternative to mixing it in. One use case is to import PartialFunctionValues's members so you can use
the valueAt method on PartialFunction in the Scala interpreter:
$ scala -cp scalatest-1.7.jar
Welcome to Scala version 2.9.1.final (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_29).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import org.scalatest._
import org.scalatest._
scala> import matchers.Matchers._
import matchers.Matchers._
scala> import PartialFunctionValues._
import PartialFunctionValues._
scala> val pf: PartialFunction[String, Int] = Map("I" -> 1, "II" -> 2, "III" -> 3, "IV" -> 4)
pf: PartialFunction[String,Int] = Map(I -> 1, II -> 2, III -> 3, IV -> 4)
scala> pf("IV") should equal (4)
scala> pf("V") should equal (5)
java.util.NoSuchElementException: key not found: V
at scala.collection.MapLike$class.default(MapLike.scala:224)
at scala.collection.immutable.Map$Map4.default(Map.scala:167)
...
Companion object that facilitates the importing of Payloads members as
an alternative to mixing it in.
Companion object that facilitates the importing of Payloads members as
an alternative to mixing it in. One use case is to import Payloads
members so you can use them in the Scala interpreter.
Outcome for a test that was pending, which contains an optional string giving more information on what exactly is needed for the test to become non-pending.
Companion object that facilitates the importing of PrivateMethodTester members as
an alternative to mixing it in.
Companion object that facilitates the importing of PrivateMethodTester members as
an alternative to mixing it in. One use case is to import PrivateMethodTester members so you can use
them in the Scala interpreter:
$scala -classpath scalatest.jar
Welcome to Scala version 2.7.5.final (Java HotSpot(TM) Client VM, Java 1.5.0_16).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import org.scalatest.PrivateMethodTester._
import org.scalatest.PrivateMethodTester._
scala> class Example {
| private def addSesame(prefix: String) = prefix + " sesame"
| }
defined class Example
scala> val example = new Example
example: Example = Example@d8b6fe
scala> val addSesame = PrivateMethod[String]('addSesame)
addSesame: org.scalatest.PrivateMethodTester.PrivateMethod[String] = org.scalatest.PrivateMethodTester$PrivateMethod@5cdf95
scala> example invokePrivate addSesame("open")
res0: String = open sesame
@author Bill Venners
Companion object that facilitates the importing of RecoverMethods's method as
an alternative to mixing it in.
Companion object that facilitates the importing of RecoverMethods's method as
an alternative to mixing it in. One use case is to import RecoverMethods's method so you can use
it in the Scala interpreter.
Companion object to trait Retries that enables its members to be imported as an
alternative to mixing them in.
Companion object to trait Retries that enables its members to be imported as an
alternative to mixing them in.
The version number of ScalaTest.
The version number of ScalaTest.
the ScalaTest version number.
Companion object to class Sequential that offers an apply factory method
for creating a Sequential instance.
Companion object to class Sequential that offers an apply factory method
for creating a Sequential instance.
One use case for this object is to run multiple specification-style suites in the Scala interpreter, like this:
scala> Sequential(new MyFirstSuite, new MyNextSuite).execute()
Companion object to class Stepwise that offers an apply factory method
for creating a Stepwise instance.
Companion object to class Stepwise that offers an apply factory method
for creating a Stepwise instance.
One use case for this object is to run multiple specification-style suites in the Scala interpreter, like this:
scala> Stepwise(new MyFirstSuite, new MyNextSuite).execute()
Companion object to Stopper that holds a factory method that produces a new Stopper whose
stopRequested method returns false until after its requestStop has been
invoked.
Companion object to Stopper that holds a factory method that produces a new Stopper whose
stopRequested method returns false until after its requestStop has been
invoked.
Outcome for a test that succeeded.
Outcome for a test that succeeded.
Note: the difference between this Succeeded object and the similarly named SucceededStatus
object is that this object indicates one test (or assertion) succeeded, whereas the SucceededStatus object indicates the absence of any failed tests or
aborted suites during a run. Both are used as the result type of Suite lifecycle methods, but Succeeded
is a possible result of withFixture, whereas SucceededStatus is a possible result of run, runNestedSuites,
runTests, or runTest. In short, Succeeded is always just about one test (or assertion), whereas SucceededStatus could be
about something larger: multiple tests or an entire suite.
Singleton status that represents an already completed run with no tests failed and no suites aborted.
Singleton status that represents an already completed run with no tests failed and no suites aborted.
Note: the difference between this SucceededStatus object and the similarly named Succeeded
object is that the Succeeded object indicates one test succeeded, whereas this SucceededStatus object indicates the absence
of any failed tests or aborted suites during a run. Both are used as the result type of Suite lifecycle methods, but Succeeded
is a possible result of withFixture, whereas SucceededStatus is a possible result of run, runNestedSuites,
runTests, or runTest. In short, Succeeded is always just about one test, whereas SucceededStatus could be
about something larger: multiple tests or an entire suite.
Companion object to class Suites that offers an apply factory method
for creating a Suites instance.
Companion object to class Suites that offers an apply factory method
for creating a Suites instance.
One use case for this object is to run multiple specification-style suites in the Scala interpreter, like this:
scala> Suites(new MyFirstSuite, new MyNextSuite).execute()
Companion object for Tag, which offers a factory method.
Companion object for Tag, which offers a factory method.
Companion object that facilitates the importing of TryValues members as
an alternative to mixing it in.
Companion object that facilitates the importing of TryValues members as
an alternative to mixing it in. One use case is to import TryValues's members so you can use
success and failure on Try in the Scala interpreter.
ScalaTest's main traits, classes, and other members, including members supporting ScalaTest's DSL for the Scala interpreter.
Package fixture deprecated types.
Scalatest support for Property-based testing.
Scalatest support for Property-based testing.
In traditional unit testing, you write tests that describe precisely what the test will do: create these objects, wire them together, call these functions, assert on the results, and so on. It is clear and deterministic, but also limited, because it only covers the exact situations you think to test. In most cases, it is not feasible to test all of the possible combinations of data that might arise in real-world use.
Property-based testing works the other way around. You describe properties -- rules that you expect your classes to live by -- and describe how to test those properties. The test system then generates relatively large amounts of synthetic data (with an emphasis on edge cases that tend to make things break), so that you can see if the properties hold true in these situations.
As a result, property-based testing is scientific in the purest sense: you are stating a hypothesis about how things should work (the property), and the system is trying to falsify that hypothesis. If the tests pass, that doesn't prove the property holds, but it at least gives you some confidence that you are probably correct.
Property-based testing is deliberately a bit random: while the edge cases get tried upfront, the system also usually generates a number of random values to try out. This makes things a bit non-deterministic -- each run will be tried with somewhat different data. To make it easier to debug, and to build regression tests, the system provides tools to re-run a failed test with precisely the same data.
TODO: Bill should insert a brief section on QuickCheck, ScalaCheck, etc, and how this system is similar and different.
In order to use the tools described here, you should import this package:
import org.scalatest._ import org.scalatest.prop._
This library is designed to work well with the types defined in Scalactic, and some functions take types such as PosZInt as parameters. So it can also be helpful to import those with:
import org.scalactic.anyvals._In order to call forAll, the function that actually
performs property checks, you will need to either extend or import GeneratorDrivenPropertyChecks,
like this:
class DocExamples extends FlatSpec with Matchers with GeneratorDrivenPropertyChecks {
There's nothing special about FlatSpec, though -- you may use any of ScalaTest's styles with property checks. GeneratorDrivenPropertyChecks extends CommonGenerators, so it also provides access to the many utilities found there.
Let's check a simple property of Strings -- that if you concatenate a String to itself, its length will be doubled:
"Strings" should "have the correct length when doubled" in { forAll { (s: String) => val s2 = s * 2 s2.length should equal (s.length * 2) } }
(Note that the examples here are all using the FlatSpec style, but will work the same way with any of ScalaTest's styles.)
As the name of the tests suggests, the property we are testing is the length of a String that has been doubled.
The test begins with forAll. This is usually the way you'll want to begin property checks, and
that line can be read as, "For all Strings, the following should be true".
The test harness will generate a number of Strings, with various contents and lengths. For each one,
we compute s * 2. (* is a function on String, which appends the String to itself as many times
as you specify.) And then we
check that the length of the doubled String is twice the length of the original one.
Let's try a more general version of this test, multiplying arbitrary Strings by arbitrary multipliers:
"Strings" should "have the correct length when multiplied" in { forAll { (s: String, n: PosZInt) => val s2 = s * n.value s2.length should equal (s.length * n.value) } }
Again, you can read the first line of the test as "For all Strings, and all non-negative Integers, the following should be true". (PosZInt is a type defined in Scalactic, which can be any positive integer, including zero. It is appropriate to use here, since multiplying a String by a negative number doesn't make sense.)
This intuitively makes sense, but when we try to run it, we get a JVM Out of Memory error! Why? Because the test system tries to test with the "edge cases" first, and one of the more important edge cases is Int.MaxValue. It is trying to multiply a String by that, which is far larger than the memory of even a big computer, and crashing.
So we want to constrain our test to sane values of n, so that it doesn't crash. We can do this by
using more specific Generators.
When we write a forAll test like the above, ScalaTest has to generate the values to be tested -- the
semi-random Strings, Ints and other types that you are testing. It does this by calling on an implicit
Generator for the desired type. The Generator generates values to test, starting with the edge cases
and then moving on to randomly-selected values.
ScalaTest has built-in Generators for many major types, including String and PosZInt, but these Generators are generic: they will try any value, including values that can break your test, as shown above. But it also provides tools to let you be more specific.
Here is the fixed version of the above test:
"Strings" should "have the correct length when multiplied" in { forAll(strings, posZIntsBetween(0, 1000)) { (s: String, n: PosZInt) => val s2 = s * n.value s2.length should equal (s.length * n.value) } }
This is using a variant of forAll, which lets you specify the Generators to use instead of just picking the implicit one. CommonGenerators.strings is the built-in Generator for Strings, the same one you were getting implicitly. (The other built-ins can be found in CommonGenerators. They are mixed into GeneratorDrivenPropertyChecks, so they are readily available.)
But CommonGenerators.posZIntsBetween is a function that creates a Generator that selects from the given values. In this case, it will create a Generator that only creates numbers from 0 to 1000 -- small enough to not blow up our computer's memory. If you try this test, this runs correctly.
The moral of the story is that, while using the built-in Generators is very convenient, and works most of the time, you should think about the data you are trying to test, and pick or create a more-specific Generator when the test calls for it.
CommonGenerators contains many functions that are helpful in common cases. In particular:
xxsBetween (where xxs might be Int, Long, Float or most other significant numeric types)
gives you a value of the desired type in the given range, as in the posZIntsBetween() example
above.Testing the built-in types isn't very interesting, though. Usually, you have your own types that you want to check the properties of. So let's build up an example piece by piece.
Say you have this simple type:
sealed trait Shape { def area: Double } case class Rectangle(width: Int, height: Int) extends Shape { require(width > 0) require(height > 0) def area: Double = width * height }
Let's confirm a nice straightforward property that is surely true: that the area is greater than zero:
"Rectangles" should "have a positive area" in { forAll { (w: PosInt, h: PosInt) => val rect = Rectangle(w, h) rect.area should be > 0.0 } }
Note that, even though our class takes ordinary Ints as parameters (and checks the values at runtime), it is actually easier to generate the legal values using Scalactic's PosInt type.
This should work, right? Actually, it doesn't -- if we run it a few times, we quickly hit an error!
[info] Rectangles [info] - should have a positive area *** FAILED *** [info] GeneratorDrivenPropertyCheckFailedException was thrown during property evaluation. [info] (DocExamples.scala:42) [info] Falsified after 2 successful property evaluations. [info] Location: (DocExamples.scala:42) [info] Occurred when passed generated values ( [info] None = PosInt(399455539), [info] None = PosInt(703518968) [info] ) [info] Init Seed: 1568878346200
TODO: fix the above error to reflect the better errors we should get when we merge in the code being forward-ported from 3.0.5.
Looking at it, we can see that the numbers being used are pretty large. What happens when we multiply them together?
scala> 399455539 * 703518968 res0: Int = -2046258840
We're hitting an Int overflow problem here: the numbers are too big to multiply together and
still get an Int. So we have to fix our area function:
case class Rectangle(width: Int, height: Int) extends Shape { require(width > 0) require(height > 0) def area: Double = width.toLong * height.toLong }
Now, when we run our property check, it consistently passes. Excellent -- we've caught a bug, because ScalaTest tried sufficiently large numbers.
Doing things as shown above works, but having to generate the parameters and construct a
Rectangle every time is a nuisance. What we really want is to create our own Generator
that just hands us Rectangles, the same way we can do for PosInt. Fortunately, this
is easy.
Generators can be composed in for comprehensions. So we can create our own Generator
for Rectangle like this:
implicit val rectGenerator = for { w <- posInts h <- posInts } yield Rectangle(w, h)
Taking that line by line:
w <- posIntsCommonGenerators.posInts is the built-in Generator for positive Ints. So this line puts
a randomly-generated positive Int in w, and
h <- posIntsthis line puts another one in h. Finally, this line:
yield Rectangle(w, h)combines w and h to make a Rectangle.
That's pretty much all you need in order to build any normal case class -- just build it
out of the Generators for the type of each field. (And if the fields are complex data
structures themselves, build Generators for them the same way, until you are just using
primitives.)
Now, our property check becomes simpler:
"Generated Rectangles" should "have a positive area" in { forAll { (rect: Rectangle) => rect.area should be > 0.0 } }
That's about as close to plain English as we can reasonably hope for!
Sometimes, not all of your generated values make sense for the property you want to check -- you know (via external information) that some of these values will never come up. In cases like this, you can create a custom Generator that only creates the values you do want, but it's often easier to just use Whenever.whenever. (Whenever is mixed into GeneratorDrivenPropertyChecks, so this is available when you need it.)
The Whenever.whenever function can be used inside of GeneratorDrivenPropertyChecks.forAll. It says that only the filtered values should be used, and anything else should be discarded. For example, look at this property:
"Fractions" should "get smaller when squared" in { forAll { (n: Float) => whenever(n > 0 && n < 1) { (n * n) should be < n } } }
We are testing a property of numbers less than 1, so we filter away everything that is not the numbers we want. This property check succeeds, because we've screened out the values that would make it fail.
You shouldn't push Whenever.whenever too far, though. This system is all about trying random data, but if too much of the random data simply isn't usable, you can't get valid answers, and the system tracks that.
For example, consider this apparently-reasonable test:
"Space Chars" should "not also be letters" in { forAll { (c: Char) => whenever (c.isSpaceChar) { assert(!c.isLetter) } } }
Although the property is true, this test will fail with an error like this:
[info] Lowercase Chars [info] - should upper-case correctly *** FAILED *** [info] Gave up after 0 successful property evaluations. 49 evaluations were discarded. [info] Init Seed: 1568855247784
Because the vast majority of Chars are not spaces, nearly all of the generated values are being discarded. As a result, the system gives up after a while. In cases like this, you usually should write a custom Generator instead.
The proportion of how many discards to permit, relative to the number of successful checks, is configuration-controllable. See GeneratorDrivenPropertyChecks for more details.
The point of Generator is to create pseudo-random values for checking properties. But it turns out to be very inconvenient if those values are actually random -- that would mean that, when a property check fails occasionally, you have no good way to invoke that specific set of circumstances again for debugging. We want "randomness", but we also want it to be deterministic, and reproducible when you need it.
To support this, all "randomness" in ScalaTest's property checking system uses the Randomizer class. You start by creating a Randomizer using an initial seed value, and call that to get your "random" value. Each call to a Randomizer function returns a new Randomizer, which you should use to fetch the next value.
GeneratorDrivenPropertyChecks.forAll uses Randomizer under the hood: each time
you run a forAll-based test, it will automatically create a new Randomizer,
which by default is seeded based on the current system time. You can override this,
as discussed below.
Since Randomizer is actually deterministic (the "random" values are unobvious, but will always be the same given the same initial seed), this means that re-running a test with the same seed will produce the same values.
If you need random data for your own Generators and property checks, you should use Randomizer in the same way; that way, your tests will also be re-runnable, when needed for debugging.
In Testing Your Own Types above, we found to our surprise that the property check failed with this error:
[info] Rectangles [info] - should have a positive area *** FAILED *** [info] GeneratorDrivenPropertyCheckFailedException was thrown during property evaluation. [info] (DocExamples.scala:42) [info] Falsified after 2 successful property evaluations. [info] Location: (DocExamples.scala:42) [info] Occurred when passed generated values ( [info] None = PosInt(399455539), [info] None = PosInt(703518968) [info] ) [info] Init Seed: 1568878346200
There must be a bug here -- but once we've fixed it, how can we make sure that we are re-testing exactly the same case that failed?
This is where the pseudo-random nature of Randomizer comes in, and why it is so important to use it consistently. So long as all of our "random" data comes from that, then all we need to do is re-run with the same seed.
That's why the Init Seed shown in the message above is crucial. We can re-use that
seed -- and therefore get exactly the same "random" data -- by using the -S flag to
ScalaTest.
So you can run this command in sbt to re-run exactly the same property check:
testOnly *DocExamples -- -z "have a positive area" -S 1568878346200
Taking that apart:
testOnly *DocExamples says that we only want to run suites whose paths end with DocExamples-z "have a positive area" says to only run tests whose names include that string.-S 1568878346200 says to run all tests with a "random" seed of 1568878346200By combining these flags, you can re-run exactly the property check you need, with the right random seed to make sure you are re-creating the failed test. You should get exactly the same failure over and over until you fix the bug, and then you can confirm your fix with confidence.
In general, forAll() works well out of the box. But you can tune several configuration parameters
when needed. See GeneratorDrivenPropertyChecks for info on how to set
configuration parameters for your test.
Sometimes, you want something in between traditional hard-coded unit tests and Generator-driven, randomized tests. Instead, you sometimes want to check your properties against a specific set of inputs.
(This is particularly useful for regression tests, when you have found certain inputs that have caused problems in the past, and want to make sure that they get consistently re-tested.)
ScalaTest supports these, by mixing in TableDrivenPropertyChecks. See the documentation for that class for the full details.
ScalaTest's main traits, classes, and other members, including members supporting ScalaTest's DSL for the Scala interpreter.