dsl

Dsl.scala also supports scalaz.MonadTrans. Considering the line counter implemented in previous example may be failed for some files, due to permission issue or other IO problem, you can use scalaz.OptionT monad transformer to mark those failed file as a None.

import scalaz._
import java.io.File
import com.thoughtworks.dsl.keywords.Monadic, Monadic.implicitMonadic
import com.thoughtworks.dsl.domains.scalaz._
import scalaz.std.stream._
def countLift(file: File): OptionT[Stream, Int] = OptionT.some {
  if (file.isDirectory) {
    file.listFiles() match {
      case null =>
        // Unable to open `file`
        !OptionT.none[Stream, Int]
      case children =>
        val child: File = !Stream(children: _*)
        !countLift(child)
    }
  } else {
    scala.io.Source.fromFile(file).getLines.size
  }
}
val countCurrentSourceFile = countLift(new File(sourcecode.File())).run
inside(countCurrentSourceFile) {
  case Stream(Some(lineCount)) =>
    lineCount should be > 0
}

Note that our keywords are adaptive to the domain it belongs to. Thus, instead of explicit !Monadic(OptionT.optionTMonadTrans.liftM(Stream(children: _*))), you can simply have !Stream(children: _*). The implicit lifting feature looks like Idris's effect monads, though the mechanisms is different from implicit lift in Idris.

The previous code requires a toStream conversion on children, because children's type Array[File] does not fit the F type parameter in scalaz.Monad.bind. The conversion can be avoided if using CanBuildFrom type class instead of monads. We provide a Each keyword to extract each element in a Scala collection. The behavior is similar to monad, except the collection type can vary. For example, you can extract each element from an Array, even when the return type (or the domain) is a Stream.

import java.io.File
import com.thoughtworks.dsl.keywords.Monadic, Monadic.implicitMonadic
import com.thoughtworks.dsl.keywords.Each
import com.thoughtworks.dsl.domains.scalaz._
import scalaz.std.stream._
def countEach(file: File): Stream[Int] = Stream {
  if (file.isDirectory) {
    file.listFiles() match {
      case null =>
        // Unable to open `file`
        !Stream.empty[Int]
      case children =>
        val child: File = !Each(children)
        !countEach(child)
    }
  } else {
    scala.io.Source.fromFile(file).getLines.size
  }
}
val countCurrentSourceFile = countEach(new File(sourcecode.File()))
inside(countCurrentSourceFile) {
  case Stream(lineCount) =>
    lineCount should be > 0
}

Unlike Haskell's do-notation or Idris's !-notation, Dsl.scala allows non-monadic keywords like Each works along with monads.

The built-in keywords.Monadic can be used as an adaptor to scalaz.Monad and scalaz.MonadTrans, to create monadic code from imperative syntax, similar to the !-notation in Idris. For example, suppose you are creating a program that counts lines of code under a directory. You want to store the result in a Stream of line count of each file.

import java.io.File
import com.thoughtworks.dsl.keywords.Monadic
import com.thoughtworks.dsl.domains.scalaz._
import scalaz.std.stream._
def countMonadic(file: File): Stream[Int] = Stream {
  if (file.isDirectory) {
    file.listFiles() match {
      case null =>
        // Unable to open `file`
        !Monadic(Stream.empty[Int])
      case children =>
        // Import this implicit conversion to omit the Monadic keyword
        import com.thoughtworks.dsl.keywords.Monadic.implicitMonadic
        val child: File = !children.toStream
        !countMonadic(child)
    }
  } else {
    scala.io.Source.fromFile(file).getLines.size
  }
}
val countCurrentSourceFile = countMonadic(new File(sourcecode.File()))
inside(countCurrentSourceFile) {
  case Stream(lineCount) =>
    lineCount should be > 0
}

If you don't need to collaborate to Stream or other domains, you can use TailRec[Unit] !! Throwable !! A or the alias domains.task.Task as the return type, which can be used as an asynchronous task that support RAII, as a higher-performance replacement of scala.concurrent.Future, scalaz.concurrent.Task or monix.eval.Task. Also, there are some keywords in keywords.AsynchronousIo to asynchronously perform Java NIO.2 IO operations in a domains.task.Task domain. For example, you can implement an HTTP client from those keywords.

import com.thoughtworks.dsl.domains.task._
import com.thoughtworks.dsl.keywords._
import com.thoughtworks.dsl.keywords.Shift.implicitShift
import com.thoughtworks.dsl.keywords.AsynchronousIo._
import java.io._
import java.net._
import java.nio._, channels._
def readAll(channel: AsynchronousByteChannel, destination: ByteBuffer): Task[Unit] = _ {
  if (destination.remaining > 0) {
    val numberOfBytesRead: Int = !Read(channel, destination)
    numberOfBytesRead match {
      case -1 =>
      case _  => !readAll(channel, destination)
    }
  } else {
    throw new IOException("The response is too big to read.")
  }
}
def writeAll[Domain](channel: AsynchronousByteChannel, destination: ByteBuffer): Task[Unit] = _ {
  while (destination.remaining > 0) {
    !Write(channel, destination)
  }
}
def httpClient(url: URL): Task[String] = _ {
  val socket = AsynchronousSocketChannel.open()
  try {
    val port = if (url.getPort == -1) 80 else url.getPort
    val address = new InetSocketAddress(url.getHost, port)
    !AsynchronousIo.Connect(socket, address)
    val request = ByteBuffer.wrap(s"GET ${url.getPath} HTTP/1.1\r\nHost:${url.getHost}\r\nConnection:Close\r\n\r\n".getBytes)
    !writeAll(socket, request)
    val response = ByteBuffer.allocate(100000)
    !readAll(socket, response)
    response.flip()
    io.Codec.UTF8.decoder.decode(response).toString
  } finally {
    socket.close()
  }
}

The usage of Task is similar to previous examples. You can check the result or exception in asynchronous handlers. But we also provides blockingAwait and some other utilities at domains.task.Task.

import com.thoughtworks.dsl.domains.task.Task.blockingAwait
val url = new URL("http://localhost:8085/ping")
val fileContent = blockingAwait(httpClient(url))
fileContent should startWith("HTTP/1.1 200 OK")

Another useful keyword for asynchronous programming is Fork, which duplicate the current control flow, and the child control flows are executed in parallel, similar to the POSIX fork system call. Fork should be used inside a com.thoughtworks.dsl.domains.task.Task#join block, which collects the result of each forked control flow.

import com.thoughtworks.dsl.keywords.Fork
import com.thoughtworks.dsl.keywords.Return
val Urls = Seq(
  new URL("http://localhost:8085/ping"),
  new URL("http://localhost:8085/pong")
)
def parallelTask: Task[Seq[String]] = {
  val url = !Fork(Urls)
  !Return(!httpClient(url))
}
inside(blockingAwait(parallelTask)) {
  case Seq(fileContent0, fileContent1) =>
    fileContent0 should startWith("HTTP/1.1 200 OK")
    fileContent1 should startWith("HTTP/1.1 200 OK")
}

try / catch / finally expressions are also supported in functions that return Stream[String] !! Throwable, because of the scala.Throwable part in the signature.

import com.thoughtworks.dsl.Dsl.!!, keywords._
var finallyBlockInvoked = 0
class MyException extends Exception
def f: Stream[String] !! Throwable = {
  while (true) {
    try {
      !new Yield("yield value")
      if (true) {
        throw new MyException
      }
    } catch {
      case e: RuntimeException =>
        throw new AssertionError("Should not catch an RuntimeException", e)
    } finally {
      finallyBlockInvoked += 1
    }
  }
  throw new AssertionError("Unreachable code")
}
val result = f { e =>
  e should be(a[MyException])
  Stream.empty
}
result should be(Stream("yield value"))
finallyBlockInvoked should be(1)

!!, or Continuation, is the preferred approach to enable multiple domains in one function. For example, you can create a function that lazily read each line of a BufferedReader to a Stream, automatically close the BufferedReader after reading the last line, and finally return the total number of lines in the Stream[String] !! Throwable !! Int domain.

import com.thoughtworks.dsl.Dsl.!!
import com.thoughtworks.dsl.keywords.Using
import com.thoughtworks.dsl.keywords.Yield
import com.thoughtworks.dsl.keywords.Shift._
import java.io._
def readerToStream(createReader: () => BufferedReader): Stream[String] !! Throwable !! Int = !! {
  val reader = !Using(createReader())
  def loop(lineNumber: Int): Stream[String] !! Throwable !! Int = _ {
    reader.readLine() match {
      case null =>
        lineNumber
      case line =>
        !Yield(line)
        !loop(lineNumber + 1)
    }
  }
  !loop(0)
}

!loop(0) is a shortcut of !Shift(loop(0)), because there is an implicit conversion from Stream[String] !! Throwable !! Int to a keywords.Shift case class, which is similar to the await keyword in JavaScript, Python or C#. A type like A !! B !! C means a domain-specific value of type C in the domain of A and B. When B is Throwable, the keywords.Using is available, which will close a resource when exiting the current function.

import scala.util.Success
var isClosed = false
def createReader() = {
  new BufferedReader(new StringReader("line1\nline2\nline3")) {
    override def close() = {
      isClosed = true
    }
  }
}
val stream = readerToStream(createReader _) { numberOfLines: Int =>
  numberOfLines should be(3)
  Function.const(Stream.empty)(_)
} { e: Throwable =>
  throw new AssertionError("Unexpected exception during readerToStream", e)
}
isClosed should be(false)
stream should be(Stream("line1", "line2", "line3"))
isClosed should be(true)

Instead of manually create the continuation-passing style function, you can also create the function from a !! block.

import com.thoughtworks.dsl.keywords.Yield
import com.thoughtworks.dsl.Dsl.!!
import scala.util.parsing.json._
def parseAndLog3(jsonContent: String, defaultValue: JSONType): Stream[String] !! JSONType = !! {
  !Yield(s"I am going to parse the JSON text $jsonContent...")
  JSON.parseRaw(jsonContent) match {
    case Some(json) =>
      !Yield(s"Succeeded to parse $jsonContent")
      json
    case None =>
      !Yield(s"Failed to parse $jsonContent")
      defaultValue
  }
}
val logs = parseAndLog3(""" { "key": "value" } """, JSONArray(Nil)) { json =>
  json should be(JSONObject(Map("key" -> "value")))
  Stream("done")
}
logs should be(Stream("I am going to parse the JSON text  { \"key\": \"value\" } ...",
                      "Succeeded to parse  { \"key\": \"value\" } ",
                      "done"))

Unlike the parseAndLog2 example, The code inside a !! block is not in an anonymous function. Instead, they are directly inside parseAndLog3, whose return type is Stream[String] !! JSONType. That is to say, the domain of those Yield keywords in parseAndLog3 is not Stream[String] any more, the domain is Stream[String] !! JSONType now, which supports more keywords, which you will learnt from the next examples, than the Stream[String] domain.

The closure in the previous example can be simplified with the help of Scala's placeholder syntax:

import com.thoughtworks.dsl.keywords.Yield
import com.thoughtworks.dsl.Dsl.!!
import scala.util.parsing.json._
def parseAndLog2(jsonContent: String, defaultValue: JSONType): Stream[String] !! JSONType = _ {
  !Yield(s"I am going to parse the JSON text $jsonContent...")
  JSON.parseRaw(jsonContent) match {
    case Some(json) =>
      !Yield(s"Succeeded to parse $jsonContent")
      json
    case None =>
      !Yield(s"Failed to parse $jsonContent")
      defaultValue
  }
}
val logs = parseAndLog2(""" { "key": "value" } """, JSONArray(Nil)) { json =>
  json should be(JSONObject(Map("key" -> "value")))
  Stream("done")
}
logs should be(Stream("I am going to parse the JSON text  { \"key\": \"value\" } ...",
                      "Succeeded to parse  { \"key\": \"value\" } ",
                      "done"))

Note that parseAndLog2 is equivelent to parseAndLog1. The code block after underscore is still inside a function whose return type is Stream[String].

Yield and Stream can be also used for logging. Suppose you have a function to parse an JSON file, you can append log records to a Stream during parsing.

import com.thoughtworks.dsl.keywords.Yield
import com.thoughtworks.dsl.Dsl.!!
import scala.util.parsing.json._
def parseAndLog1(jsonContent: String, defaultValue: JSONType): Stream[String] !! JSONType = { (callback: JSONType => Stream[String]) =>
  !Yield(s"I am going to parse the JSON text $jsonContent...")
  JSON.parseRaw(jsonContent) match {
    case Some(json) =>
      !Yield(s"Succeeded to parse $jsonContent")
      callback(json)
    case None =>
      !Yield(s"Failed to parse $jsonContent")
      callback(defaultValue)
  }
}

Since the function produces both a JSONType and a Stream of logs, the return type is now Stream[String] !! JSONType, where !! is (JSONType => Stream[String]) => Stream[String], an alias of continuation-passing style function, indicating it produces both a JSONType and a Stream of logs.

val logs = parseAndLog1(""" { "key": "value" } """, JSONArray(Nil)) { json =>
  json should be(JSONObject(Map("key" -> "value")))
  Stream("done")
}
logs should be(Stream("I am going to parse the JSON text  { \"key\": \"value\" } ...",
                      "Succeeded to parse  { \"key\": \"value\" } ",
                      "done"))

Suppose you want to create an Xorshift random number generator. The generated numbers should be stored in a lazily evaluated infinite Stream, which can be implemented as a recursive function that produce the next random number in each iteration, with the help of our built-in domain-specific keyword Yield.

import com.thoughtworks.dsl.Dsl.reset
import com.thoughtworks.dsl.keywords.Yield
def xorshiftRandomGenerator(seed: Int): Stream[Int] = {
  val tmp1 = seed ^ (seed << 13)
  val tmp2 = tmp1 ^ (tmp1 >>> 17)
  val tmp3 = tmp2 ^ (tmp2 << 5)
  !Yield(tmp3)
  xorshiftRandomGenerator(tmp3)
}: @reset
val myGenerator = xorshiftRandomGenerator(seed = 123)
myGenerator(0) should be(31682556)
myGenerator(1) should be(-276305998)
myGenerator(2) should be(2101636938)

Yield is an keyword to produce a value for a lazily evaluated Stream. That is to say, Stream is the domain where the DSL Yield can be used, which was interpreted like the yield keyword in C#, JavaScript or Python. Note that the body of xorshiftRandomGenerator is annotated as @reset, which enables the !-notation in the code block. Alternatively, you can also use the ResetEverywhere compiler plug-in, which enable !-notation for every methods and functions.