The domain-specific interpreter for Keyword
in Domain
,
which is a dependent type type class that registers an asynchronous callback function,
to handle the Value
inside Keyword
.
The domain-specific interpreter for Keyword
in Domain
,
which is a dependent type type class that registers an asynchronous callback function,
to handle the Value
inside Keyword
.
杨博 (Yang Bo)
The value held inside Keyword
.
Provides utilities to perform for
-comprehension on Dsl.Keyword.
Provides utilities to perform for
-comprehension on Dsl.Keyword.
Add the following import statement to enable for
-comprehension on keywords:
import com.thoughtworks.dsl.comprehension._
This example implements the same feature as the example on Scaladoc of keywords.Yield,
except this example use for
-comprehension instead of !-notation.
import com.thoughtworks.dsl.Dsl import com.thoughtworks.dsl.keywords._ import com.thoughtworks.dsl.comprehension._ def gccFlagBuilder(sourceFile: String, includes: String*) = { for { _ <- Yield("gcc") _ <- Yield("-c") _ <- Yield(sourceFile) include <- Each(includes) _ <- Yield("-I") _ <- Yield(include) r <- Continue } yield r: String } gccFlagBuilder("main.c", "lib1/include", "lib2/include").to[Stream] should be(Stream("gcc", "-c", "main.c", "-I", "lib1/include", "-I", "lib2/include"))
Alternatively, you can use Scala native yield
keyword to produce the last value.
def gccFlagBuilder2(sourceFile: String, includes: String*) = { for { _ <- Yield("gcc") _ <- Yield("-c") _ <- Yield(sourceFile) include <- Each(includes) _ <- Yield("-I") } yield include } gccFlagBuilder2("main.c", "lib1/include", "lib2/include").to[Stream] should be(Stream("gcc", "-c", "main.c", "-I", "lib1/include", "-I", "lib2/include"))
You can also omit the explicit constructor of keywords.Yield with the help of implicit conversion keywords.Yield.implicitYield.
import com.thoughtworks.dsl.keywords.Yield.implicitYield def augmentString = () def wrapString = ()
Note that scala.Predef.augmentString and scala.Predef.wrapString must be disabled in order to use flatMap
for keywords.Yield.
def gccFlagBuilder3(sourceFile: String, includes: String*) = { for { _ <- "gcc" _ <- "-c" _ <- sourceFile include <- Each(includes) _ <- "-I" } yield include } gccFlagBuilder3("main.c", "lib1/include", "lib2/include").to[Stream] should be(Stream("gcc", "-c", "main.c", "-I", "lib1/include", "-I", "lib2/include"))
for
/ yield
expressions can be used on keywords.
import com.thoughtworks.dsl.keywords._ def cartesianProduct = for { i <- Each(Array(1, 2, 3)) j <- Each(Vector(1, 10, 100, 1000)) } yield i * j
The results of for
/ yield
expressions are also keywords.
cartesianProduct should be(a[Dsl.Keyword[_, _]])
You can use !-notation extract the value from the produced keyword.
def resultAsList = List(!cartesianProduct) resultAsList should be(List(1, 10, 100, 1000, 2, 20, 200, 2000, 3, 30, 300, 3000)) def resultAsSet: Set[Int] = !Return(!cartesianProduct) resultAsSet should be(Set(1, 10, 100, 1000, 2, 20, 200, 2000, 3, 30, 300, 3000))
Alternatively, comprehension.ComprehensionOps.to can be used to convert the result of a keyword to other types of values as well.
cartesianProduct.to[List] should be(List(1, 10, 100, 1000, 2, 20, 200, 2000, 3, 30, 300, 3000))
Contains built-in domain-specific Keywords and their corresponding interpreters.
This project, Dsl.scala, is a framework to create embedded Domain-Specific Languages.
DSLs written in Dsl.scala are collaborative with others DSLs and Scala control flows. DSL users can create functions that contains interleaved DSLs implemented by different vendors, along with ordinary Scala control flows.
We also provide some built-in DSLs for asynchronous programming, collection manipulation, and adapters to scalaz.Monad or cats.Monad. Those built-in DSLs can be used as a replacement of
for
comprehension, scala-continuations, scala-async, Monadless, effectful and ThoughtWorks Each.Introduction
Reinventing control flow in DSL
Embedded DSLs usually consist of a set of domain-specific keywords, which can be embedded in the their hosting languages.
Ideally, a domain-specific keyword should be an optional extension, which can be present everywhere in the ordinary control flow of the hosting language. However, previous embedded DSLs usually badly interoperate with hosting language control flow. Instead, they reinvent control flow in their own DSL.
For example, the akka provides a DSL to create finite-state machines, which consists of some domain-specific keywords like when, goto and stay. Unfortunately, you cannot embedded those keywords into your ordinary
if
/while
/try
control flows, because Akka's DSL is required to be split into small closures, preventing ordinary control flows from crossing the boundary of those closures.TensorFlow's control flow operations and Caolan's async library are examples of reinventing control flow in languages other than Scala.
Monad: the generic interface of control flow
It's too trivial to reinvent the whole set of control flows for each DSL. A simpler approach is only implementing a minimal interface required for control flows for each domain, while the syntax of other control flow operations are derived from the interface, shared between different domains.
Since computation can be represented as monads, some libraries use monad as the interface of control flow, including scalaz.Monad, cats.Monad and com.twitter.algebird.Monad. A DSL author only have to implement two abstract method in scalaz.Monad, and all the derived control flow operations like scalaz.syntax.MonadOps.whileM, scalaz.syntax.BindOps.ifM are available. In addition, those monadic data type can be created and composed from Scala's built-in
for
comprehension.For example, you can use the same syntax or
for
comprehension to create random value generators and data-binding expressions, as long as there are Monad instances for org.scalacheck.Gen and com.thoughtworks.binding.Binding respectively.Although the effort of creating a DSL is minimized with the help of monads, the syntax is still unsatisfactory. Methods in
MonadOps
still seem like a duplicate of ordinary control flow, andfor
comprehension supports only a limited set of functionality in comparison to ordinary control flows.if
/while
/try
and other block expressions cannot appear in the enumerator clause offor
comprehension.Enabling ordinary control flows in DSL via macros
An idea to avoid inconsistency between domain-specific control flow and ordinary control flow is converting ordinary control flow to domain-specific control flow at compiler time.
For example, scala.async provides a macro to generate asynchronous control flow. The users just wrap normal synchronous code in a async block, and it runs asynchronously.
This approach can be generalized to any monadic data types. ThoughtWorks Each, Monadless and effectful are macros that convert ordinary control flow to monadic control flow.
For example, with the help of ThoughtWorks Each, Binding.scala is used to create reactive HTML templating from ordinary Scala control flow.
Delimited continuations
Another generic interface of control flow is continuation, which is known as the mother of all monads, where control flows in specific domain can be supported by specific final result types of continuations.
scala-continuations and stateless-future are two delimited continuation implementations. Both projects can convert ordinary control flow to continuation-passing style closure chains at compiler time.
For example, stateless-future-akka, based on
stateless-future
, provides a special final result type for akka actors. Unlike akka.actor.FSM's inconsistent control flows, users can create complex finite-state machines from simple ordinary control flows along withstateless-future-akka
's domain-specific keywordnextMessage
.Collaborative DSLs
All the above approaches lack of the ability to collaborate with other DSLs. Each of the above DSLs can be only exclusively enabled in a code block. For example, scala-continuations enables calls to
@cps
method in scala.util.continuations.reset blocks, and ThoughtWorks Each enables the magiceach
method for scalaz.Monad in com.thoughtworks.each.Monadic.monadic blocks. It is impossible to enable both DSLs in one function.This Dsl.scala project resolves this problem.
We also provide adapters to all the above kinds of DSLs. Instead of switching different DSLs between different functions, DSL users can use multiple DSLs together in one function, by simply adding our Scala compiler plug-in.
Dsl.scala also supports scalaz.MonadTrans. Considering the line counter implemented in previous example may be failed for some files, due to permission issue or other IO problem, you can use scalaz.OptionT monad transformer to mark those failed file as a None.
Note that our keywords are adaptive to the domain it belongs to. Thus, instead of explicit
!Monadic(OptionT.optionTMonadTrans.liftM(Stream(children: _*)))
, you can simply have!Stream(children: _*)
. The implicit lifting feature looks like Idris's effect monads, though the mechanisms is different fromimplicit lift
in Idris.The previous code requires a
toStream
conversion onchildren
, becausechildren
's typeArray[File]
does not fit theF
type parameter in scalaz.Monad.bind. The conversion can be avoided if using CanBuildFrom type class instead of monads. We provide a Each keyword to extract each element in a Scala collection. The behavior is similar to monad, except the collection type can vary. For example, you can extract each element from an Array, even when the return type (or the domain) is a Stream.Unlike Haskell's do-notation or Idris's !-notation, Dsl.scala allows non-monadic keywords like Each works along with monads.
The built-in keywords.Monadic can be used as an adaptor to scalaz.Monad and scalaz.MonadTrans, to create monadic code from imperative syntax, similar to the !-notation in Idris. For example, suppose you are creating a program that counts lines of code under a directory. You want to store the result in a Stream of line count of each file.
If you don't need to collaborate to Stream or other domains, you can use
TailRec[Unit] !! Throwable !! A
or the alias domains.task.Task as the return type, which can be used as an asynchronous task that support RAII, as a higher-performance replacement of scala.concurrent.Future, scalaz.concurrent.Task or monix.eval.Task. Also, there are some keywords in keywords.AsynchronousIo to asynchronously perform Java NIO.2 IO operations in a domains.task.Task domain. For example, you can implement an HTTP client from those keywords.The usage of
Task
is similar to previous examples. You can check the result or exception in asynchronous handlers. But we also provides blockingAwait and some other utilities at domains.task.Task.Another useful keyword for asynchronous programming is Fork, which duplicate the current control flow, and the child control flows are executed in parallel, similar to the POSIX
fork
system call. Fork should be used inside a com.thoughtworks.dsl.domains.task.Task#join block, which collects the result of each forked control flow.try
/catch
/finally
expressions are also supported in functions that returnStream[String] !! Throwable
, because of the scala.Throwable part in the signature.!!, or Continuation, is the preferred approach to enable multiple domains in one function. For example, you can create a function that lazily read each line of a BufferedReader to a Stream, automatically close the BufferedReader after reading the last line, and finally return the total number of lines in the
Stream[String] !! Throwable !! Int
domain.!loop(0)
is a shortcut of!Shift(loop(0))
, because there is an implicit conversion fromStream[String] !! Throwable !! Int
to a keywords.Shift case class, which is similar to theawait
keyword in JavaScript, Python or C#. A type likeA !! B !! C
means a domain-specific value of typeC
in the domain ofA
andB
. WhenB
is Throwable, the keywords.Using is available, which will close a resource when exiting the current function.Instead of manually create the continuation-passing style function, you can also create the function from a !! block.
Unlike the
parseAndLog2
example, The code inside a!!
block is not in an anonymous function. Instead, they are directly insideparseAndLog3
, whose return type isStream[String] !! JSONType
. That is to say, the domain of those Yield keywords inparseAndLog3
is notStream[String]
any more, the domain isStream[String] !! JSONType
now, which supports more keywords, which you will learnt from the next examples, than theStream[String]
domain.The closure in the previous example can be simplified with the help of Scala's placeholder syntax:
Note that
parseAndLog2
is equivelent toparseAndLog1
. The code block after underscore is still inside a function whose return type isStream[String]
.Yield and Stream can be also used for logging. Suppose you have a function to parse an JSON file, you can append log records to a Stream during parsing.
Since the function produces both a JSONType and a Stream of logs, the return type is now
Stream[String] !! JSONType
, where !! is(JSONType => Stream[String]) => Stream[String]
, an alias of continuation-passing style function, indicating it produces both a JSONType and a Stream of logs.Suppose you want to create an Xorshift random number generator. The generated numbers should be stored in a lazily evaluated infinite Stream, which can be implemented as a recursive function that produce the next random number in each iteration, with the help of our built-in domain-specific keyword Yield.
Yield is an keyword to produce a value for a lazily evaluated Stream. That is to say, Stream is the domain where the DSL Yield can be used, which was interpreted like the
yield
keyword in C#, JavaScript or Python. Note that the body ofxorshiftRandomGenerator
is annotated as@reset
, which enables the !-notation in the code block. Alternatively, you can also use the ResetEverywhere compiler plug-in, which enable !-notation for every methods and functions.domains.cats for using !-notation with cats.
domains.scalaz for using !-notation with scalaz.
Dsl for the guideline to create your custom DSL.