yaidom

Introduction

Yet another immutable DOM-like API. Hence the name yaidom. This is not an implementation of W3C DOM. Instead, this is a Scala-ish DOM-like API. Foremost, that means that this API is centered around Scala Collections of immutable nodes.

By design, some characteristics of this API are:

• The DOM-like trees of this API are (deeply) immutable and therefore thread-safe. This facilitates safe multi-threaded querying of these trees, at the cost of a relatively high memory footprint.
• This API leverages the highly expressive Scala Collections API, on the inside and from the outside. Querying yaidom trees is easy to achieve using Scala for comprehensions, manipulating immutable Scala collections. It should often even be attractive to use this API instead of XPath, XSLT and/or XQuery, at the cost of somewhat more verbosity, loss of (schema) typing information, and loss of a standard XML query language. Much of the attraction of yaidom for querying lies in its simplicity. That is, understanding the Scala Collections API (from the perspective of a user) is almost all that is needed for understanding querying in yaidom. In comparison, XSLT is known to be a hard language to master.
• Yaidom is also Scala-ish in the use of Options over nulls. It is clear from method and variable names where Options are used. Also, methods whose names start with "find" tend to return Options.
• This API explicitly models and distinguishes between qualified names and expanded names. The same holds for namespace declarations and namespace scopes. See for example http://www.w3.org/TR/xml-names11/. The clear distinction between qualified names and expanded names helps in making yaidom queries easy to understand. Yaidom queries indeed have simple and precise semantics. Moreover, the four concepts (QNames, ENames, declarations and scopes) obey some obvious properties, that help in keeping yaidom small and simple, both in API and implementation.
• Namespaces are first-class citizens in yaidom. DTDs are not (but DTDs do not understand namespaces anyway). See for example http://docstore.mik.ua/orelly/xml/xmlnut/ch04_04.htm.
• This API has good interop with standard Java XML APIs, that is, JAXP. Printing and parsing XML (and details of "escaping") are delegated to JAXP, without taking away any control over JAXP configuration. By configuring JAXP objects (for parsing or printing XML), we keep some control over the XML string representations of serialized yaidom elements.

The yaidom API is most suitable for processing "data-oriented" XML, roughly having the following properties:

• Tags and text are not mixed.
• Inter-element whitespace is typically "ignorable", and only used for formatting.
• The structure of the XML is known, and typically described in an XSD.
• Namespaces are heavily used.
• DTDs are not used.
• The exact string representation is not important, such as short or long form of empty elements, the order of attributes, including namespace declarations, even the exact namespace prefixes used, etc. (Yaidom elements do keep the order of non-namespace-declaration attributes.) A "DOM-centric" view of XML may well suffice for processing such XML.

That is not to say that yaidom can only be used for processing such "data-oriented" XML.

Yaidom is not a very ambitious API:

• The API has no XPath(-like) support, and does not unify nodes with sequences of nodes. That makes code using yaidom more verbose than XPath(-like) code found in APIs like Scala's XML library or Anti-XML. On the other hand, code using yaidom is very easy to reason about, and yaidom is also easy to implement.
• Yaidom does not simplify printing/parsing of XML. The upside is that yaidom has a very good power-to-weight ratio.
• Yaidom only knows about XML, not about HTML. On the other hand, TagSoup can be used as SAX Parser.
• Yaidom does not know about XML Schema types (or DTD types), for example types of attributes.
• APIs such as JAXP, JDOM and XOM are more feature-rich, for example in support for base URIs.
• The API does not try to define equality for XML trees, because that's a very vague notion for XML. At least that is the short story. See "resolved" eu.cdevreeze.yaidom.resolved.Elem for "bare bones" elements that do obey some notion of equality.

Yaidom has been inspired by Anti-XML. The Anti-XML library tackles some weaknesses of Scala's standard XML API, w.r.t. robustness, ease of use and performance. Yaidom tries to achieve the same (in a different way), but yaidom is less ambitious, foremost in not offering any XPath(-like) support.

Different element types in yaidom

Yaidom considers XML to be trees of nodes, rather than text strings obeying "XML rules". Think InfoSet and DOM, rather than the XML spec. By considering XML to be node trees, round-tripping (from XML string to DOM-like tree, and back) cannot be lossless. Again, if lossless round-tripping is important (like is the case for XML editors), yaidom (like DOM) may not be a good fit.

Yet yaidom has more than just one type of (DOM-like) element. First of all, yaidom distinguishes between:

• A uniform querying API for DOM-like elements, as traits eu.cdevreeze.yaidom.ParentElemLike and sub-traits. These partially abstract traits are mixed in by concrete element classes, but can also be mixed in by different element classes not offered by yaidom. In other words, these traits make yaidom extensible, as well as uniform (as querying API). These traits implement a rich querying API in terms of only a few abstract methods.
• Concrete element classes, like eu.cdevreeze.yaidom.Elem, mixing in these traits. By implementing only a few abstract methods required by the mixed-in traits, these concrete element classes get a rich element querying API almost for free.

Concrete element classes in yaidom that mix in trait eu.cdevreeze.yaidom.ParentElemLike (or sub-traits) are:

• The "default" yaidom element type: eu.cdevreeze.yaidom.Elem. These elements are immutable and thread-safe. They do not know about their parents, and do not keep namespace declarations, but they do know about in-scope namespaces.
• The element builder type: eu.cdevreeze.yaidom.ElemBuilder. These objects are also immutable and thread-safe, but are builders of the above-mentioned elements. Whereas Elems know about in-scope namespaces but not about (own) namespace declarations, ElemBuilders know about (own) namespace declarations but not about in-scope namespaces.
• The "stripped" element type: eu.cdevreeze.yaidom.resolved.Elem. These elements do not know about namespace prefixes, but hold ("resolved") namespace URIs instead, and are therefore easy to compare for some (sensible) notion of equality.
• The "indexed" element type: eu.cdevreeze.yaidom.indexed.Elem. Unlike "standard" yaidom elements, these elements are a "top-down notion" of elements, that know about their ancestry. Still, these elements are immutable. (They are "views" on "standard" yaidom elements, each as a pair of a root element and an element path.)
• The wrapper type around DOM nodes: eu.cdevreeze.yaidom.dom.DomElem. They are ParentElemLike "views" backed by org.w3c.dom.Element elements. These views are mutable and "volatile", so far less safe to use than the immutable elements mentioned above.

Often these different element types can be used well together. For example:

• The "default" element type is useful for querying XML trees (without having to worry about mutable state and thread-safety)
• The "element builders" are useful for constructing (parts of) such element trees
• The "resolved" elements may be handy for equality comparisons and pattern matching on (parts of) XML trees
• The "indexed" elements are useful for processing XML where context is important, such as XML schemas
• The "DOM wrappers" may be handy for in-place updates

In the spirit of the "DOM node wrappers", one could easily come up with similar wrappers around JDOM, XOM, etc., mixing in trait eu.cdevreeze.yaidom.ParentElemLike (or subtraits). Yaidom does not offer these wrappers, but some test cases show that such wrappers are easy to develop.

Examples

Below follow some examples. The first example queries for all book elements having a price below 90. It can be written as follows, assuming a book store Document with the appropriate structure:

val bookstoreElm = doc.documentElement
require(bookstoreElm.localName == "Bookstore")

val bookElms =
  for {
    bookElm <- bookstoreElm \ (_.localName == "Book")
    price <- bookElm \@ EName("Price")
    if price.toInt < 90
  } yield bookElm

The for-comprehension is equivalent to:

val bookElms =
for {
  bookElm <- bookstoreElm filterChildElems { _.localName == "Book" }
  price <- bookElm.attributeOption(EName("Price"))
  if price.toInt < 90
} yield bookElm

All books having (at least) Jeffrey Ullman as author can be found as follows:

val ullmanBookElms =
for {
  bookElm <- bookstoreElm filterChildElems { _.localName == "Book" }
  if (bookElm \\ (_.localName == "Author")) exists { e =>
    ((e.getChildElem(_.localName == "First_Name")).text == "Jeffrey") &&
    ((e.getChildElem(_.localName == "Last_Name")).text == "Ullman")
  }
} yield bookElm

An alternative way to write the same query, but using eu.cdevreeze.yaidom.ElemPath instances instead, is as follows:

val ullmanBookElms =
for {
  authorPath <- bookstoreElm filterElemPaths { e =>
    (e.localName == "Author") &&
    ((e.getChildElem(_.localName == "First_Name")).text == "Jeffrey") &&
    ((e.getChildElem(_.localName == "Last_Name")).text == "Ullman")
  }
  bookPath <- authorPath findAncestorPath { _.endsWithName(EName("Book")) }
} yield bookstoreElm.getWithElemPath(bookPath)

This is conceptually similar to navigating in XPath to the grandparent nodes of the matching Author elements.

Alternatively, using eu.cdevreeze.yaidom.indexed.Elem instances, we could write:

val ullmanBookElms =
for {
  authorElm <- indexed.Elem(bookstoreElm) filterElems { e =>
    (e.localName == "Author") &&
    ((e.getChildElem(_.localName == "First_Name")).text == "Jeffrey") &&
    ((e.getChildElem(_.localName == "Last_Name")).text == "Ullman")
  }
  bookElm <- authorElm findAncestor { _.resolvedName == EName("Book") }
} yield bookElm

What's in the API, and what are the dependencies?

This package contains the following parts, in order of dependencies (starting with the class without any dependencies):

• Basic concepts such as eu.cdevreeze.yaidom.QName, eu.cdevreeze.yaidom.EName, eu.cdevreeze.yaidom.Declarations and eu.cdevreeze.yaidom.Scope. At the same level is class eu.cdevreeze.yaidom.ElemPath.
• Trait eu.cdevreeze.yaidom.ParentElemLike, which offers the core yaidom querying API. It is extended by trait eu.cdevreeze.yaidom.ElemLike, which is extended by trait eu.cdevreeze.yaidom.PathAwareElemLike, which in turn is extended by trait eu.cdevreeze.yaidom.UpdatableElemLike. At the same level is trait eu.cdevreeze.yaidom.HasText. All these APIs turn small abstract APIs into rich APIs with implementations, and are mixed in by "element" classes.
• The "node tree type hierarchy", as sealed trait eu.cdevreeze.yaidom.Node and its subtypes, such as eu.cdevreeze.yaidom.Elem (which indeed mixes in the above-mentioned traits UpdatableElemLike and HasText).
• Class eu.cdevreeze.yaidom.Document (a Document is not considered a Node itself by yaidom).
• Trait eu.cdevreeze.yaidom.NodeBuilder and its subtypes, such as eu.cdevreeze.yaidom.ElemBuilder. Node builders can be used in a DSL-like fashion, for creation of Elems. Node builders postpone the choice of Scopes (whereas the Nodes that they create all must have a fixed Scope), so node builders are indeed intended to be handy for creation of node trees. At the same level are eu.cdevreeze.yaidom.ConverterToElem, eu.cdevreeze.yaidom.ElemConverter, etc.
• Class eu.cdevreeze.yaidom.DocBuilder (a DocBuilder is not considered a NodeBuilder itself by yaidom).

Dependencies are all uni-directional. All types in this package are (deeply) immutable. That holds even for the eu.cdevreeze.yaidom.NodeBuilder instances.

Parsing and printing of XML are not handled in this package. Even the toString methods for nodes use the NodeBuilder DSL syntax rather than XML string syntax. Hence the complex details of character escaping, "ignorable whitespace" etc. are not handled in this package. Parsing and printing of XML are offered by the eu.cdevreeze.yaidom.parse and eu.cdevreeze.yaidom.print subpackages, which depend on the eu.cdevreeze.yaidom.convert subpackage. Those subpackages depend on this package, and not the other way around. Put differently, they are in this namespace.

Yaidom also offers packages eu.cdevreeze.yaidom.resolved, eu.cdevreeze.yaidom.indexed and eu.cdevreeze.yaidom.xlink. The resolved package offers "bare bones" elements, stripped down to "essentials" (replacing prefixes by namespace URIs, removing comments, etc.), such that those elements can be compared for some notion of equality. The indexed package offers immutable elements knowing their ancestry. The xlink package offers some basic XLink support.

There is also a package eu.cdevreeze.yaidom.dom for ElemLike wrappers around (mutable!) DOM elements.