Evaluate the polynomial represented by this object at the point x.
Evaluate the polynomial represented by this object at the point x.
A representation of a point. The representation is as follows: The outer indexed sequence represents groups of features in feature space generated by the same feature expander. The inner sequence is an iterable sequence of key value pairs. This representation is used because it allows efficient encoding of sparse feature spaces. For instance, we can very easily encode multi-part bag of word models as follows. Let's say the 0th index in the outer sequence represents the title of HTML documents and the 1st index represents text inside the body tags in the HTML document.
case class Doc(title: String, body: String) { def multiPartBagOfWords = IndexedSeq(tokens(title), tokens(body)) private[this] def tokens(s: String) = s.trim.toLowerCase.split("\\W+").groupBy(identity).mapValues(_.size.toDouble) } val fox = Doc("fox story", "The quick brown fox jumps over the lazy dog") val p: PolynomialEvaluationAlgo = ... p at fox.multiPartBagOfWords
this polynomial evaluated at x.
A tail-recursive variant of hSmall that won't stack overflow on deep trees.
A tail-recursive variant of hSmall that won't stack overflow on deep trees. This function is slower in empirical tests that hSmall.
an input vector
a stack of trees and current sums at those trees (replaces call stack in non-tail-recursive hSmall).
the current sum.
this polynomial evaluated at x.
Accumulate, in a depth-first way, the evaluation of the polynomial.
Accumulate, in a depth-first way, the evaluation of the polynomial.
NOTE: This algorithm is recursive (but not tail-recursive) by seems to be about 20% faster than the following tail-recursive equivalent. This is probably due to additional object creation in the tail-recursive method. Tuple2 instances must be created and the linked list containers probably needs to be created. In the non-tail recursive version, we don't do any of this and only perform arithmetic operations (aside from the iterators and the Options created in the descendant lookup).
We aren't too afraid of stack overflows because these trees will typically be shallow. This is because most use cases don't involve polynomials in thousands of variables (or of degree in the thousands). That being said, stack overflows are a real possibility (in test at a depth of ~3000). To combat this, hBig is provided but not yet integrated into the at function.
import collection.mutable.{Stack => MStack} def h3(x: IndexedSeq[Seq[(String, Double)]], s: MStack[(MapTreeLike[String, Value], Double)], z: Double): Double = { if (s.isEmpty) z else { val h = s.pop for (i <- h._1.value.applicableFeatures; p <- x(i); c <- h._1.descendants.get(p._1)) s.push((c, p._2 * h._2)) h3(x, s, z + h._1.value.weight * h._2) } }
an input vector
size of input vector (computed once for possible speed improvement).
a tree containing the information necessary to compute the higher order inner product
product of the items in the path from the root to leaf.
this polynomial evaluated at x.
An algorithm for efficiently evaluating polynomials at a given point. Evaluating first order polynomials is obviously a sub case, which is important because first order polynomial evaluation is isomorphic to linear regression, which may be the standard use case.
As an example, imagine that we wanted to evaluate Z(u,v,x,y) = wu,vuv + wu,v,xuvx + wu,v,yuvy for coefficients W = [wu,v, wu,v,x, wu,v,y]T.
This is:
That Z can be factored indicates there is a way to structure the algorithm to efficiently reuse computations. The way to achieve this is to structure the possible polynomials as a tree and traverse and aggregate over the tree. As the tree is traversed, the computation is accumulated. As a concrete example, let's take the above example and show it using real code. Then a motivation of the example will be provided.
The computation tree works as follows: the edge labels are multiplied by the associated coefficient (0 if non-existent) to get the node values. Node values are added together to get the inner product. So, every time we descend farther into the tree, we multiply the running product by the value we extract from the input vector X and every time a weight is found, it is multiplied by the current product and added to the running sum. The process recurses until the tree can no longer by traversed. The sum is then returned.
While constructing a PolynomialEvaluator via direct means is entirely possible, it is less straightforward than using a builder to do it. Below, we show a better way to construct PolynomialEvaluator instances where we just specify the terms in the polynomial and the associated coefficient values. Note linear regression is the special case when all of the inner maps contain exactly one element.
Notice the values in the inner map look a little weird. These are the indices into the input vector x from which the key comes. This is for efficiency purposes but allows the algorithm to dramatically prune the search space while accumulating over the tree.