org.allenai.nlpstack.parse.poly

eval

package eval

Visibility
  1. Public
  2. All

Type Members

  1. trait ParseScore extends (Option[PolytreeParse]) ⇒ Double

    A ParseScore maps a candidate parse to a score.

  2. sealed abstract class ParseStatistic extends AnyRef

    A ParseStatistic accrues a particular statistic over (candidate parse, gold parse) pairs.

    A ParseStatistic accrues a particular statistic over (candidate parse, gold parse) pairs. Every time .notify() is called, the statistic is updated.

  3. case class PathAccuracyScore(goldParses: Map[String, PolytreeParse]) extends ParseScore with Product with Serializable

    The PathAccuracyScore computes the percentage of a candidate parse's tokens that have a completely correct breadcrumb path (i.e.

    The PathAccuracyScore computes the percentage of a candidate parse's tokens that have a completely correct breadcrumb path (i.e. if you follow a token's breadcrumbs to the nexus in both the candidate and the gold parse, you encounter the same set of tokens in the same order).

Value Members

  1. object Evaluate

  2. object ParseEvaluator

  3. object PathAccuracy extends ParseStatistic with Product with Serializable

    FirstMistakeAggregator collates the "first" incorrect transition decisions that a parser makes with respect to gold parses.

    FirstMistakeAggregator collates the "first" incorrect transition decisions that a parser makes with respect to gold parses. For instance, say there's a parse whose gold transition representation is [Sh, Sh, Rt('nsubj), Rt('det)], and a candidate parse comes up with [Sh, Rt('adjp), Sh, Rt('det)]. The "first" incorrect transition decision is the pair (Rt('adjp), Sh). In other words, the first mistake that the candidate parse made was to guess transition Rt('adjp) when it should have guessed Sh.

  4. object UnlabeledBreadcrumbAccuracy extends ParseStatistic with Product with Serializable

    UnlabeledBreadcrumbAccuracy stores the statistics necessary to compute Unlabeled Attachment Score (UAS), which is the percentage of correct breadcrumbs over a set of candidate parses.

Ungrouped