A ParseScore maps a candidate parse to a score.
A ParseStatistic accrues a particular statistic over (candidate parse, gold parse) pairs.
The PathAccuracyScore computes the percentage of a candidate parse's tokens that have a completely correct breadcrumb path (i.e.
The PathAccuracyScore computes the percentage of a candidate parse's tokens that have a completely correct breadcrumb path (i.e. if you follow a token's breadcrumbs to the nexus in both the candidate and the gold parse, you encounter the same set of tokens in the same order).
FirstMistakeAggregator collates the "first" incorrect transition decisions that a parser makes with respect to gold parses.
FirstMistakeAggregator collates the "first" incorrect transition decisions that a parser makes with respect to gold parses. For instance, say there's a parse whose gold transition representation is [Sh, Sh, Rt('nsubj), Rt('det)], and a candidate parse comes up with [Sh, Rt('adjp), Sh, Rt('det)]. The "first" incorrect transition decision is the pair (Rt('adjp), Sh). In other words, the first mistake that the candidate parse made was to guess transition Rt('adjp) when it should have guessed Sh.
UnlabeledBreadcrumbAccuracy stores the statistics necessary to compute Unlabeled Attachment Score (UAS), which is the percentage of correct breadcrumbs over a set of candidate parses.
A ParseStatistic accrues a particular statistic over (candidate parse, gold parse) pairs. Every time .notify() is called, the statistic is updated.