Compute the "Area Under the Curve" for a collection of predictions.
Compute the "Area Under the Curve" for a collection of predictions. Uses the Trapezoid method to compute the area.
Internally a linspace is defined using the given number of samples. Each point in the linspace represents a threshold which is used to build a confusion matrix. The area is then defined on this list of confusion matrices.
AUCMetric which is given to the aggregate selects the function to apply on the confusion matrix prior to the AUC calculation.
Which function to apply on the confusion matrix.
Number of samples to use for the curve definition.
Which function to apply on the list of confusion matrices prior to the AUC calculation.
Special Case for a Binary Confusion Matrix to make it easier to compose with other binary aggregators
Special Case for a Binary Confusion Matrix to make it easier to compose with other binary aggregators
Threshold to apply on predictions
Split predictions into Tensorflow Model Analysis compatible CalibrationHistogramBucket buckets.
Split predictions into Tensorflow Model Analysis compatible CalibrationHistogramBucket buckets.
If a prediction is less than the lower bound, it belongs to the bucket [-inf, lower bound) If it is greater than or equal to the upper bound, it belongs to the bucket (upper bound, inf]
Left boundary, inclusive
Right boundary, exclusive
Number of buckets in the histogram
Histogram bucket.
Histogram bucket.
Lower bound on bucket, inclusive
Upper bound on bucket, exclusive
Number of predictions in this bucket
Sum of label values for this bucket
Sum of prediction values for this bucket
Generate a Classification Report for a collection of binary predictions.
Generate a Classification Report for a collection of binary predictions. The output of this aggregator will be a Report object.
Threshold to apply to get the predictions.
Beta parameter used in the f-score calculation.
Generic Consfusion Matrix Aggregator for any dimension.
Generic Consfusion Matrix Aggregator for any dimension. Thresholds must be applied to make a prediction prior to using this aggregator.
List of possible label values
Compute a series of points for a collection of predictions.
Compute a series of points for a collection of predictions.
Internally a linspace is defined using the given number of samples. Each point in the linspace represents a threshold which is used to build a confusion matrix. The (x,y) location of the line is then returned.
AUCMetric which is given to the aggregate selects the function to apply on the confusion matrix prior to the AUC calculation.
Which function to apply on the confusion matrix.
Number of samples to use for the curve definition.
Returns the mean average precision (MAP) of all the predictions.
Returns the mean average precision (MAP) of all the predictions. If a query has an empty ground truth set, the average precision will be zero
Aggregator which combines an unbounded list of other aggregators.
Aggregator which combines an unbounded list of other aggregators. Each aggregator in the list is tagged by a string. The string(aka name) could be used to retrieve the aggregated value from the Map emitted by the "present" function.
Generate a Classification Report for a collection of multiclass predictions.
Generate a Classification Report for a collection of multiclass predictions. A report is generated for each class by treating the predictions as binary of either "class" or "not class". The output of this aggregator will be a map of classes and their Report objects.
List of possible label values.
Beta parameter used in the f-score calculation.
Compute the average NDCG value of all the predictions, truncated at ranking position k.
Compute the average NDCG value of all the predictions, truncated at ranking position k. The discounted cumulative gain at position k is computed as: sumi=1k (2{relevance of ith item} - 1) / log(i + 1), and the NDCG is obtained by dividing the DCG value on the ground truth set. In the current implementation, the relevance value is binary. If a query has an empty ground truth set, zero will be used as ndcg
See the following paper for detail:
IR evaluation methods for retrieving highly relevant documents. K. Jarvelin and J. Kekalainen
the position to compute the truncated ndcg, must be positive
Compute the average precision of all the predictions, truncated at ranking position k.
Compute the average precision of all the predictions, truncated at ranking position k.
If for a prediction, the ranking algorithm returns n (n is less than k) results, the precision value will be computed as #(relevant items retrieved) / k. This formula also applies when the size of the ground truth set is less than k.
If a prediction has an empty ground truth set, zero will be used as precision together
See the following paper for detail:
IR evaluation methods for retrieving highly relevant documents. K. Jarvelin and J. Kekalainen
the position to compute the truncated precision, must be positive
Generic Prediction Object used by most aggregators
Generic Prediction Object used by most aggregators
Type of the Real Value
Type of the Predicted Value
Real value for this entry. Also normally seen as label.
Predicted value. Can be a class or a score depending on the aggregator.
Classification Report
Classification Report
Measurement of what percentage of values were predicted incorrectly.