Evaluation (deeplearning4j-nn 0.6.0 API)

java.lang.Object
- org.deeplearning4j.eval.Evaluation

All Implemented Interfaces:

Serializable
```
public class Evaluation
extends Object
implements Serializable
```
Evaluation metrics: precision, recall, f1

Author:

Adam Gibson

See Also:

Serialized Form

Field Summary

Fields
Modifier and Type	Field and Description
`protected ConfusionMatrix<Integer>`	`confusion`
`protected static double`	`DEFAULT_EDGE_VALUE`
`protected Counter<Integer>`	`falseNegatives`
`protected Counter<Integer>`	`falsePositives`
`protected List<String>`	`labelsList`
`protected static org.slf4j.Logger`	`log`
`protected int`	`numRowCounter`
`protected Counter<Integer>`	`trueNegatives`
`protected Counter<Integer>`	`truePositives`

Constructor Summary

Constructors
Constructor and Description
`Evaluation()`
`Evaluation(int numClasses)` The number of classes to account for in the evaluation
`Evaluation(List<String> labels)` The labels to include with the evaluation.
`Evaluation(Map<Integer,String> labels)` Use a map to generate labels Pass in a label index with the actual label you want to use for output

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`double`	`accuracy()` Accuracy: (TP + TN) / (P + N)
`void`	`addToConfusion(Integer real, Integer guess)` Adds to the confusion matrix
`int`	`classCount(Integer clazz)` Returns the number of times the given label has actually occurred
`String`	`confusionToString()` Get a String representation of the confusion matrix
`void`	`eval(org.nd4j.linalg.api.ndarray.INDArray realOutcomes, org.nd4j.linalg.api.ndarray.INDArray guesses)` Collects statistics on the real outcomes vs the guesses.
`void`	`eval(org.nd4j.linalg.api.ndarray.INDArray trueLabels, org.nd4j.linalg.api.ndarray.INDArray input, ComputationGraph network)` Evaluate the output using the given true labels, the input to the multi layer network and the multi layer network to use for evaluation
`void`	`eval(org.nd4j.linalg.api.ndarray.INDArray trueLabels, org.nd4j.linalg.api.ndarray.INDArray input, MultiLayerNetwork network)` Evaluate the output using the given true labels, the input to the multi layer network and the multi layer network to use for evaluation
`void`	`eval(int predictedIdx, int actualIdx)` Evaluate a single prediction (one prediction at a time)
`void`	`evalTimeSeries(org.nd4j.linalg.api.ndarray.INDArray labels, org.nd4j.linalg.api.ndarray.INDArray predicted)` Convenience method for evaluation of time series.
`void`	`evalTimeSeries(org.nd4j.linalg.api.ndarray.INDArray labels, org.nd4j.linalg.api.ndarray.INDArray predicted, org.nd4j.linalg.api.ndarray.INDArray outputMask)` Evaluate a time series, whether the output is masked usind a masking array.
`double`	`f1()` TP: true positive FP: False Positive FN: False Negative F1 score: 2 * TP / (2TP + FP + FN)
`double`	`f1(Integer classLabel)` Calculate f1 score for a given class
`double`	`falseAlarmRate()` False Alarm Rate (FAR) reflects rate of misclassified to classified records http://ro.ecu.edu.au/cgi/viewcontent.cgi?article=1058&context=isw
`double`	`falseNegativeRate()` False negative rate based on guesses so far Takes into account all known classes and outputs average fnr across all of them
`double`	`falseNegativeRate(Integer classLabel)` Returns the false negative rate for a given label
`double`	`falseNegativeRate(Integer classLabel, double edgeCase)` Returns the false negative rate for a given label
`Map<Integer,Integer>`	`falseNegatives()` False negatives: correctly rejected
`double`	`falsePositiveRate()` False positive rate based on guesses so far Takes into account all known classes and outputs average fpr across all of them
`double`	`falsePositiveRate(Integer classLabel)` Returns the false positive rate for a given label
`double`	`falsePositiveRate(Integer classLabel, double edgeCase)` Returns the false positive rate for a given label
`Map<Integer,Integer>`	`falsePositives()` False positive: wrong guess
`String`	`getClassLabel(Integer clazz)`
`ConfusionMatrix<Integer>`	`getConfusionMatrix()` Returns the confusion matrix variable
`int`	`getNumRowCounter()`
`void`	`incrementFalseNegatives(Integer classLabel)`
`void`	`incrementFalsePositives(Integer classLabel)`
`void`	`incrementTrueNegatives(Integer classLabel)`
`void`	`incrementTruePositives(Integer classLabel)`
`void`	`merge(Evaluation other)` Merge the other evaluation object into this one.
`Map<Integer,Integer>`	`negative()` Total negatives true negatives + false negatives
`Map<Integer,Integer>`	`positive()` Returns all of the positive guesses: true positive + false negative
`double`	`precision()` Precision based on guesses so far Takes into account all known classes and outputs average precision across all of them
`double`	`precision(Integer classLabel)` Returns the precision for a given label
`double`	`precision(Integer classLabel, double edgeCase)` Returns the precision for a given label
`double`	`recall()` Recall based on guesses so far Takes into account all known classes and outputs average recall across all of them
`double`	`recall(Integer classLabel)` Returns the recall for a given label
`double`	`recall(Integer classLabel, double edgeCase)` Returns the recall for a given label
`String`	`stats()`
`String`	`stats(boolean suppressWarnings)` Method to obtain the classification report as a String
`Map<Integer,Integer>`	`trueNegatives()` True negatives: correctly rejected
`Map<Integer,Integer>`	`truePositives()` True positives: correctly rejected

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - truePositives
```
protected Counter<Integer> truePositives
```
  - falsePositives
```
protected Counter<Integer> falsePositives
```
  - trueNegatives
```
protected Counter<Integer> trueNegatives
```
  - falseNegatives
```
protected Counter<Integer> falseNegatives
```
  - confusion
```
protected ConfusionMatrix<Integer> confusion
```
  - numRowCounter
```
protected int numRowCounter
```
  - labelsList
```
protected List<String> labelsList
```
  - log
```
protected static org.slf4j.Logger log
```
  - DEFAULT_EDGE_VALUE
```
protected static final double DEFAULT_EDGE_VALUE
```
    See Also:
    
    Constant Field Values
- Constructor Detail
  - Evaluation
```
public Evaluation()
```
  - Evaluation
```
public Evaluation(int numClasses)
```
    The number of classes to account for in the evaluation
    
    Parameters:
    
    numClasses - the number of classes to account for in the evaluation
  - Evaluation
```
public Evaluation(List<String> labels)
```
    The labels to include with the evaluation. This constructor can be used for generating labeled output rather than just numbers for the labels
    
    Parameters:
    
    labels - the labels to use for the output
  - Evaluation
```
public Evaluation(Map<Integer,String> labels)
```
    Use a map to generate labels Pass in a label index with the actual label you want to use for output
    
    Parameters:
    
    labels - a map of label index to label value
- Method Detail
  - eval
```
public void eval(org.nd4j.linalg.api.ndarray.INDArray trueLabels,
                 org.nd4j.linalg.api.ndarray.INDArray input,
                 ComputationGraph network)
```
    Evaluate the output using the given true labels, the input to the multi layer network and the multi layer network to use for evaluation
    
    Parameters:
    
    trueLabels - the labels to ise
    
    input - the input to the network to use for evaluation
    
    network - the network to use for output
  - eval
```
public void eval(org.nd4j.linalg.api.ndarray.INDArray trueLabels,
                 org.nd4j.linalg.api.ndarray.INDArray input,
                 MultiLayerNetwork network)
```
    Evaluate the output using the given true labels, the input to the multi layer network and the multi layer network to use for evaluation
    
    Parameters:
    
    trueLabels - the labels to ise
    
    input - the input to the network to use for evaluation
    
    network - the network to use for output
  - eval
```
public void eval(org.nd4j.linalg.api.ndarray.INDArray realOutcomes,
                 org.nd4j.linalg.api.ndarray.INDArray guesses)
```
    Collects statistics on the real outcomes vs the guesses. This is for logistic outcome matrices.
    Note that an IllegalArgumentException is thrown if the two passed in matrices aren't the same length.
    
    Parameters:
    
    realOutcomes - the real outcomes (labels - usually binary)
    
    guesses - the guesses/prediction (usually a probability vector)
  - evalTimeSeries
```
public void evalTimeSeries(org.nd4j.linalg.api.ndarray.INDArray labels,
                           org.nd4j.linalg.api.ndarray.INDArray predicted)
```
    Convenience method for evaluation of time series. Reshapes time series (3d) to 2d, then calls eval
    
    See Also:
    
    eval(INDArray, INDArray)
  - evalTimeSeries
```
public void evalTimeSeries(org.nd4j.linalg.api.ndarray.INDArray labels,
                           org.nd4j.linalg.api.ndarray.INDArray predicted,
                           org.nd4j.linalg.api.ndarray.INDArray outputMask)
```
    Evaluate a time series, whether the output is masked usind a masking array. That is, the mask array specified whether the output at a given time step is actually present, or whether it is just padding.
    For example, for N examples, nOut output size, and T time series length: labels and predicted will have shape [N,nOut,T], and outputMask will have shape [N,T].
    
    See Also:
    
    evalTimeSeries(INDArray, INDArray)
  - eval
```
public void eval(int predictedIdx,
                 int actualIdx)
```
    Evaluate a single prediction (one prediction at a time)
    
    Parameters:
    
    predictedIdx - Index of class predicted by the network
    
    actualIdx - Index of actual class
  - stats
```
public String stats()
```
  - stats
```
public String stats(boolean suppressWarnings)
```
    Method to obtain the classification report as a String
    
    Parameters:
    
    suppressWarnings - whether or not to output warnings related to the evaluation results
    
    Returns:
    
    A (multi-line) String with accuracy, precision, recall, f1 score etc
  - precision
```
public double precision(Integer classLabel)
```
    Returns the precision for a given label
    
    Parameters:
    
    classLabel - the label
    
    Returns:
    
    the precision for the label
  - precision
```
public double precision(Integer classLabel,
                        double edgeCase)
```
    Returns the precision for a given label
    
    Parameters:
    
    classLabel - the label
    
    edgeCase - What to output in case of 0/0
    
    Returns:
    
    the precision for the label
  - precision
```
public double precision()
```
    Precision based on guesses so far Takes into account all known classes and outputs average precision across all of them
    
    Returns:
    
    the total precision based on guesses so far
  - recall
```
public double recall(Integer classLabel)
```
    Returns the recall for a given label
    
    Parameters:
    
    classLabel - the label
    
    Returns:
    
    Recall rate as a double
  - recall
```
public double recall(Integer classLabel,
                     double edgeCase)
```
    Returns the recall for a given label
    
    Parameters:
    
    classLabel - the label
    
    edgeCase - What to output in case of 0/0
    
    Returns:
    
    Recall rate as a double
  - recall
```
public double recall()
```
    Recall based on guesses so far Takes into account all known classes and outputs average recall across all of them
    
    Returns:
    
    the recall for the outcomes
  - falsePositiveRate
```
public double falsePositiveRate(Integer classLabel)
```
    Returns the false positive rate for a given label
    
    Parameters:
    
    classLabel - the label
    
    Returns:
    
    fpr as a double
  - falsePositiveRate
```
public double falsePositiveRate(Integer classLabel,
                                double edgeCase)
```
    Returns the false positive rate for a given label
    
    Parameters:
    
    classLabel - the label
    
    edgeCase - What to output in case of 0/0
    
    Returns:
    
    fpr as a double
  - falsePositiveRate
```
public double falsePositiveRate()
```
    False positive rate based on guesses so far Takes into account all known classes and outputs average fpr across all of them
    
    Returns:
    
    the fpr for the outcomes
  - falseNegativeRate
```
public double falseNegativeRate(Integer classLabel)
```
    Returns the false negative rate for a given label
    
    Parameters:
    
    classLabel - the label
    
    Returns:
    
    fnr as a double
  - falseNegativeRate
```
public double falseNegativeRate(Integer classLabel,
                                double edgeCase)
```
    Returns the false negative rate for a given label
    
    Parameters:
    
    classLabel - the label
    
    edgeCase - What to output in case of 0/0
    
    Returns:
    
    fnr as a double
  - falseNegativeRate
```
public double falseNegativeRate()
```
    False negative rate based on guesses so far Takes into account all known classes and outputs average fnr across all of them
    
    Returns:
    
    the fnr for the outcomes
  - falseAlarmRate
```
public double falseAlarmRate()
```
    False Alarm Rate (FAR) reflects rate of misclassified to classified records http://ro.ecu.edu.au/cgi/viewcontent.cgi?article=1058&context=isw
    
    Returns:
    
    the fpr for the outcomes
  - f1
```
public double f1(Integer classLabel)
```
    Calculate f1 score for a given class
    
    Parameters:
    
    classLabel - the label to calculate f1 for
    
    Returns:
    
    the f1 score for the given label
  - f1
```
public double f1()
```
    TP: true positive FP: False Positive FN: False Negative F1 score: 2 * TP / (2TP + FP + FN)
    
    Returns:
    
    the f1 score or harmonic mean based on current guesses
  - accuracy
```
public double accuracy()
```
    Accuracy: (TP + TN) / (P + N)
    
    Returns:
    
    the accuracy of the guesses so far
  - truePositives
```
public Map<Integer,Integer> truePositives()
```
    True positives: correctly rejected
    
    Returns:
    
    the total true positives so far
  - trueNegatives
```
public Map<Integer,Integer> trueNegatives()
```
    True negatives: correctly rejected
    
    Returns:
    
    the total true negatives so far
  - falsePositives
```
public Map<Integer,Integer> falsePositives()
```
    False positive: wrong guess
    
    Returns:
    
    the count of the false positives
  - falseNegatives
```
public Map<Integer,Integer> falseNegatives()
```
    False negatives: correctly rejected
    
    Returns:
    
    the total false negatives so far
  - negative
```
public Map<Integer,Integer> negative()
```
    Total negatives true negatives + false negatives
    
    Returns:
    
    the overall negative count
  - positive
```
public Map<Integer,Integer> positive()
```
    Returns all of the positive guesses: true positive + false negative
  - incrementTruePositives
```
public void incrementTruePositives(Integer classLabel)
```
  - incrementTrueNegatives
```
public void incrementTrueNegatives(Integer classLabel)
```
  - incrementFalseNegatives
```
public void incrementFalseNegatives(Integer classLabel)
```
  - incrementFalsePositives
```
public void incrementFalsePositives(Integer classLabel)
```
  - addToConfusion
```
public void addToConfusion(Integer real,
                           Integer guess)
```
    Adds to the confusion matrix
    
    Parameters:
    
    real - the actual guess
    
    guess - the system guess
  - classCount
```
public int classCount(Integer clazz)
```
    Returns the number of times the given label has actually occurred
    
    Parameters:
    
    clazz - the label
    
    Returns:
    
    the number of times the label actually occurred
  - getNumRowCounter
```
public int getNumRowCounter()
```
  - getClassLabel
```
public String getClassLabel(Integer clazz)
```
  - getConfusionMatrix
```
public ConfusionMatrix<Integer> getConfusionMatrix()
```
    Returns the confusion matrix variable
    
    Returns:
    
    confusion matrix variable for this evaluation
  - merge
```
public void merge(Evaluation other)
```
    Merge the other evaluation object into this one. The result is that this Evaluation instance contains the counts etc from both
    
    Parameters:
    
    other - Evaluation object to merge into this one.
  - confusionToString
```
public String confusionToString()
```
    Get a String representation of the confusion matrix

Class Evaluation

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

truePositives

falsePositives

trueNegatives

falseNegatives

confusion

numRowCounter

labelsList

log

DEFAULT_EDGE_VALUE

Constructor Detail

Evaluation

Evaluation

Evaluation

Evaluation

Method Detail

eval

eval

eval

evalTimeSeries

evalTimeSeries

eval

stats

stats

precision

precision

precision

recall

recall

recall

falsePositiveRate

falsePositiveRate

falsePositiveRate

falseNegativeRate

falseNegativeRate

falseNegativeRate

falseAlarmRate

f1

f1

accuracy

truePositives

trueNegatives

falsePositives

falseNegatives

negative

positive

incrementTruePositives

incrementTrueNegatives

incrementFalseNegatives

incrementFalsePositives

addToConfusion

classCount

getNumRowCounter

getClassLabel

getConfusionMatrix

merge

confusionToString