org.deeplearning4j.eval

Class ROC

java.lang.Object

org.deeplearning4j.eval.ROC

All Implemented Interfaces:

Serializable

public class ROC
extends Object
implements Serializable

ROC (Receiver Operating Characteristic) for binary classifiers, using the specified number of threshold steps.

Some ROC implementations will automatically calculate the threshold points based on the data set to give a 'smoother' ROC curve (or optimal cut points for diagnostic purposes). This implementation currently uses fixed steps of size 1.0 / thresholdSteps, as this allows easy implementation for batched and distributed evaluation scenarios (where the full data set is not available in memory on any one machine at once).

The data is assumed to be binary classification - nColumns == 1 (single binary output variable) or nColumns == 2 (probability distribution over 2 classes, with column 1 being values for 'positive' examples)

Author:

Alex Black

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ROC.ROCValue

Constructor Summary

Constructors

Constructor and Description
ROC(int thresholdSteps)

Method Summary

Modifier and Type	Method and Description
double	calculateAUC() Calculate the AUC - Area Under Curve Utilizes trapezoidal integration internally
void	eval(org.nd4j.linalg.api.ndarray.INDArray labels, org.nd4j.linalg.api.ndarray.INDArray predictions) Evaluate (collect statistics for) the given minibatch of data.
void	evalTimeSeries(org.nd4j.linalg.api.ndarray.INDArray labels, org.nd4j.linalg.api.ndarray.INDArray predictions) Evaluate (collect statistics for) the given minibatch of data time series (3d) data, with no mask array
void	evalTimeSeries(org.nd4j.linalg.api.ndarray.INDArray labels, org.nd4j.linalg.api.ndarray.INDArray predicted, org.nd4j.linalg.api.ndarray.INDArray outputMask) Evaluate (collect statistics for) the given minibatch of data time series (3d) data, with optional (nullable) output mask array.
List<ROC.ROCValue>	getResults() Get the ROC curve, as a set of points
double[][]	getResultsAsArray() Get the ROC curve, as a set of (falsePositive, truePositive) points

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ROC

public ROC(int thresholdSteps)

Parameters:

thresholdSteps - Number of threshold steps to use for the ROC calculation

Method Detail

eval

public void eval(org.nd4j.linalg.api.ndarray.INDArray labels,
                 org.nd4j.linalg.api.ndarray.INDArray predictions)

Evaluate (collect statistics for) the given minibatch of data. For time series (3 dimensions) use evalTimeSeries(INDArray, INDArray) or evalTimeSeries(INDArray, INDArray, INDArray)

Parameters:

labels - Labels / true outcomes

predictions - Predictions

evalTimeSeries

public void evalTimeSeries(org.nd4j.linalg.api.ndarray.INDArray labels,
                           org.nd4j.linalg.api.ndarray.INDArray predictions)

Evaluate (collect statistics for) the given minibatch of data time series (3d) data, with no mask array

Parameters:

labels - Labels / true outcomes

predictions - Predictions

evalTimeSeries

public void evalTimeSeries(org.nd4j.linalg.api.ndarray.INDArray labels,
                           org.nd4j.linalg.api.ndarray.INDArray predicted,
                           org.nd4j.linalg.api.ndarray.INDArray outputMask)

Evaluate (collect statistics for) the given minibatch of data time series (3d) data, with optional (nullable) output mask array. labels/predictions arrays should be 3d time series ([minibatchSize, 1 or 2, timeSeriesLength]), and mask array should be a 2d array with shape [minibatchSize, timeSeriesLength] with values 0 or 1

Parameters:

labels - Labels / true outcomes

predicted - Predictions

getResults

public List<ROC.ROCValue> getResults()

Get the ROC curve, as a set of points

Returns:

ROC curve, as a list of points

getResultsAsArray

public double[][] getResultsAsArray()

Get the ROC curve, as a set of (falsePositive, truePositive) points

Returns a 2d array of {falsePositive, truePositive values}.
Size is [2][thresholdSteps], with out[0][.] being false positives, and out[1][.] being true positives

Returns:

ROC curve as double[][]

calculateAUC

public double calculateAUC()

Calculate the AUC - Area Under Curve
Utilizes trapezoidal integration internally

Returns:

AUC