smile.validation

Class Validation

java.lang.Object

smile.validation.Validation

public class Validation
extends Object

A utility class for validating predictive models on test data.

Author:

Haifeng

Constructor Summary

Constructors

Constructor and Description
Validation()

Method Summary

Methods

Modifier and Type	Method and Description
static <T> double[]	bootstrap(int k, ClassifierTrainer<T> trainer, T[] x, int[] y) Bootstrap accuracy estimation of a classification model.
static <T> double[]	bootstrap(int k, ClassifierTrainer<T> trainer, T[] x, int[] y, ClassificationMeasure measure) Bootstrap performance estimation of a classification model.
static <T> double[][]	bootstrap(int k, ClassifierTrainer<T> trainer, T[] x, int[] y, ClassificationMeasure[] measures) Bootstrap performance estimation of a classification model.
static <T> double[]	bootstrap(int k, RegressionTrainer<T> trainer, T[] x, double[] y) Bootstrap RMSE estimation of a regression model.
static <T> double[]	bootstrap(int k, RegressionTrainer<T> trainer, T[] x, double[] y, RegressionMeasure measure) Bootstrap performance estimation of a regression model.
static <T> double[][]	bootstrap(int k, RegressionTrainer<T> trainer, T[] x, double[] y, RegressionMeasure[] measures) Bootstrap performance estimation of a regression model.
static <T> double	cv(int k, ClassifierTrainer<T> trainer, T[] x, int[] y) Cross validation of a classification model.
static <T> double	cv(int k, ClassifierTrainer<T> trainer, T[] x, int[] y, ClassificationMeasure measure) Cross validation of a classification model.
static <T> double[]	cv(int k, ClassifierTrainer<T> trainer, T[] x, int[] y, ClassificationMeasure[] measures) Cross validation of a classification model.
static <T> double	cv(int k, RegressionTrainer<T> trainer, T[] x, double[] y) Cross validation of a regression model.
static <T> double	cv(int k, RegressionTrainer<T> trainer, T[] x, double[] y, RegressionMeasure measure) Cross validation of a regression model.
static <T> double[]	cv(int k, RegressionTrainer<T> trainer, T[] x, double[] y, RegressionMeasure[] measures) Cross validation of a regression model.
static <T> double	loocv(ClassifierTrainer<T> trainer, T[] x, int[] y) Leave-one-out cross validation of a classification model.
static <T> double	loocv(ClassifierTrainer<T> trainer, T[] x, int[] y, ClassificationMeasure measure) Leave-one-out cross validation of a classification model.
static <T> double[]	loocv(ClassifierTrainer<T> trainer, T[] x, int[] y, ClassificationMeasure[] measures) Leave-one-out cross validation of a classification model.
static <T> double	loocv(RegressionTrainer<T> trainer, T[] x, double[] y) Leave-one-out cross validation of a regression model.
static <T> double	loocv(RegressionTrainer<T> trainer, T[] x, double[] y, RegressionMeasure measure) Leave-one-out cross validation of a regression model.
static <T> double[]	loocv(RegressionTrainer<T> trainer, T[] x, double[] y, RegressionMeasure[] measures) Leave-one-out cross validation of a regression model.
static <T> double	test(Classifier<T> classifier, T[] x, int[] y) Tests a classifier on a validation set.
static <T> double	test(Classifier<T> classifier, T[] x, int[] y, ClassificationMeasure measure) Tests a classifier on a validation set.
static <T> double[]	test(Classifier<T> classifier, T[] x, int[] y, ClassificationMeasure[] measures) Tests a classifier on a validation set.
static <T> double	test(Regression<T> regression, T[] x, double[] y) Tests a regression model on a validation set.
static <T> double	test(Regression<T> regression, T[] x, double[] y, RegressionMeasure measure) Tests a regression model on a validation set.
static <T> double[]	test(Regression<T> regression, T[] x, double[] y, RegressionMeasure[] measures) Tests a regression model on a validation set.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Validation

public Validation()

Method Detail

test

public static <T> double test(Classifier<T> classifier,
              T[] x,
              int[] y)

Tests a classifier on a validation set.

Type Parameters:

T - the data type of input objects.

Parameters:

classifier - a trained classifier to be tested.

x - the test data set.

y - the test data labels.

Returns:

the accuracy on the test dataset

test

public static <T> double test(Regression<T> regression,
              T[] x,
              double[] y)

Tests a regression model on a validation set.

Type Parameters:

T - the data type of input objects.

Parameters:

regression - a trained regression model to be tested.

x - the test data set.

y - the test data response values.

Returns:

root mean squared error

test

public static <T> double test(Classifier<T> classifier,
              T[] x,
              int[] y,
              ClassificationMeasure measure)

Tests a classifier on a validation set.

Type Parameters:

T - the data type of input objects.

Parameters:

classifier - a trained classifier to be tested.

x - the test data set.

y - the test data labels.

measure - the performance measures of classification.

Returns:

the test results with the same size of order of measures

test

public static <T> double[] test(Classifier<T> classifier,
                T[] x,
                int[] y,
                ClassificationMeasure[] measures)

Tests a classifier on a validation set.

Type Parameters:

T - the data type of input objects.

Parameters:

classifier - a trained classifier to be tested.

x - the test data set.

y - the test data labels.

measures - the performance measures of classification.

Returns:

the test results with the same size of order of measures

test

public static <T> double test(Regression<T> regression,
              T[] x,
              double[] y,
              RegressionMeasure measure)

Tests a regression model on a validation set.

Type Parameters:

T - the data type of input objects.

Parameters:

regression - a trained regression model to be tested.

x - the test data set.

y - the test data response values.

measure - the performance measure of regression.

Returns:

the test results with the same size of order of measures

test

public static <T> double[] test(Regression<T> regression,
                T[] x,
                double[] y,
                RegressionMeasure[] measures)

Tests a regression model on a validation set.

Type Parameters:

T - the data type of input objects.

Parameters:

regression - a trained regression model to be tested.

x - the test data set.

y - the test data response values.

measures - the performance measures of regression.

Returns:

the test results with the same size of order of measures

loocv

public static <T> double loocv(ClassifierTrainer<T> trainer,
               T[] x,
               int[] y)

Leave-one-out cross validation of a classification model.

Type Parameters:

T - the data type of input objects.

Parameters:

trainer - a classifier trainer that is properly parameterized.

x - the test data set.

y - the test data labels.

Returns:

the accuracy on test dataset

loocv

public static <T> double loocv(RegressionTrainer<T> trainer,
               T[] x,
               double[] y)

Leave-one-out cross validation of a regression model.

Type Parameters:

T - the data type of input objects.

Parameters:

trainer - a regression model trainer that is properly parameterized.

x - the test data set.

y - the test data response values.

Returns:

root mean squared error

loocv

public static <T> double loocv(ClassifierTrainer<T> trainer,
               T[] x,
               int[] y,
               ClassificationMeasure measure)

Leave-one-out cross validation of a classification model.

Type Parameters:

T - the data type of input objects.

Parameters:

trainer - a classifier trainer that is properly parameterized.

x - the test data set.

y - the test data labels.

measure - the performance measure of classification.

Returns:

the test results with the same size of order of measures

loocv

public static <T> double[] loocv(ClassifierTrainer<T> trainer,
                 T[] x,
                 int[] y,
                 ClassificationMeasure[] measures)

Leave-one-out cross validation of a classification model.

Type Parameters:

T - the data type of input objects.

Parameters:

trainer - a classifier trainer that is properly parameterized.

x - the test data set.

y - the test data labels.

measures - the performance measures of classification.

Returns:

the test results with the same size of order of measures

loocv

public static <T> double loocv(RegressionTrainer<T> trainer,
               T[] x,
               double[] y,
               RegressionMeasure measure)

Leave-one-out cross validation of a regression model.

Type Parameters:

T - the data type of input objects.

Parameters:

trainer - a regression model trainer that is properly parameterized.

x - the test data set.

y - the test data response values.

measure - the performance measure of regression.

Returns:

the test results with the same size of order of measures

loocv

public static <T> double[] loocv(RegressionTrainer<T> trainer,
                 T[] x,
                 double[] y,
                 RegressionMeasure[] measures)

Leave-one-out cross validation of a regression model.

Type Parameters:

T - the data type of input objects.

Parameters:

trainer - a regression model trainer that is properly parameterized.

x - the test data set.

y - the test data response values.

measures - the performance measures of regression.

Returns:

the test results with the same size of order of measures

cv

public static <T> double cv(int k,
            ClassifierTrainer<T> trainer,
            T[] x,
            int[] y)

Cross validation of a classification model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold cross validation.

trainer - a classifier trainer that is properly parameterized.

x - the test data set.

y - the test data labels.

Returns:

the accuracy on test dataset

cv

public static <T> double cv(int k,
            RegressionTrainer<T> trainer,
            T[] x,
            double[] y)

Cross validation of a regression model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold cross validation.

trainer - a regression model trainer that is properly parameterized.

x - the test data set.

y - the test data response values.

Returns:

root mean squared error

cv

public static <T> double cv(int k,
            ClassifierTrainer<T> trainer,
            T[] x,
            int[] y,
            ClassificationMeasure measure)

Cross validation of a classification model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold cross validation.

trainer - a classifier trainer that is properly parameterized.

x - the test data set.

y - the test data labels.

measure - the performance measure of classification.

Returns:

the test results with the same size of order of measures

cv

public static <T> double[] cv(int k,
              ClassifierTrainer<T> trainer,
              T[] x,
              int[] y,
              ClassificationMeasure[] measures)

Cross validation of a classification model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold cross validation.

trainer - a classifier trainer that is properly parameterized.

x - the test data set.

y - the test data labels.

measures - the performance measures of classification.

Returns:

the test results with the same size of order of measures

cv

public static <T> double cv(int k,
            RegressionTrainer<T> trainer,
            T[] x,
            double[] y,
            RegressionMeasure measure)

Cross validation of a regression model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold cross validation.

trainer - a regression model trainer that is properly parameterized.

x - the test data set.

y - the test data response values.

measure - the performance measure of regression.

Returns:

the test results with the same size of order of measures

cv

public static <T> double[] cv(int k,
              RegressionTrainer<T> trainer,
              T[] x,
              double[] y,
              RegressionMeasure[] measures)

Cross validation of a regression model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold cross validation.

trainer - a regression model trainer that is properly parameterized.

x - the test data set.

y - the test data response values.

measures - the performance measures of regression.

Returns:

the test results with the same size of order of measures

bootstrap

public static <T> double[] bootstrap(int k,
                     ClassifierTrainer<T> trainer,
                     T[] x,
                     int[] y)

Bootstrap accuracy estimation of a classification model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-round bootstrap estimation.

trainer - a classifier trainer that is properly parameterized.

x - the test data set.

y - the test data labels.

Returns:

the k-round accuracies

bootstrap

public static <T> double[] bootstrap(int k,
                     RegressionTrainer<T> trainer,
                     T[] x,
                     double[] y)

Bootstrap RMSE estimation of a regression model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-round bootstrap estimation.

trainer - a regression model trainer that is properly parameterized.

x - the test data set.

y - the test data response values.

Returns:

the k-round root mean squared errors

bootstrap

public static <T> double[] bootstrap(int k,
                     ClassifierTrainer<T> trainer,
                     T[] x,
                     int[] y,
                     ClassificationMeasure measure)

Bootstrap performance estimation of a classification model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold bootstrap estimation.

trainer - a classifier trainer that is properly parameterized.

x - the test data set.

y - the test data labels.

measure - the performance measures of classification.

Returns:

k-by-m test result matrix, where k is the number of bootstrap samples and m is the number of performance measures.

bootstrap

public static <T> double[][] bootstrap(int k,
                       ClassifierTrainer<T> trainer,
                       T[] x,
                       int[] y,
                       ClassificationMeasure[] measures)

Bootstrap performance estimation of a classification model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold bootstrap estimation.

trainer - a classifier trainer that is properly parameterized.

x - the test data set.

y - the test data labels.

measures - the performance measures of classification.

Returns:

k-by-m test result matrix, where k is the number of bootstrap samples and m is the number of performance measures.

bootstrap

public static <T> double[] bootstrap(int k,
                     RegressionTrainer<T> trainer,
                     T[] x,
                     double[] y,
                     RegressionMeasure measure)

Bootstrap performance estimation of a regression model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold bootstrap estimation.

trainer - a regression model trainer that is properly parameterized.

x - the test data set.

y - the test data response values.

measure - the performance measure of regression.

Returns:

k-by-m test result matrix, where k is the number of bootstrap samples and m is the number of performance measures.

bootstrap

public static <T> double[][] bootstrap(int k,
                       RegressionTrainer<T> trainer,
                       T[] x,
                       double[] y,
                       RegressionMeasure[] measures)

Bootstrap performance estimation of a regression model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold bootstrap estimation.

trainer - a regression model trainer that is properly parameterized.

x - the test data set.

y - the test data response values.

measures - the performance measures of regression.

Returns:

k-by-m test result matrix, where k is the number of bootstrap samples and m is the number of performance measures.