smile.validation

Class Validation

java.lang.Object

smile.validation.Validation

public class Validation
extends java.lang.Object

A utility class for validating predictive models on test data.

Constructor Summary

Constructors

Constructor and Description
Validation()

Method Summary

Modifier and Type	Method and Description
static <T> double[]	bootstrap(int k, ClassifierTrainer<T> trainer, T[] x, int[] y) Bootstrap accuracy estimation of a classification model.
static <T> double[]	bootstrap(int k, ClassifierTrainer<T> trainer, T[] x, int[] y, ClassificationMeasure measure) Bootstrap performance estimation of a classification model.
static <T> double[][]	bootstrap(int k, ClassifierTrainer<T> trainer, T[] x, int[] y, ClassificationMeasure[] measures) Bootstrap performance estimation of a classification model.
static <T> double[]	bootstrap(int k, RegressionTrainer<T> trainer, T[] x, double[] y) Bootstrap RMSE estimation of a regression model.
static <T> double[]	bootstrap(int k, RegressionTrainer<T> trainer, T[] x, double[] y, RegressionMeasure measure) Bootstrap performance estimation of a regression model.
static <T> double[][]	bootstrap(int k, RegressionTrainer<T> trainer, T[] x, double[] y, RegressionMeasure[] measures) Bootstrap performance estimation of a regression model.
static <T> double	cv(int k, ClassifierTrainer<T> trainer, T[] x, int[] y) Cross validation of a classification model.
static <T> double	cv(int k, ClassifierTrainer<T> trainer, T[] x, int[] y, ClassificationMeasure measure) Cross validation of a classification model.
static <T> double[]	cv(int k, ClassifierTrainer<T> trainer, T[] x, int[] y, ClassificationMeasure[] measures) Cross validation of a classification model.
static <T> double	cv(int k, RegressionTrainer<T> trainer, T[] x, double[] y) Cross validation of a regression model.
static <T> double	cv(int k, RegressionTrainer<T> trainer, T[] x, double[] y, RegressionMeasure measure) Cross validation of a regression model.
static <T> double[]	cv(int k, RegressionTrainer<T> trainer, T[] x, double[] y, RegressionMeasure[] measures) Cross validation of a regression model.
static <T> double	loocv(ClassifierTrainer<T> trainer, T[] x, int[] y) Leave-one-out cross validation of a classification model.
static <T> double	loocv(ClassifierTrainer<T> trainer, T[] x, int[] y, ClassificationMeasure measure) Leave-one-out cross validation of a classification model.
static <T> double[]	loocv(ClassifierTrainer<T> trainer, T[] x, int[] y, ClassificationMeasure[] measures) Leave-one-out cross validation of a classification model.
static <T> double	loocv(RegressionTrainer<T> trainer, T[] x, double[] y) Leave-one-out cross validation of a regression model.
static <T> double	loocv(RegressionTrainer<T> trainer, T[] x, double[] y, RegressionMeasure measure) Leave-one-out cross validation of a regression model.
static <T> double[]	loocv(RegressionTrainer<T> trainer, T[] x, double[] y, RegressionMeasure[] measures) Leave-one-out cross validation of a regression model.
static <T> double	test(Classifier<T> classifier, T[] x, int[] y) Tests a classifier on a validation set.
static <T> double	test(Classifier<T> classifier, T[] x, int[] y, ClassificationMeasure measure) Tests a classifier on a validation set.
static <T> double[]	test(Classifier<T> classifier, T[] x, int[] y, ClassificationMeasure[] measures) Tests a classifier on a validation set.
static <T> double	test(Regression<T> regression, T[] x, double[] y) Tests a regression model on a validation set.
static <T> double	test(Regression<T> regression, T[] x, double[] y, RegressionMeasure measure) Tests a regression model on a validation set.
static <T> double[]	test(Regression<T> regression, T[] x, double[] y, RegressionMeasure[] measures) Tests a regression model on a validation set.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Validation

public Validation()

Method Detail

test

public static <T> double test(Classifier<T> classifier,
                              T[] x,
                              int[] y)

Tests a classifier on a validation set.

Type Parameters:

T - the data type of input objects.

Parameters:

classifier - a trained classifier to be tested.

x - the test data set.

y - the test data labels.

Returns:

the accuracy on the test dataset

test

public static <T> double test(Regression<T> regression,
                              T[] x,
                              double[] y)

Tests a regression model on a validation set.

Type Parameters:

T - the data type of input objects.

Parameters:

regression - a trained regression model to be tested.

x - the test data set.

y - the test data response values.

Returns:

root mean squared error

test

public static <T> double test(Classifier<T> classifier,
                              T[] x,
                              int[] y,
                              ClassificationMeasure measure)

Tests a classifier on a validation set.

Type Parameters:

T - the data type of input objects.

Parameters:

classifier - a trained classifier to be tested.

x - the test data set.

y - the test data labels.

measure - the performance measures of classification.

Returns:

the test results with the same size of order of measures

test

public static <T> double[] test(Classifier<T> classifier,
                                T[] x,
                                int[] y,
                                ClassificationMeasure[] measures)

Tests a classifier on a validation set.

Type Parameters:

T - the data type of input objects.

Parameters:

classifier - a trained classifier to be tested.

x - the test data set.

y - the test data labels.

measures - the performance measures of classification.

Returns:

the test results with the same size of order of measures

test

public static <T> double test(Regression<T> regression,
                              T[] x,
                              double[] y,
                              RegressionMeasure measure)

Tests a regression model on a validation set.

Type Parameters:

T - the data type of input objects.

Parameters:

regression - a trained regression model to be tested.

x - the test data set.

y - the test data response values.

measure - the performance measure of regression.

Returns:

the test results with the same size of order of measures

test

public static <T> double[] test(Regression<T> regression,
                                T[] x,
                                double[] y,
                                RegressionMeasure[] measures)

Tests a regression model on a validation set.

Type Parameters:

T - the data type of input objects.

Parameters:

regression - a trained regression model to be tested.

x - the test data set.

y - the test data response values.

measures - the performance measures of regression.

Returns:

the test results with the same size of order of measures

loocv

public static <T> double loocv(ClassifierTrainer<T> trainer,
                               T[] x,
                               int[] y)

Leave-one-out cross validation of a classification model.

Type Parameters:

T - the data type of input objects.

Parameters:

trainer - a classifier trainer that is properly parameterized.

x - the test data set.

y - the test data labels.

Returns:

the accuracy on test dataset

loocv

public static <T> double loocv(RegressionTrainer<T> trainer,
                               T[] x,
                               double[] y)

Leave-one-out cross validation of a regression model.

Type Parameters:

T - the data type of input objects.

Parameters:

trainer - a regression model trainer that is properly parameterized.

x - the test data set.

y - the test data response values.

Returns:

root mean squared error

loocv

public static <T> double loocv(ClassifierTrainer<T> trainer,
                               T[] x,
                               int[] y,
                               ClassificationMeasure measure)

Leave-one-out cross validation of a classification model.

Type Parameters:

T - the data type of input objects.

Parameters:

trainer - a classifier trainer that is properly parameterized.

x - the test data set.

y - the test data labels.

measure - the performance measure of classification.

Returns:

the test results with the same size of order of measures

loocv

public static <T> double[] loocv(ClassifierTrainer<T> trainer,
                                 T[] x,
                                 int[] y,
                                 ClassificationMeasure[] measures)

Leave-one-out cross validation of a classification model.

Type Parameters:

T - the data type of input objects.

Parameters:

trainer - a classifier trainer that is properly parameterized.

x - the test data set.

y - the test data labels.

measures - the performance measures of classification.

Returns:

the test results with the same size of order of measures

loocv

public static <T> double loocv(RegressionTrainer<T> trainer,
                               T[] x,
                               double[] y,
                               RegressionMeasure measure)

Leave-one-out cross validation of a regression model.

Type Parameters:

T - the data type of input objects.

Parameters:

trainer - a regression model trainer that is properly parameterized.

x - the test data set.

y - the test data response values.

measure - the performance measure of regression.

Returns:

the test results with the same size of order of measures

loocv

public static <T> double[] loocv(RegressionTrainer<T> trainer,
                                 T[] x,
                                 double[] y,
                                 RegressionMeasure[] measures)

Leave-one-out cross validation of a regression model.

Type Parameters:

T - the data type of input objects.

Parameters:

trainer - a regression model trainer that is properly parameterized.

x - the test data set.

y - the test data response values.

measures - the performance measures of regression.

Returns:

the test results with the same size of order of measures

cv

public static <T> double cv(int k,
                            ClassifierTrainer<T> trainer,
                            T[] x,
                            int[] y)

Cross validation of a classification model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold cross validation.

trainer - a classifier trainer that is properly parameterized.

x - the test data set.

y - the test data labels.

Returns:

the accuracy on test dataset

cv

public static <T> double cv(int k,
                            RegressionTrainer<T> trainer,
                            T[] x,
                            double[] y)

Cross validation of a regression model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold cross validation.

trainer - a regression model trainer that is properly parameterized.

x - the test data set.

y - the test data response values.

Returns:

root mean squared error

cv

public static <T> double cv(int k,
                            ClassifierTrainer<T> trainer,
                            T[] x,
                            int[] y,
                            ClassificationMeasure measure)

Cross validation of a classification model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold cross validation.

trainer - a classifier trainer that is properly parameterized.

x - the test data set.

y - the test data labels.

measure - the performance measure of classification.

Returns:

the test results with the same size of order of measures

cv

public static <T> double[] cv(int k,
                              ClassifierTrainer<T> trainer,
                              T[] x,
                              int[] y,
                              ClassificationMeasure[] measures)

Cross validation of a classification model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold cross validation.

trainer - a classifier trainer that is properly parameterized.

x - the test data set.

y - the test data labels.

measures - the performance measures of classification.

Returns:

the test results with the same size of order of measures

cv

public static <T> double cv(int k,
                            RegressionTrainer<T> trainer,
                            T[] x,
                            double[] y,
                            RegressionMeasure measure)

Cross validation of a regression model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold cross validation.

trainer - a regression model trainer that is properly parameterized.

x - the test data set.

y - the test data response values.

measure - the performance measure of regression.

Returns:

the test results with the same size of order of measures

cv

public static <T> double[] cv(int k,
                              RegressionTrainer<T> trainer,
                              T[] x,
                              double[] y,
                              RegressionMeasure[] measures)

Cross validation of a regression model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold cross validation.

trainer - a regression model trainer that is properly parameterized.

x - the test data set.

y - the test data response values.

measures - the performance measures of regression.

Returns:

the test results with the same size of order of measures

bootstrap

public static <T> double[] bootstrap(int k,
                                     ClassifierTrainer<T> trainer,
                                     T[] x,
                                     int[] y)

Bootstrap accuracy estimation of a classification model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-round bootstrap estimation.

trainer - a classifier trainer that is properly parameterized.

x - the test data set.

y - the test data labels.

Returns:

the k-round accuracies

bootstrap

public static <T> double[] bootstrap(int k,
                                     RegressionTrainer<T> trainer,
                                     T[] x,
                                     double[] y)

Bootstrap RMSE estimation of a regression model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-round bootstrap estimation.

trainer - a regression model trainer that is properly parameterized.

x - the test data set.

y - the test data response values.

Returns:

the k-round root mean squared errors

bootstrap

public static <T> double[] bootstrap(int k,
                                     ClassifierTrainer<T> trainer,
                                     T[] x,
                                     int[] y,
                                     ClassificationMeasure measure)

Bootstrap performance estimation of a classification model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold bootstrap estimation.

trainer - a classifier trainer that is properly parameterized.

x - the test data set.

y - the test data labels.

measure - the performance measures of classification.

Returns:

k-by-m test result matrix, where k is the number of bootstrap samples and m is the number of performance measures.

bootstrap

public static <T> double[][] bootstrap(int k,
                                       ClassifierTrainer<T> trainer,
                                       T[] x,
                                       int[] y,
                                       ClassificationMeasure[] measures)

Bootstrap performance estimation of a classification model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold bootstrap estimation.

trainer - a classifier trainer that is properly parameterized.

x - the test data set.

y - the test data labels.

measures - the performance measures of classification.

Returns:

k-by-m test result matrix, where k is the number of bootstrap samples and m is the number of performance measures.

bootstrap

public static <T> double[] bootstrap(int k,
                                     RegressionTrainer<T> trainer,
                                     T[] x,
                                     double[] y,
                                     RegressionMeasure measure)

Bootstrap performance estimation of a regression model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold bootstrap estimation.

trainer - a regression model trainer that is properly parameterized.

x - the test data set.

y - the test data response values.

measure - the performance measure of regression.

Returns:

k-by-m test result matrix, where k is the number of bootstrap samples and m is the number of performance measures.

bootstrap

public static <T> double[][] bootstrap(int k,
                                       RegressionTrainer<T> trainer,
                                       T[] x,
                                       double[] y,
                                       RegressionMeasure[] measures)

Bootstrap performance estimation of a regression model.

Type Parameters:

T - the data type of input objects.

Parameters:

k - k-fold bootstrap estimation.

trainer - a regression model trainer that is properly parameterized.

x - the test data set.

y - the test data response values.

measures - the performance measures of regression.

Returns:

k-by-m test result matrix, where k is the number of bootstrap samples and m is the number of performance measures.