smile.classification

Class RandomForest

java.lang.Object

smile.classification.RandomForest

All Implemented Interfaces:

java.io.Serializable, java.util.function.ToDoubleFunction<smile.data.Tuple>, java.util.function.ToIntFunction<smile.data.Tuple>, Classifier<smile.data.Tuple>, DataFrameClassifier, SoftClassifier<smile.data.Tuple>

public class RandomForest
extends java.lang.Object
implements SoftClassifier<smile.data.Tuple>, DataFrameClassifier

Random forest for classification. Random forest is an ensemble classifier that consists of many decision trees and outputs the majority vote of individual trees. The method combines bagging idea and the random selection of features.

Each tree is constructed using the following algorithm:

1. If the number of cases in the training set is N, randomly sample N cases with replacement from the original data. This sample will be the training set for growing the tree.
2. If there are M input variables, a number m << M is specified such that at each node, m variables are selected at random out of the M and the best split on these m is used to split the node. The value of m is held constant during the forest growing.
3. Each tree is grown to the largest extent possible. There is no pruning.

The advantages of random forest are:

• For many data sets, it produces a highly accurate classifier.
• It runs efficiently on large data sets.
• It can handle thousands of input variables without variable deletion.
• It gives estimates of what variables are important in the classification.
• It generates an internal unbiased estimate of the generalization error as the forest building progresses.
• It has an effective method for estimating missing data and maintains accuracy when a large proportion of the data are missing.

The disadvantages are

• Random forests are prone to over-fitting for some datasets. This is even more pronounced on noisy data.
• For data including categorical variables with different number of levels, random forests are biased in favor of those attributes with more levels. Therefore, the variable importance scores from random forest are not reliable for this type of data.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
RandomForest(smile.data.formula.Formula formula, int k, java.util.List<smile.classification.RandomForest.Tree> trees, double error, double[] importance) Constructor.
RandomForest(smile.data.formula.Formula formula, int k, java.util.List<smile.classification.RandomForest.Tree> trees, double error, double[] importance, smile.util.IntSet labels) Constructor.

Method Summary

Modifier and Type	Method and Description
double	error() Returns the out-of-bag estimation of error rate.
static RandomForest	fit(smile.data.formula.Formula formula, smile.data.DataFrame data) Fits a random forest for classification.
static RandomForest	fit(smile.data.formula.Formula formula, smile.data.DataFrame data, int ntrees, int mtry, SplitRule rule, int maxDepth, int maxNodes, int nodeSize, double subsample) Fits a random forest for classification.
static RandomForest	fit(smile.data.formula.Formula formula, smile.data.DataFrame data, int ntrees, int mtry, SplitRule rule, int maxDepth, int maxNodes, int nodeSize, double subsample, java.util.Optional<int[]> classWeight) Fits a random forest for regression.
static RandomForest	fit(smile.data.formula.Formula formula, smile.data.DataFrame data, int ntrees, int mtry, SplitRule rule, int maxDepth, int maxNodes, int nodeSize, double subsample, java.util.Optional<int[]> classWeight, java.util.function.LongSupplier seedGenerator) Fits a random forest for regression.
static RandomForest	fit(smile.data.formula.Formula formula, smile.data.DataFrame data, int ntrees, int mtry, SplitRule rule, int maxDepth, int maxNodes, int nodeSize, double subsample, java.util.Optional<int[]> classWeight, java.util.Optional<java.util.stream.LongStream> seeds) Fits a random forest for classification.
static RandomForest	fit(smile.data.formula.Formula formula, smile.data.DataFrame data, java.util.Properties prop) Fits a random forest for classification.
smile.data.formula.Formula	formula() Returns the formula associated with the model.
double[]	importance() Returns the variable importance.
int	predict(smile.data.Tuple x) Predicts the class label of an instance.
int	predict(smile.data.Tuple x, double[] posteriori) Predicts the class label of an instance and also calculate a posteriori probabilities.
RandomForest	prune(smile.data.DataFrame test) Returns a new random forest by reduced error pruning.
smile.data.type.StructType	schema() Returns the design matrix schema.
int	size() Returns the number of trees in the model.
int[][]	test(smile.data.DataFrame data) Test the model on a validation dataset.
DecisionTree[]	trees() Returns the decision trees.
void	trim(int ntrees) Trims the tree model set to a smaller size in case of over-fitting.
int	vote(smile.data.Tuple x, double[] posteriori) Predict and estimate the probability by voting.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface smile.classification.Classifier

applyAsDouble, applyAsInt, f, predict

Methods inherited from interface smile.classification.DataFrameClassifier

predict

Constructor Detail

RandomForest

public RandomForest(smile.data.formula.Formula formula,
                    int k,
                    java.util.List<smile.classification.RandomForest.Tree> trees,
                    double error,
                    double[] importance)

Constructor.

Parameters:

formula - a symbolic description of the model to be fitted.

k - the number of classes.

trees - forest of decision trees.

error - the out-of-bag estimation of error rate.

importance - variable importance

RandomForest

public RandomForest(smile.data.formula.Formula formula,
                    int k,
                    java.util.List<smile.classification.RandomForest.Tree> trees,
                    double error,
                    double[] importance,
                    smile.util.IntSet labels)

Constructor.

Parameters:

formula - a symbolic description of the model to be fitted.

k - the number of classes.

trees - forest of decision trees.

error - the out-of-bag estimation of error rate.

importance - variable importance

labels - class labels

Method Detail

fit

public static RandomForest fit(smile.data.formula.Formula formula,
                               smile.data.DataFrame data)

Fits a random forest for classification.

Parameters:

formula - a symbolic description of the model to be fitted.

data - the data frame of the explanatory and response variables.

fit

public static RandomForest fit(smile.data.formula.Formula formula,
                               smile.data.DataFrame data,
                               java.util.Properties prop)

Fits a random forest for classification.

Parameters:

formula - a symbolic description of the model to be fitted.

data - the data frame of the explanatory and response variables.

fit

public static RandomForest fit(smile.data.formula.Formula formula,
                               smile.data.DataFrame data,
                               int ntrees,
                               int mtry,
                               SplitRule rule,
                               int maxDepth,
                               int maxNodes,
                               int nodeSize,
                               double subsample)

Fits a random forest for classification.

Parameters:

formula - a symbolic description of the model to be fitted.

data - the data frame of the explanatory and response variables.

ntrees - the number of trees.

mtry - the number of input variables to be used to determine the decision at a node of the tree. p/3 seems to give generally good performance, where p is the number of variables.

maxDepth - the maximum depth of the tree.

maxNodes - the maximum number of leaf nodes in the tree.

nodeSize - the number of instances in a node below which the tree will not split, setting nodeSize = 5 generally gives good results.

subsample - the sampling rate for training tree. 1.0 means sampling with replacement. < 1.0 means sampling without replacement.

fit

public static RandomForest fit(smile.data.formula.Formula formula,
                               smile.data.DataFrame data,
                               int ntrees,
                               int mtry,
                               SplitRule rule,
                               int maxDepth,
                               int maxNodes,
                               int nodeSize,
                               double subsample,
                               java.util.Optional<int[]> classWeight)

Fits a random forest for regression.

Parameters:

formula - a symbolic description of the model to be fitted.

data - the data frame of the explanatory and response variables.

ntrees - the number of trees.

mtry - the number of input variables to be used to determine the decision at a node of the tree. p/3 seems to give generally good performance, where p is the number of variables.

maxDepth - the maximum depth of the tree.

maxNodes - the maximum number of leaf nodes in the tree.

nodeSize - the number of instances in a node below which the tree will not split, setting nodeSize = 5 generally gives good results.

subsample - the sampling rate for training tree. 1.0 means sampling with replacement. < 1.0 means sampling without replacement.

classWeight - Priors of the classes. The weight of each class is roughly the ratio of samples in each class. For example, if there are 400 positive samples and 100 negative samples, the classWeight should be [1, 4] (assuming label 0 is of negative, label 1 is of positive).

fit

public static RandomForest fit(smile.data.formula.Formula formula,
                               smile.data.DataFrame data,
                               int ntrees,
                               int mtry,
                               SplitRule rule,
                               int maxDepth,
                               int maxNodes,
                               int nodeSize,
                               double subsample,
                               java.util.Optional<int[]> classWeight,
                               java.util.function.LongSupplier seedGenerator)

Fits a random forest for regression.

Parameters:

formula - a symbolic description of the model to be fitted.

data - the data frame of the explanatory and response variables.

ntrees - the number of trees.

mtry - the number of input variables to be used to determine the decision at a node of the tree. p/3 seems to give generally good performance, where p is the number of variables.

maxDepth - the maximum depth of the tree.

maxNodes - the maximum number of leaf nodes in the tree.

nodeSize - the number of instances in a node below which the tree will not split, setting nodeSize = 5 generally gives good results.

subsample - the sampling rate for training tree. 1.0 means sampling with replacement. < 1.0 means sampling without replacement.

classWeight - Priors of the classes. The weight of each class is roughly the ratio of samples in each class. For example, if there are 400 positive samples and 100 negative samples, the classWeight should be [1, 4] (assuming label 0 is of negative, label 1 is of positive).

seedGenerator - RNG seed generator.

fit

public static RandomForest fit(smile.data.formula.Formula formula,
                               smile.data.DataFrame data,
                               int ntrees,
                               int mtry,
                               SplitRule rule,
                               int maxDepth,
                               int maxNodes,
                               int nodeSize,
                               double subsample,
                               java.util.Optional<int[]> classWeight,
                               java.util.Optional<java.util.stream.LongStream> seeds)

Fits a random forest for classification.

Parameters:

formula - a symbolic description of the model to be fitted.

data - the data frame of the explanatory and response variables.

ntrees - the number of trees.

mtry - the number of random selected features to be used to determine the decision at a node of the tree. floor(sqrt(dim)) seems to give generally good performance, where dim is the number of variables.

maxDepth - the maximum depth of the tree.

maxNodes - the maximum number of leaf nodes in the tree.

nodeSize - the minimum size of leaf nodes.

subsample - the sampling rate for training tree. 1.0 means sampling with replacement. < 1.0 means sampling without replacement.

rule - Decision tree split rule.

classWeight - Priors of the classes. The weight of each class is roughly the ratio of samples in each class. For example, if there are 400 positive samples and 100 negative samples, the classWeight should be [1, 4] (assuming label 0 is of negative, label 1 is of positive).

seeds - optional RNG seeds for each regression tree.

formula

public smile.data.formula.Formula formula()

Description copied from interface: DataFrameClassifier

Returns the formula associated with the model.

Specified by:

formula in interface DataFrameClassifier

schema

public smile.data.type.StructType schema()

Description copied from interface: DataFrameClassifier

Returns the design matrix schema.

Specified by:

schema in interface DataFrameClassifier

error

public double error()

Returns the out-of-bag estimation of error rate. The OOB estimate is quite accurate given that enough trees have been grown. Otherwise the OOB estimate can bias upward.

Returns:

the out-of-bag estimation of error rate

importance

public double[] importance()

Returns the variable importance. Every time a split of a node is made on variable the (GINI, information gain, etc.) impurity criterion for the two descendent nodes is less than the parent node. Adding up the decreases for each individual variable over all trees in the forest gives a fast measure of variable importance that is often very consistent with the permutation importance measure.

Returns:

the variable importance

size

public int size()

Returns the number of trees in the model.

Returns:

the number of trees in the model

trees

public DecisionTree[] trees()

Returns the decision trees.

trim

public void trim(int ntrees)

Trims the tree model set to a smaller size in case of over-fitting. Or if extra decision trees in the model don't improve the performance, we may remove them to reduce the model size and also improve the speed of prediction.

Parameters:

ntrees - the new (smaller) size of tree model set.

predict

public int predict(smile.data.Tuple x)

Description copied from interface: Classifier

Predicts the class label of an instance.

Specified by:

predict in interface Classifier<smile.data.Tuple>

Specified by:

predict in interface DataFrameClassifier

Parameters:

x - the instance to be classified.

Returns:

the predicted class label.

predict

public int predict(smile.data.Tuple x,
                   double[] posteriori)

Description copied from interface: SoftClassifier

Predicts the class label of an instance and also calculate a posteriori probabilities. Classifiers may NOT support this method since not all classification algorithms are able to calculate such a posteriori probabilities.

Specified by:

predict in interface SoftClassifier<smile.data.Tuple>

Parameters:

x - an instance to be classified.

posteriori - the array to store a posteriori probabilities on output.

Returns:

the predicted class label

vote

public int vote(smile.data.Tuple x,
                double[] posteriori)

Predict and estimate the probability by voting.

test

public int[][] test(smile.data.DataFrame data)

Test the model on a validation dataset.

Parameters:

data - the test data set.

Returns:

the predictions with first 1, 2, ..., decision trees.

prune

public RandomForest prune(smile.data.DataFrame test)

Returns a new random forest by reduced error pruning.

Parameters:

test - the test data set to evaluate the errors of nodes.

Returns:

a new pruned random forest.