smile.classification

Class LDA

java.lang.Object

smile.classification.LDA

All Implemented Interfaces:

java.io.Serializable, java.util.function.ToDoubleFunction<double[]>, java.util.function.ToIntFunction<double[]>, Classifier<double[]>, SoftClassifier<double[]>

public class LDA
extends java.lang.Object
implements SoftClassifier<double[]>

Linear discriminant analysis. LDA is based on the Bayes decision theory and assumes that the conditional probability density functions are normally distributed. LDA also makes the simplifying homoscedastic assumption (i.e. that the class covariances are identical) and that the covariances have full rank. With these assumptions, the discriminant function of an input being in a class is purely a function of this linear combination of independent variables.

LDA is closely related to ANOVA (analysis of variance) and linear regression analysis, which also attempt to express one dependent variable as a linear combination of other features or measurements. In the other two methods, however, the dependent variable is a numerical quantity, while for LDA it is a categorical variable (i.e. the class label). Logistic regression and probit regression are more similar to LDA, as they also explain a categorical variable. These other methods are preferable in applications where it is not reasonable to assume that the independent variables are normally distributed, which is a fundamental assumption of the LDA method.

One complication in applying LDA (and Fisher's discriminant) to real data occurs when the number of variables/features does not exceed the number of samples. In this case, the covariance estimates do not have full rank, and so cannot be inverted. This is known as small sample size problem.

See Also:

FLD, QDA, RDA, NaiveBayes, Serialized Form

Constructor Summary

Constructors

Constructor and Description
LDA(double[] priori, double[][] mu, double[] eigen, smile.math.matrix.DenseMatrix scaling) Constructor.
LDA(double[] priori, double[][] mu, double[] eigen, smile.math.matrix.DenseMatrix scaling, smile.util.IntSet labels) Constructor.

Method Summary

Modifier and Type	Method and Description
static LDA	fit(double[][] x, int[] y) Learns linear discriminant analysis.
static LDA	fit(double[][] x, int[] y, double[] priori, double tol) Learns linear discriminant analysis.
static LDA	fit(double[][] x, int[] y, java.util.Properties prop) Learns linear discriminant analysis.
static LDA	fit(smile.data.formula.Formula formula, smile.data.DataFrame data) Learns linear discriminant analysis.
static LDA	fit(smile.data.formula.Formula formula, smile.data.DataFrame data, java.util.Properties prop) Learns linear discriminant analysis.
int	predict(double[] x) Predicts the class label of an instance.
int	predict(double[] x, double[] posteriori) Predicts the class label of an instance and also calculate a posteriori probabilities.
double[]	priori() Returns a priori probabilities.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface smile.classification.Classifier

applyAsDouble, applyAsInt, f, predict

Constructor Detail

LDA

public LDA(double[] priori,
           double[][] mu,
           double[] eigen,
           smile.math.matrix.DenseMatrix scaling)

Constructor.

Parameters:

priori - a priori probabilities of each class.

mu - the mean vectors of each class.

eigen - the eigen values of common variance matrix.

scaling - the eigen vectors of common covariance matrix.

LDA

public LDA(double[] priori,
           double[][] mu,
           double[] eigen,
           smile.math.matrix.DenseMatrix scaling,
           smile.util.IntSet labels)

Constructor.

Parameters:

priori - a priori probabilities of each class.

mu - the mean vectors of each class.

eigen - the eigen values of common variance matrix.

scaling - the eigen vectors of common covariance matrix.

labels - class labels

Method Detail

fit

public static LDA fit(smile.data.formula.Formula formula,
                      smile.data.DataFrame data)

Learns linear discriminant analysis.

Parameters:

formula - a symbolic description of the model to be fitted.

data - the data frame of the explanatory and response variables.

fit

public static LDA fit(smile.data.formula.Formula formula,
                      smile.data.DataFrame data,
                      java.util.Properties prop)

Learns linear discriminant analysis.

Parameters:

formula - a symbolic description of the model to be fitted.

data - the data frame of the explanatory and response variables.

fit

public static LDA fit(double[][] x,
                      int[] y)

Learns linear discriminant analysis.

Parameters:

x - training samples.

y - training labels in [0, k), where k is the number of classes.

fit

public static LDA fit(double[][] x,
                      int[] y,
                      java.util.Properties prop)

Learns linear discriminant analysis.

Parameters:

x - training samples.

y - training labels.

fit

public static LDA fit(double[][] x,
                      int[] y,
                      double[] priori,
                      double tol)

Learns linear discriminant analysis.

Parameters:

x - training samples.

y - training labels.

priori - the priori probability of each class. If null, it will be estimated from the training data.

tol - a tolerance to decide if a covariance matrix is singular; it will reject variables whose variance is less than tol².

priori

public double[] priori()

Returns a priori probabilities.

predict

public int predict(double[] x)

Description copied from interface: Classifier

Predicts the class label of an instance.

Specified by:

predict in interface Classifier<double[]>

Parameters:

x - the instance to be classified.

Returns:

the predicted class label.

predict

public int predict(double[] x,
                   double[] posteriori)

Description copied from interface: SoftClassifier

Predicts the class label of an instance and also calculate a posteriori probabilities. Classifiers may NOT support this method since not all classification algorithms are able to calculate such a posteriori probabilities.

Specified by:

predict in interface SoftClassifier<double[]>

Parameters:

x - an instance to be classified.

posteriori - the array to store a posteriori probabilities on output.

Returns:

the predicted class label