smile.classification

Class FLD

java.lang.Object

smile.classification.FLD

All Implemented Interfaces:

Classifier<double[]>, Projection<double[]>

public class FLD
extends Object
implements Classifier<double[]>, Projection<double[]>

Fisher's linear discriminant. Fisher defined the separation between two distributions to be the ratio of the variance between the classes to the variance within the classes, which is, in some sense, a measure of the signal-to-noise ratio for the class labeling. FLD finds a linear combination of features which maximizes the separation after the projection. The resulting combination may be used for dimensionality reduction before later classification.

The terms Fisher's linear discriminant and LDA are often used interchangeably, although FLD actually describes a slightly different discriminant, which does not make some of the assumptions of LDA such as normally distributed classes or equal class covariances. When the assumptions of LDA are satisfied, FLD is equivalent to LDA.

FLD is also closely related to principal component analysis (PCA), which also looks for linear combinations of variables which best explain the data. As a supervised method, FLD explicitly attempts to model the difference between the classes of data. On the other hand, PCA is a unsupervised method and does not take into account any difference in class.

One complication in applying FLD (and LDA) to real data occurs when the number of variables/features does not exceed the number of samples. In this case, the covariance estimates do not have full rank, and so cannot be inverted. This is known as small sample size problem.

Author:

Haifeng Li

See Also:

LDA, PCA

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	FLD.Trainer Trainer for Fisher's linear discriminant.

Constructor Summary

Constructors

Constructor and Description
FLD(double[][] x, int[] y) Constructor.
FLD(double[][] x, int[] y, int L) Constructor.
FLD(double[][] x, int[] y, int L, double tol) Constructor.

Method Summary

Methods

Modifier and Type	Method and Description
double[][]	getProjection() Returns the projection matrix W.
int	predict(double[] x) Predicts the class label of an instance.
int	predict(double[] x, double[] posteriori) Predicts the class label of an instance and also calculate a posteriori probabilities.
double[]	project(double[] x) Project a data point to the feature space.
double[][]	project(double[][] x) Project a set of data toe the feature space.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FLD

public FLD(double[][] x,
   int[] y)

Constructor. Learn Fisher's linear discriminant.

Parameters:

x - training instances.

y - training labels in [0, k), where k is the number of classes.

FLD

public FLD(double[][] x,
   int[] y,
   int L)

Constructor. Learn Fisher's linear discriminant.

Parameters:

x - training instances.

y - training labels in [0, k), where k is the number of classes.

L - the dimensionality of mapped space.

FLD

public FLD(double[][] x,
   int[] y,
   int L,
   double tol)

Constructor. Learn Fisher's linear discriminant.

Parameters:

x - training instances.

y - training labels in [0, k), where k is the number of classes.

L - the dimensionality of mapped space.

tol - a tolerance to decide if a covariance matrix is singular; it will reject variables whose variance is less than tol².

Method Detail

predict

public int predict(double[] x)

Description copied from interface: Classifier

Predicts the class label of an instance.

Specified by:

predict in interface Classifier<double[]>

Parameters:

x - the instance to be classified.

Returns:

the predicted class label

predict

public int predict(double[] x,
          double[] posteriori)

Predicts the class label of an instance and also calculate a posteriori probabilities. Not supported.

Specified by:

predict in interface Classifier<double[]>

Parameters:

x - the instance to be classified.

posteriori - the array to store a posteriori probabilities on output.

Returns:

the predicted class label

project

public double[] project(double[] x)

Description copied from interface: Projection

Project a data point to the feature space.

Specified by:

project in interface Projection<double[]>

project

public double[][] project(double[][] x)

Description copied from interface: Projection

Project a set of data toe the feature space.

Specified by:

project in interface Projection<double[]>

getProjection

public double[][] getProjection()

Returns the projection matrix W. The dimension reduced data can be obtained by y = W' * x.