smile.projection

Class RandomProjection

java.lang.Object

smile.projection.RandomProjection

All Implemented Interfaces:

Projection<double[]>

public class RandomProjection
extends Object
implements Projection<double[]>

Random projection is a promising dimensionality reduction technique for learning mixtures of Gaussians. According to Johnson-Lindenstrauss lemma, any n data points in high dimension can be mapped down to d = O(log n / ε²) dimension without distorting their pairwise distances by more than (1 + ε). However, this reduced dimension is still far too high. Let ε = 1, we need 2^d data points, and this usually exceeds n by many orders of magnitude.

Fortunately, we can reduce the dimension of the data far more drastically for the particular case of mixtures of Gaussians. In fact, we can map the data into just d = O(log k) dimensions, where k is the number of Gaussians. Therefore, the amount of data we will need is only polynomial in k. Note that this projected dimension is independent of the number of data points and of their original dimension. Experiments show that a value of log k works nicely.

Besides, even if the original clusters are highly eccentric (that is, far from spherical), random projection will make them more spherical. Note that eccentric clusters are problematic for the EM algorithm because intermediate covariance matrices may become singular or close to singular. Note that for high enough dimension, almost the entire Gaussian distribution lies in a thin shell.

References

1. S. Dasgupta. Experiments with random projection. UAI, 2000.
2. D. Achlioptas. Database-friendly random projections. 2001.
3. Chinmay Hegde, Michael Wakin, and Richard Baraniuk. Random projections for manifold learning. NIPS, 2007.

Author:

Haifeng Li

Constructor Summary

Constructors

Constructor and Description
RandomProjection(int n, int p) Constructor.
RandomProjection(int n, int p, boolean sparse) Constructor.

Method Summary

Methods

Modifier and Type	Method and Description
double[][]	getProjection() Returns the projection matrix.
double[]	project(double[] x) Project a data point to the feature space.
double[][]	project(double[][] x) Project a set of data toe the feature space.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RandomProjection

public RandomProjection(int n,
                int p)

Constructor. Generate a non-sparse random projection.

Parameters:

n - the dimension of input space.

p - the dimension of feature space.

RandomProjection

public RandomProjection(int n,
                int p,
                boolean sparse)

Constructor.

Parameters:

n - the dimension of input space.

p - the dimension of feature space.

sparse - true to generate a sparse random projection proposed by Achlioptas.

Method Detail

getProjection

public double[][] getProjection()

Returns the projection matrix. The dimension reduced data can be obtained by y = W * x.

project

public double[] project(double[] x)

Description copied from interface: Projection

Project a data point to the feature space.

Specified by:

project in interface Projection<double[]>

project

public double[][] project(double[][] x)

Description copied from interface: Projection

Project a set of data toe the feature space.

Specified by:

project in interface Projection<double[]>