smile.projection

Class RandomProjection

java.lang.Object

smile.projection.RandomProjection

All Implemented Interfaces:

java.io.Serializable, LinearProjection, Projection<double[]>

public class RandomProjection
extends java.lang.Object
implements LinearProjection, java.io.Serializable

Random projection is a promising dimensionality reduction technique for learning mixtures of Gaussians. According to Johnson-Lindenstrauss lemma, any n data points in high dimension can be mapped down to d = O(log n / ε²) dimension without distorting their pairwise distances by more than (1 + ε). However, this reduced dimension is still far too high. Let ε = 1, we need 2^d data points, and this usually exceeds n by many orders of magnitude.

Fortunately, we can reduce the dimension of the data far more drastically for the particular case of mixtures of Gaussians. In fact, we can map the data into just d = O(log k) dimensions, where k is the number of Gaussians. Therefore, the amount of data we will need is only polynomial in k. Note that this projected dimension is independent of the number of data points and of their original dimension. Experiments show that a value of log k works nicely.

Besides, even if the original clusters are highly eccentric (that is, far from spherical), random projection will make them more spherical. Note that eccentric clusters are problematic for the EM algorithm because intermediate covariance matrices may become singular or close to singular. Note that for high enough dimension, almost the entire Gaussian distribution lies in a thin shell.

References

1. S. Dasgupta. Experiments with random projection. UAI, 2000.
2. D. Achlioptas. Database-friendly random projections. 2001.
3. Chinmay Hegde, Michael Wakin, and Richard Baraniuk. Random projections for manifold learning. NIPS, 2007.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
RandomProjection(smile.math.matrix.DenseMatrix projection) Constructor.

Method Summary

Modifier and Type	Method and Description
smile.math.matrix.DenseMatrix	getProjection() Returns the projection matrix.
static RandomProjection	of(int n, int p) Generates a non-sparse random projection.
static RandomProjection	sparse(int n, int p) Generates a sparse random projection.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface smile.projection.LinearProjection

project, project

Constructor Detail

RandomProjection

public RandomProjection(smile.math.matrix.DenseMatrix projection)

Constructor.

Parameters:

projection - the projection matrix.

Method Detail

of

public static RandomProjection of(int n,
                                  int p)

Generates a non-sparse random projection.

Parameters:

n - the dimension of input space.

p - the dimension of feature space.

sparse

public static RandomProjection sparse(int n,
                                      int p)

Generates a sparse random projection.

Parameters:

n - the dimension of input space.

p - the dimension of feature space.

getProjection

public smile.math.matrix.DenseMatrix getProjection()

Description copied from interface: LinearProjection

Returns the projection matrix. The dimension reduced data can be obtained by y = W * x.

Specified by:

getProjection in interface LinearProjection