smile.projection

Class GHA

java.lang.Object

smile.projection.GHA

All Implemented Interfaces:

Projection<double[]>

public class GHA
extends Object
implements Projection<double[]>

Generalized Hebbian Algorithm. GHA is a linear feed-forward neural network model for unsupervised learning with applications primarily in principal components analysis. It is single-layer process -- that is, a synaptic weight changes only depending on the response of the inputs and outputs of that layer.

It guarantees that GHA finds the first k eigenvectors of the covariance matrix, assuming that the associated eigenvalues are distinct. The convergence theorem is forumulated in terms of a time-varying learning rate η. In practice, the learning rate η is chosen to be a small constant, in which case convergence is guaranteed with mean-squared error in synaptic weights of order η.

It also has a simple and predictable trade-off between learning speed and accuracy of convergence as set by the learning rate parameter η. It was shown that a larger learning rate η leads to faster convergence and larger asymptotic mean-square error, which is intuitively satisfying.

Compared to regular batch PCA algorithm based on eigen decomposition, GHA is an adaptive method and works with an arbitrarily large sample size. The storage requirement is modest. Another attractive feature is that, in a nonstationary environment, it has an inherent ability to track gradual changes in the optimal solution in an inexpensive way.

References

1. Terence D. Sanger. Optimal unsupervised learning in a single-layer linear feedforward neural network. Neural Networks 2(6):459-473, 1989.
2. Simon Haykin. Neural Networks: A Comprehensive Foundation (2 ed.). 1998.

Author:

Haifeng Li

See Also:

PCA

Constructor Summary

Constructors

Constructor and Description
GHA(double[][] w, double r) Constructor.
GHA(int n, int p, double r) Constructor.

Method Summary

Modifier and Type	Method and Description
double	getLearningRate() Returns the learning rate.
double[][]	getProjection() Returns the projection matrix.
double	learn(double[] x) Update the model with a new sample.
double[]	project(double[] x) Project a data point to the feature space.
double[][]	project(double[][] x) Project a set of data toe the feature space.
GHA	setLearningRate(double r) Set the learning rate.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GHA

public GHA(int n,
           int p,
           double r)

Constructor.

Parameters:

n - the dimension of input space.

p - the dimension of feature space.

r - the learning rate.

GHA

public GHA(double[][] w,
           double r)

Constructor.

Parameters:

w - the initial projection matrix.

r - the learning rate.

Method Detail

getProjection

public double[][] getProjection()

Returns the projection matrix. When GHA converges, the column of projection matrix are the first p eigenvectors of covariance matrix, ordered by decreasing eigenvalues. The dimension reduced data can be obtained by y = W * x.

getLearningRate

public double getLearningRate()

Returns the learning rate.

setLearningRate

public GHA setLearningRate(double r)

Set the learning rate.

project

public double[] project(double[] x)

Description copied from interface: Projection

Project a data point to the feature space.

Specified by:

project in interface Projection<double[]>

project

public double[][] project(double[][] x)

Description copied from interface: Projection

Project a set of data toe the feature space.

Specified by:

project in interface Projection<double[]>

learn

public double learn(double[] x)

Update the model with a new sample.

Parameters:

x - the centered learning sample whose E(x) = 0.

Returns:

the approximation error for input sample.