smile.projection

Class GHA

java.lang.Object

smile.projection.GHA

All Implemented Interfaces:

java.io.Serializable, LinearProjection, Projection<double[]>

public class GHA
extends java.lang.Object
implements LinearProjection, java.io.Serializable

Generalized Hebbian Algorithm. GHA is a linear feed-forward neural network model for unsupervised learning with applications primarily in principal components analysis. It is single-layer process -- that is, a synaptic weight changes only depending on the response of the inputs and outputs of that layer.

It guarantees that GHA finds the first k eigenvectors of the covariance matrix, assuming that the associated eigenvalues are distinct. The convergence theorem is formulated in terms of a time-varying learning rate η. In practice, the learning rate η is chosen to be a small constant, in which case convergence is guaranteed with mean-squared error in synaptic weights of order η.

It also has a simple and predictable trade-off between learning speed and accuracy of convergence as set by the learning rate parameter η. It was shown that a larger learning rate η leads to faster convergence and larger asymptotic mean-square error, which is intuitively satisfying.

Compared to regular batch PCA algorithm based on eigen decomposition, GHA is an adaptive method and works with an arbitrarily large sample size. The storage requirement is modest. Another attractive feature is that, in a nonstationary environment, it has an inherent ability to track gradual changes in the optimal solution in an inexpensive way.

References

1. Terence D. Sanger. Optimal unsupervised learning in a single-layer linear feedforward neural network. Neural Networks 2(6):459-473, 1989.
2. Simon Haykin. Neural Networks: A Comprehensive Foundation (2 ed.). 1998.

See Also:

PCA, Serialized Form

Constructor Summary

Constructors

Constructor and Description
GHA(double[][] w, double r) Constructor.
GHA(int n, int p, double r) Constructor.

Method Summary

Modifier and Type	Method and Description
double	getLearningRate() Returns the learning rate.
smile.math.matrix.Matrix	getProjection() Returns the projection matrix.
GHA	setLearningRate(double r) Set the learning rate.
double	update(double[] x) Update the model with a new sample.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface smile.projection.LinearProjection

project, project

Constructor Detail

GHA

public GHA(int n,
           int p,
           double r)

Constructor.

Parameters:

n - the dimension of input space.

p - the dimension of feature space.

r - the learning rate.

GHA

public GHA(double[][] w,
           double r)

Constructor.

Parameters:

w - the initial projection matrix.

r - the learning rate.

Method Detail

getProjection

public smile.math.matrix.Matrix getProjection()

Returns the projection matrix. When GHA converges, the column of projection matrix are the first p eigenvectors of covariance matrix, ordered by decreasing eigenvalues. The dimension reduced data can be obtained by y = W * x.

Specified by:

getProjection in interface LinearProjection

getLearningRate

public double getLearningRate()

Returns the learning rate.

setLearningRate

public GHA setLearningRate(double r)

Set the learning rate.

update

public double update(double[] x)

Update the model with a new sample.

Parameters:

x - the centered learning sample whose E(x) = 0.

Returns:

the approximation error for input sample.