smile.base.mlp

Class Layer

java.lang.Object

smile.base.mlp.Layer

All Implemented Interfaces:

java.io.Serializable

Direct Known Subclasses:

HiddenLayer, OutputLayer

public abstract class Layer
extends java.lang.Object
implements java.io.Serializable

A layer in the neural network.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected double[]	bias The bias.
protected java.lang.ThreadLocal<double[]>	biasGradient The bias gradient.
protected java.lang.ThreadLocal<double[]>	biasUpdate The bias update.
protected int	n The number of neurons in this layer
protected java.lang.ThreadLocal<double[]>	output The output vector.
protected java.lang.ThreadLocal<double[]>	outputGradient The output gradient.
protected int	p The number of input variables.
protected java.lang.ThreadLocal<double[]>	rmsBiasGradient The accumulate bias gradient.
protected java.lang.ThreadLocal<smile.math.matrix.Matrix>	rmsWeightGradient The accumulate weight gradient.
protected smile.math.matrix.Matrix	weight The affine transformation matrix.
protected java.lang.ThreadLocal<smile.math.matrix.Matrix>	weightGradient The weight gradient.
protected java.lang.ThreadLocal<smile.math.matrix.Matrix>	weightUpdate The weight update.

Constructor Summary

Constructors

Constructor and Description
Layer(int n, int p) Constructor.
Layer(smile.math.matrix.Matrix weight, double[] bias) Constructor.

Method Summary

Modifier and Type	Method and Description
abstract void	backpropagate(double[] lowerLayerGradient) Propagates the errors back to a lower layer.
void	computeGradient(double[] x) Computes the parameter gradient for a sample of (mini-)batch.
void	computeGradientUpdate(double[] x, double learningRate, double momentum, double decay) Computes the parameter gradient and update the weights.
abstract void	f(double[] x) The activation or output function.
int	getInputSize() Returns the dimension of input vector (not including bias value).
int	getOutputSize() Returns the dimension of output vector.
double[]	gradient() Returns the output gradient vector.
static smile.base.mlp.HiddenLayerBuilder	linear(int n) Returns a hidden layer with linear activation function.
static smile.base.mlp.OutputLayerBuilder	mle(int n, OutputFunction f) Returns an output layer with (log-)likelihood cost function.
static smile.base.mlp.OutputLayerBuilder	mse(int n, OutputFunction f) Returns an output layer with mean squared error cost function.
double[]	output() Returns the output vector.
void	propagate(double[] x) Propagates signals from a lower layer to this layer.
static smile.base.mlp.HiddenLayerBuilder	rectifier(int n) Returns a hidden layer with rectified linear activation function.
static smile.base.mlp.HiddenLayerBuilder	sigmoid(int n) Returns a hidden layer with sigmoid activation function.
static smile.base.mlp.HiddenLayerBuilder	tanh(int n) Returns a hidden layer with hyperbolic tangent activation function.
void	update(int m, double learningRate, double momentum, double decay, double rho, double epsilon) Adjust network weights by back-propagation algorithm.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

n

protected int n

The number of neurons in this layer

p

protected int p

The number of input variables.

weight

protected smile.math.matrix.Matrix weight

The affine transformation matrix.

bias

protected double[] bias

The bias.

output

protected transient java.lang.ThreadLocal<double[]> output

The output vector.

outputGradient

protected transient java.lang.ThreadLocal<double[]> outputGradient

The output gradient.

weightGradient

protected transient java.lang.ThreadLocal<smile.math.matrix.Matrix> weightGradient

The weight gradient.

biasGradient

protected transient java.lang.ThreadLocal<double[]> biasGradient

The bias gradient.

rmsWeightGradient

protected transient java.lang.ThreadLocal<smile.math.matrix.Matrix> rmsWeightGradient

The accumulate weight gradient.

rmsBiasGradient

protected transient java.lang.ThreadLocal<double[]> rmsBiasGradient

The accumulate bias gradient.

weightUpdate

protected transient java.lang.ThreadLocal<smile.math.matrix.Matrix> weightUpdate

The weight update.

biasUpdate

protected transient java.lang.ThreadLocal<double[]> biasUpdate

The bias update.

Constructor Detail

Layer

public Layer(int n,
             int p)

Constructor. Randomly initialized weights and zero bias.

Parameters:

n - the number of neurons.

p - the number of input variables (not including bias value).

Layer

public Layer(smile.math.matrix.Matrix weight,
             double[] bias)

Constructor.

Parameters:

weight - the weight matrix.

bias - the bias vector.

Method Detail

getOutputSize

public int getOutputSize()

Returns the dimension of output vector.

getInputSize

public int getInputSize()

Returns the dimension of input vector (not including bias value).

output

public double[] output()

Returns the output vector.

gradient

public double[] gradient()

Returns the output gradient vector.

propagate

public void propagate(double[] x)

Propagates signals from a lower layer to this layer.

Parameters:

x - the lower layer signals.

f

public abstract void f(double[] x)

The activation or output function.

Parameters:

x - the input and output values.

backpropagate

public abstract void backpropagate(double[] lowerLayerGradient)

Propagates the errors back to a lower layer.

Parameters:

lowerLayerGradient - the gradient vector of lower layer.

computeGradientUpdate

public void computeGradientUpdate(double[] x,
                                  double learningRate,
                                  double momentum,
                                  double decay)

Computes the parameter gradient and update the weights.

Parameters:

x - the input vector.

learningRate - the learning rate.

momentum - the momentum factor

decay - weight decay factor

computeGradient

public void computeGradient(double[] x)

Computes the parameter gradient for a sample of (mini-)batch.

Parameters:

x - the input vector.

update

public void update(int m,
                   double learningRate,
                   double momentum,
                   double decay,
                   double rho,
                   double epsilon)

Adjust network weights by back-propagation algorithm.

Parameters:

m - the size of mini-batch.

learningRate - the learning rate.

momentum - the momentum factor

decay - weight decay factor

linear

public static smile.base.mlp.HiddenLayerBuilder linear(int n)

Returns a hidden layer with linear activation function.

Parameters:

n - the number of neurons.

rectifier

public static smile.base.mlp.HiddenLayerBuilder rectifier(int n)

Returns a hidden layer with rectified linear activation function.

Parameters:

n - the number of neurons.

sigmoid

public static smile.base.mlp.HiddenLayerBuilder sigmoid(int n)

Returns a hidden layer with sigmoid activation function.

Parameters:

n - the number of neurons.

tanh

public static smile.base.mlp.HiddenLayerBuilder tanh(int n)

Returns a hidden layer with hyperbolic tangent activation function.

Parameters:

n - the number of neurons.

mse

public static smile.base.mlp.OutputLayerBuilder mse(int n,
                                                    OutputFunction f)

Returns an output layer with mean squared error cost function.

Parameters:

n - the number of neurons.

f - the output function.

mle

public static smile.base.mlp.OutputLayerBuilder mle(int n,
                                                    OutputFunction f)

Returns an output layer with (log-)likelihood cost function.

Parameters:

n - the number of neurons.

f - the output function.