org.deeplearning4j.nn.conf.layers

Class BatchNormalization.Builder

java.lang.Object

org.deeplearning4j.nn.conf.layers.Layer.Builder<T>

org.deeplearning4j.nn.conf.layers.BaseLayer.Builder<T>

org.deeplearning4j.nn.conf.layers.FeedForwardLayer.Builder<BatchNormalization.Builder>

org.deeplearning4j.nn.conf.layers.BatchNormalization.Builder

Enclosing class:

BatchNormalization

public static class BatchNormalization.Builder
extends FeedForwardLayer.Builder<BatchNormalization.Builder>

Field Summary

Fields

Modifier and Type	Field and Description
protected double	beta
protected List<LayerConstraint>	betaConstraints
protected double	decay
protected double	eps
protected double	gamma
protected List<LayerConstraint>	gammaConstraints
protected boolean	isMinibatch
protected boolean	lockGammaBeta

Fields inherited from class org.deeplearning4j.nn.conf.layers.FeedForwardLayer.Builder

nIn, nOut

Fields inherited from class org.deeplearning4j.nn.conf.layers.BaseLayer.Builder

activationFn, biasInit, biasUpdater, dist, gradientNormalization, gradientNormalizationThreshold, iupdater, l1, l1Bias, l2, l2Bias, weightInit, weightNoise

Fields inherited from class org.deeplearning4j.nn.conf.layers.Layer.Builder

allParamConstraints, biasConstraints, iDropout, layerName, weightConstraints

Constructor Summary

Constructors

Constructor and Description
Builder()
Builder(boolean lockGammaBeta)
Builder(double decay, boolean isMinibatch)
Builder(double gamma, double beta)
Builder(double gamma, double beta, boolean lockGammaBeta)

Method Summary

Modifier and Type	Method and Description
BatchNormalization.Builder	beta(double beta) Used only when 'true' is passed to lockGammaBeta(boolean).
BatchNormalization	build()
BatchNormalization.Builder	constrainBeta(LayerConstraint... constraints) Set constraints to be applied to the beta parameter of this batch normalisation layer.
BatchNormalization.Builder	constrainGamma(LayerConstraint... constraints) Set constraints to be applied to the gamma parameter of this batch normalisation layer.
BatchNormalization.Builder	decay(double decay) At test time: we can use a global estimate of the mean and variance, calculated using a moving average of the batch means/variances.
BatchNormalization.Builder	eps(double eps) Epsilon value for batch normalization; small floating point value added to variance (algorithm 1 in http://arxiv.org/pdf/1502.03167v3.pdf) to reduce/avoid underflow issues. Default: 1e-5
BatchNormalization.Builder	gamma(double gamma) Used only when 'true' is passed to lockGammaBeta(boolean).
BatchNormalization.Builder	lockGammaBeta(boolean lockGammaBeta) If set to true: lock the gamma and beta parameters to the values for each activation, specified by gamma(double) and beta(double).
BatchNormalization.Builder	minibatch(boolean minibatch) If doing minibatch training or not.

Methods inherited from class org.deeplearning4j.nn.conf.layers.FeedForwardLayer.Builder

nIn, nOut, units

Methods inherited from class org.deeplearning4j.nn.conf.layers.BaseLayer.Builder

activation, activation, biasInit, biasUpdater, dist, gradientNormalization, gradientNormalizationThreshold, l1, l1Bias, l2, l2Bias, updater, updater, weightInit, weightInit, weightNoise

Methods inherited from class org.deeplearning4j.nn.conf.layers.Layer.Builder

constrainAllParameters, constrainBias, constrainWeights, dropOut, dropOut, name

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

decay

protected double decay

eps

protected double eps

isMinibatch

protected boolean isMinibatch

lockGammaBeta

protected boolean lockGammaBeta

gamma

protected double gamma

beta

protected double beta

betaConstraints

protected List<LayerConstraint> betaConstraints

gammaConstraints

protected List<LayerConstraint> gammaConstraints

Constructor Detail

Builder

public Builder(double decay,
               boolean isMinibatch)

Builder

public Builder(double gamma,
               double beta)

Builder

public Builder(double gamma,
               double beta,
               boolean lockGammaBeta)

Builder

public Builder(boolean lockGammaBeta)

Builder

public Builder()

Method Detail

minibatch

public BatchNormalization.Builder minibatch(boolean minibatch)

If doing minibatch training or not. Default: true. Under most circumstances, this should be set to true. If doing full batch training (i.e., all examples in a single DataSet object - very small data sets) then this should be set to false. Affects how global mean/variance estimates are calculated.

Parameters:

minibatch - Minibatch parameter

gamma

public BatchNormalization.Builder gamma(double gamma)

Used only when 'true' is passed to lockGammaBeta(boolean). Value is not used otherwise.
Default: 1.0

Parameters:

gamma - Gamma parameter for all activations, used only with locked gamma/beta configuration mode

beta

public BatchNormalization.Builder beta(double beta)

Used only when 'true' is passed to lockGammaBeta(boolean). Value is not used otherwise.
Default: 0.0

Parameters:

beta - Beta parameter for all activations, used only with locked gamma/beta configuration mode

eps

public BatchNormalization.Builder eps(double eps)

Epsilon value for batch normalization; small floating point value added to variance (algorithm 1 in http://arxiv.org/pdf/1502.03167v3.pdf) to reduce/avoid underflow issues.
Default: 1e-5

Parameters:

eps - Epsilon values to use

decay

public BatchNormalization.Builder decay(double decay)

At test time: we can use a global estimate of the mean and variance, calculated using a moving average of the batch means/variances. This moving average is implemented as:
globalMeanEstimate = decay * globalMeanEstimate + (1-decay) * batchMean
globalVarianceEstimate = decay * globalVarianceEstimate + (1-decay) * batchVariance

Parameters:

decay - Decay value to use for global stats calculation

lockGammaBeta

public BatchNormalization.Builder lockGammaBeta(boolean lockGammaBeta)

If set to true: lock the gamma and beta parameters to the values for each activation, specified by gamma(double) and beta(double). Default: false -> learn gamma and beta parameter values during network training.

Parameters:

lockGammaBeta - If true: use fixed beta/gamma values. False: learn during

constrainBeta

public BatchNormalization.Builder constrainBeta(LayerConstraint... constraints)

Set constraints to be applied to the beta parameter of this batch normalisation layer. Default: no constraints.
Constraints can be used to enforce certain conditions (non-negativity of parameters, max-norm regularization, etc). These constraints are applied at each iteration, after the parameters have been updated.

Parameters:

constraints - Constraints to apply to the beta parameter of this layer

constrainGamma

public BatchNormalization.Builder constrainGamma(LayerConstraint... constraints)

Set constraints to be applied to the gamma parameter of this batch normalisation layer. Default: no constraints.
Constraints can be used to enforce certain conditions (non-negativity of parameters, max-norm regularization, etc). These constraints are applied at each iteration, after the parameters have been updated.

Parameters:

constraints - Constraints to apply to the gamma parameter of this layer

build

public BatchNormalization build()

Specified by:

build in class Layer.Builder<BatchNormalization.Builder>