org.deeplearning4j.nn.conf.dropout

Class GaussianDropout

java.lang.Object

org.deeplearning4j.nn.conf.dropout.GaussianDropout

All Implemented Interfaces:

Serializable, Cloneable, IDropout

public class GaussianDropout
extends Object
implements IDropout

Gaussian dropout. This is a multiplicative Gaussian noise (mean 1) on the input activations.

Each input activation x is independently set to:
x <- x * y, where y ~ N(1, stdev = sqrt((1-rate)/rate))
Dropout schedules (i.e., varying probability p as a function of iteration/epoch) are also supported.

Note 1: As per all IDropout instances, GaussianDropout is applied at training time only - and is automatically not applied at test time (for evaluation, etc)
Note 2: Frequently, dropout is not applied to (or, has higher retain probability for) input (first layer) layers. Dropout is also often not applied to output layers.

See: "Multiplicative Gaussian Noise" in Srivastava et al. 2014: Dropout: A Simple Way to Prevent Neural Networks from Overfitting http://www.cs.toronto.edu/~rsalakhu/papers/srivastava14a.pdf

Author:

Alex Black

See Also:

Serialized Form

Constructor Summary

Constructors

Modifier	Constructor and Description
	GaussianDropout(double rate)
protected	GaussianDropout(double rate, org.nd4j.linalg.schedule.ISchedule rateSchedule)
	GaussianDropout(org.nd4j.linalg.schedule.ISchedule rateSchedule)

Method Summary

Modifier and Type	Method and Description
org.nd4j.linalg.api.ndarray.INDArray	applyDropout(org.nd4j.linalg.api.ndarray.INDArray inputActivations, int iteration, int epoch, boolean inPlace)
GaussianDropout	clone()

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GaussianDropout

public GaussianDropout(double rate)

Parameters:

rate - Rate parameter, see GaussianDropout

GaussianDropout

public GaussianDropout(org.nd4j.linalg.schedule.ISchedule rateSchedule)

Parameters:

rateSchedule - Schedule for rate parameter, see GaussianDropout

GaussianDropout

protected GaussianDropout(double rate,
                          org.nd4j.linalg.schedule.ISchedule rateSchedule)

Method Detail

applyDropout

public org.nd4j.linalg.api.ndarray.INDArray applyDropout(org.nd4j.linalg.api.ndarray.INDArray inputActivations,
                                                         int iteration,
                                                         int epoch,
                                                         boolean inPlace)

Specified by:

applyDropout in interface IDropout

Parameters:

inputActivations - Input activations array

iteration - Current iteration number

epoch - Current epoch number

inPlace - If true: modify the input activations in-place. False: Copy the input activations and apply dropout on the copy instead

Returns:

clone

public GaussianDropout clone()

Specified by:

clone in interface IDropout

Overrides:

clone in class Object