org.deeplearning4j.util

Class NetworkUtils

java.lang.Object

org.deeplearning4j.util.NetworkUtils

public class NetworkUtils
extends Object

Method Summary

Modifier and Type	Method and Description
static Double	getLearningRate(ComputationGraph net, String layerName) Get the current learning rate, for the specified layer, from the network.
static Double	getLearningRate(MultiLayerNetwork net, int layerNumber) Get the current learning rate, for the specified layer, fromthe network.
static INDArray	output(Model model, INDArray input) Currently supports MultiLayerNetwork and ComputationGraph models.
protected static INDArray	rebuildUpdaterStateArray(INDArray origUpdaterState, List<UpdaterBlock> orig, List<UpdaterBlock> newUpdater) Rebuild the updater state after a learning rate change.
static void	removeInstances(List<?> list, Class<?> remove) Remove any instances of the specified type from the list.
static void	removeInstancesWithWarning(List<?> list, Class<?> remove, String warning)
static void	setLearningRate(ComputationGraph net, double newLr) Set the learning rate for all layers in the network to the specified value.
static void	setLearningRate(ComputationGraph net, ISchedule newLrSchedule) Set the learning rate schedule for all layers in the network to the specified schedule.
static void	setLearningRate(ComputationGraph net, String layerName, double newLr) Set the learning rate for a single layer in the network to the specified value.
static void	setLearningRate(ComputationGraph net, String layerName, ISchedule lrSchedule) Set the learning rate schedule for a single layer in the network to the specified value. Note also that setLearningRate(ComputationGraph, ISchedule) should also be used in preference, when all layers need to be set to a new LR schedule. This schedule will replace any/all existing schedules, and also any fixed learning rate values. Note also that the iteration/epoch counts will not be reset.
static void	setLearningRate(MultiLayerNetwork net, double newLr) Set the learning rate for all layers in the network to the specified value.
static void	setLearningRate(MultiLayerNetwork net, int layerNumber, double newLr) Set the learning rate for a single layer in the network to the specified value.
static void	setLearningRate(MultiLayerNetwork net, int layerNumber, ISchedule lrSchedule) Set the learning rate schedule for a single layer in the network to the specified value. Note also that setLearningRate(MultiLayerNetwork, ISchedule) should also be used in preference, when all layers need to be set to a new LR schedule. This schedule will replace any/all existing schedules, and also any fixed learning rate values. Note also that the iteration/epoch counts will not be reset.
static void	setLearningRate(MultiLayerNetwork net, ISchedule newLrSchedule) Set the learning rate schedule for all layers in the network to the specified schedule.
static ComputationGraph	toComputationGraph(MultiLayerNetwork net) Convert a MultiLayerNetwork to a ComputationGraph

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

toComputationGraph

public static ComputationGraph toComputationGraph(MultiLayerNetwork net)

Convert a MultiLayerNetwork to a ComputationGraph

Returns:

ComputationGraph equivalent to this network (including parameters and updater state)

setLearningRate

public static void setLearningRate(MultiLayerNetwork net,
                                   double newLr)

Set the learning rate for all layers in the network to the specified value. Note that if any learning rate schedules are currently present, these will be removed in favor of the new (fixed) learning rate.

Note: This method not free from a performance point of view: a proper learning rate schedule should be used in preference to calling this method at every iteration.

Parameters:

net - Network to set the LR for

newLr - New learning rate for all layers

setLearningRate

public static void setLearningRate(MultiLayerNetwork net,
                                   ISchedule newLrSchedule)

Set the learning rate schedule for all layers in the network to the specified schedule. This schedule will replace any/all existing schedules, and also any fixed learning rate values.
Note that the iteration/epoch counts will not be reset. Use MultiLayerConfiguration#setIterationCount(int) and MultiLayerConfiguration.setEpochCount(int) if this is required

Parameters:

newLrSchedule - New learning rate schedule for all layers

setLearningRate

public static void setLearningRate(MultiLayerNetwork net,
                                   int layerNumber,
                                   double newLr)

Set the learning rate for a single layer in the network to the specified value. Note that if any learning rate schedules are currently present, these will be removed in favor of the new (fixed) learning rate.

Note: This method not free from a performance point of view: a proper learning rate schedule should be used in preference to calling this method at every iteration. Note also that setLearningRate(MultiLayerNetwork, double) should also be used in preference, when all layers need to be set to a new LR

Parameters:

layerNumber - Number of the layer to set the LR for

newLr - New learning rate for a single layers

setLearningRate

public static void setLearningRate(MultiLayerNetwork net,
                                   int layerNumber,
                                   ISchedule lrSchedule)

Set the learning rate schedule for a single layer in the network to the specified value.
Note also that setLearningRate(MultiLayerNetwork, ISchedule) should also be used in preference, when all layers need to be set to a new LR schedule.
This schedule will replace any/all existing schedules, and also any fixed learning rate values.
Note also that the iteration/epoch counts will not be reset. Use MultiLayerConfiguration#setIterationCount(int) and MultiLayerConfiguration.setEpochCount(int) if this is required

Parameters:

layerNumber - Number of the layer to set the LR schedule for

lrSchedule - New learning rate for a single layer

getLearningRate

public static Double getLearningRate(MultiLayerNetwork net,
                                     int layerNumber)

Get the current learning rate, for the specified layer, fromthe network. Note: If the layer has no learning rate (no parameters, or an updater without a learning rate) then null is returned

Parameters:

net - Network

layerNumber - Layer number to get the learning rate for

Returns:

Learning rate for the specified layer, or null

setLearningRate

public static void setLearningRate(ComputationGraph net,
                                   double newLr)

Set the learning rate for all layers in the network to the specified value. Note that if any learning rate schedules are currently present, these will be removed in favor of the new (fixed) learning rate.

Note: This method not free from a performance point of view: a proper learning rate schedule should be used in preference to calling this method at every iteration.

Parameters:

net - Network to set the LR for

newLr - New learning rate for all layers

setLearningRate

public static void setLearningRate(ComputationGraph net,
                                   ISchedule newLrSchedule)

Set the learning rate schedule for all layers in the network to the specified schedule. This schedule will replace any/all existing schedules, and also any fixed learning rate values.
Note that the iteration/epoch counts will not be reset. Use ComputationGraphConfiguration#setIterationCount(int) and ComputationGraphConfiguration#setEpochCount(int) if this is required

Parameters:

newLrSchedule - New learning rate schedule for all layers

setLearningRate

public static void setLearningRate(ComputationGraph net,
                                   String layerName,
                                   double newLr)

Set the learning rate for a single layer in the network to the specified value. Note that if any learning rate schedules are currently present, these will be removed in favor of the new (fixed) learning rate.

Note: This method not free from a performance point of view: a proper learning rate schedule should be used in preference to calling this method at every iteration. Note also that setLearningRate(ComputationGraph, double) should also be used in preference, when all layers need to be set to a new LR

Parameters:

layerName - Name of the layer to set the LR for

newLr - New learning rate for a single layers

setLearningRate

public static void setLearningRate(ComputationGraph net,
                                   String layerName,
                                   ISchedule lrSchedule)

Set the learning rate schedule for a single layer in the network to the specified value.
Note also that setLearningRate(ComputationGraph, ISchedule) should also be used in preference, when all layers need to be set to a new LR schedule.
This schedule will replace any/all existing schedules, and also any fixed learning rate values.
Note also that the iteration/epoch counts will not be reset. Use ComputationGraphConfiguration#setIterationCount(int) and ComputationGraphConfiguration#setEpochCount(int) if this is required

Parameters:

layerName - Name of the layer to set the LR schedule for

lrSchedule - New learning rate for a single layer

getLearningRate

public static Double getLearningRate(ComputationGraph net,
                                     String layerName)

Get the current learning rate, for the specified layer, from the network. Note: If the layer has no learning rate (no parameters, or an updater without a learning rate) then null is returned

Parameters:

net - Network

layerName - Layer name to get the learning rate for

Returns:

Learning rate for the specified layer, or null

output

public static INDArray output(Model model,
                              INDArray input)

Currently supports MultiLayerNetwork and ComputationGraph models. Pull requests to support additional org.deeplearning4j models are welcome.

Parameters:

model - Model to use

input - Inputs to the model

Returns:

output Outputs of the model

See Also:

ComputationGraph.outputSingle(INDArray...), MultiLayerNetwork.output(INDArray)

removeInstances

public static void removeInstances(List<?> list,
                                   Class<?> remove)

Remove any instances of the specified type from the list. This includes any subtypes.

Parameters:

list - List. May be null

remove - Type of objects to remove

removeInstancesWithWarning

public static void removeInstancesWithWarning(List<?> list,
                                              Class<?> remove,
                                              String warning)

rebuildUpdaterStateArray

protected static INDArray rebuildUpdaterStateArray(INDArray origUpdaterState,
                                                   List<UpdaterBlock> orig,
                                                   List<UpdaterBlock> newUpdater)

Rebuild the updater state after a learning rate change. With updaters like Adam, they have 2 components... m and v array, for a total updater state size of 2*numParams. Because we combine across parameters and layers where possible (smaller number of larger operations -> more efficient) we can sometimes need to rearrange the updater state array. For example, if the original updater state for Adam is organized like [mParam1, mParam2, vParam1, vParam2] in one block and we change the learning rate for one of the layers, param 1 and param2 now belong to different updater blocks. Consequently, we need to rearrange the updater state to be like [mParam1][vParam1] in block 1, [mParam2][vParam2] in block 2

Parameters:

origUpdaterState - Original updater state view array

orig - Original updater blocks

newUpdater - New updater blocks

Returns:

New state view array