optimize

Type Members

class ApproximateGradientFunction[K, T] extends DiffFunction[T]

Approximates a gradient by finite differences.
trait ApproximateLineSearch extends MinimizingLineSearch

A line search optimizes a function of one variable without analytic gradient information.
class BacktrackingLineSearch extends ApproximateLineSearch

Implements the Backtracking Linesearch like that in LBFGS-C (which is (c) 2007-2010 Naoaki Okazaki under BSD)
trait BatchDiffFunction[T] extends DiffFunction[T] with (T, IndexedSeq[Int]) ⇒ Double

A diff function that supports subsets of the data.
case class BatchSize(size: Int) extends OptimizationOption with Product with Serializable
class CachedBatchDiffFunction[T] extends BatchDiffFunction[T]
class CachedDiffFunction[T] extends DiffFunction[T]
class CompactHessian extends NumericOps[CompactHessian]
abstract class CubicLineSearch extends SerializableLogging with MinimizingLineSearch
trait DiffFunction[T] extends StochasticDiffFunction[T]

Represents a differentiable function whose output is guaranteed to be consistent
class EmpiricalHessian[T] extends AnyRef

The empirical hessian evaluates the derivative for multiplcation.
sealed class FirstOrderException extends RuntimeException
abstract class FirstOrderMinimizer[T, DF <: StochasticDiffFunction[T]] extends Minimizer[T, DF] with SerializableLogging
class FisherDiffFunction[T] extends SecondOrderFunction[T, FisherMatrix[T]]
class FisherMatrix[T] extends AnyRef

The Fisher matrix approximates the Hessian by E[grad grad'].
trait IterableOptimizationPackage[Function, Vector, State] extends OptimizationPackage[Function, Vector]
case class L1Regularization(value: Double = 1.0) extends OptimizationOption with Product with Serializable
case class L2Regularization(value: Double = 1.0) extends OptimizationOption with Product with Serializable
class LBFGS[T] extends FirstOrderMinimizer[T, DiffFunction[T]] with SerializableLogging

Port of LBFGS to Scala.
trait LineSearch extends ApproximateLineSearch

A line search optimizes a function of one variable without analytic gradient information.
class LineSearchFailed extends FirstOrderException
case class MaxIterations(num: Int) extends OptimizationOption with Product with Serializable
trait Minimizer[T, -F] extends AnyRef

Anything that can minimize a function
trait MinimizingLineSearch extends AnyRef
class NaNHistory extends FirstOrderException
class OWLQN[K, T] extends LBFGS[T] with SerializableLogging

Implements the Orthant-wise Limited Memory QuasiNewton method, which is a variant of LBFGS that handles L1 regularization.
sealed trait OptimizationOption extends (OptParams) ⇒ OptParams
trait OptimizationPackage[Function, Vector] extends AnyRef
trait OptimizationPackageLowPriority extends AnyRef
class ProjectedQuasiNewton extends FirstOrderMinimizer[DenseVector[Double], DiffFunction[DenseVector[Double]]] with Projecting[DenseVector[Double]] with SerializableLogging
trait Projecting[T] extends AnyRef
trait SecondOrderFunction[T, H] extends DiffFunction[T]

Represents a function for which we can easily compute the Hessian.
class SpectralProjectedGradient[T, DF <: DiffFunction[T]] extends FirstOrderMinimizer[T, DF] with Projecting[T] with SerializableLogging

SPG is a Spectral Projected Gradient minimizer; it minimizes a differentiable function subject to the optimum being in some set, given by the projection operator projection
class StepSizeOverflow extends FirstOrderException
case class StepSizeScale(alpha: Double = 1.0) extends OptimizationOption with Product with Serializable
class StepSizeUnderflow extends FirstOrderException
class StochasticAveragedGradient[T] extends FirstOrderMinimizer[T, BatchDiffFunction[T]]
trait StochasticDiffFunction[T] extends (T) ⇒ Double

A differentiable function whose output is not guaranteed to be the same across consecutive invocations.
abstract class StochasticGradientDescent[T] extends FirstOrderMinimizer[T, StochasticDiffFunction[T]] with SerializableLogging

Minimizes a function using stochastic gradient descent
class StrongWolfeLineSearch extends CubicLineSearch
case class Tolerance(fvalTolerance: Double = 1.0E-5, gvalTolerance: Double = 1.0E-6) extends OptimizationOption with Product with Serializable
class TruncatedNewtonMinimizer[T, H] extends Minimizer[T, SecondOrderFunction[T, H]] with SerializableLogging

Implements a TruncatedNewton Trust region method (like Tron).

Value Members

object AdaptiveGradientDescent

Implements the L2^2 and L1 updates from Duchi et al 2010 Adaptive Subgradient Methods for Online Learning and Stochastic Optimization.
object BatchDiffFunction
object DiffFunction
object EmpiricalHessian
object FirstOrderMinimizer extends Serializable
object FisherMatrix
object GradientTester extends SerializableLogging

Class that compares the computed gradient with an empirical gradient based on finite differences.
object LBFGS extends Serializable
object LineSearch
object OptimizationOption
object OptimizationPackage
object PreferBatch extends OptimizationOption with Product with Serializable
object PreferOnline extends OptimizationOption with Product with Serializable
object ProjectedQuasiNewton extends Serializable
object SecondOrderFunction
object StochasticGradientDescent extends Serializable
package flow
def iterations[Objective, Vector, State](fn: Objective, init: Vector, options: OptimizationOption*)(implicit optimization: IterableOptimizationPackage[Objective, Vector, State]): Iterator[State]

Returns a sequence of states representing the iterates of a solver, given an breeze.optimize.IterableOptimizationPackage that knows how to minimize The actual state class varies with the kind of function passed in.
Returns a sequence of states representing the iterates of a solver, given an breeze.optimize.IterableOptimizationPackage that knows how to minimize The actual state class varies with the kind of function passed in. Typically, they have a .x value of type Vector that is the current point being evaluated, and .value is the current objective value
Objective
Vector
fn
init
options
optimization
returns
package linear
def minimize[Objective, Vector](fn: Objective, init: Vector, options: OptimizationOption*)(implicit optimization: OptimizationPackage[Objective, Vector]): Vector

Minimizes a function, given an breeze.optimize.OptimizationPackage that knows how to minimize
Minimizes a function, given an breeze.optimize.OptimizationPackage that knows how to minimize
Objective
Vector
fn
init
options
optimization
returns

package optimize

Type Members

class ApproximateGradientFunction[K, T] extends DiffFunction[T]

trait ApproximateLineSearch extends MinimizingLineSearch

class BacktrackingLineSearch extends ApproximateLineSearch

trait BatchDiffFunction[T] extends DiffFunction[T] with (T, IndexedSeq[Int]) ⇒ Double

case class BatchSize(size: Int) extends OptimizationOption with Product with Serializable

class CachedBatchDiffFunction[T] extends BatchDiffFunction[T]

class CachedDiffFunction[T] extends DiffFunction[T]

class CompactHessian extends NumericOps[CompactHessian]

abstract class CubicLineSearch extends SerializableLogging with MinimizingLineSearch

trait DiffFunction[T] extends StochasticDiffFunction[T]

class EmpiricalHessian[T] extends AnyRef

sealed class FirstOrderException extends RuntimeException

abstract class FirstOrderMinimizer[T, DF <: StochasticDiffFunction[T]] extends Minimizer[T, DF] with SerializableLogging

class FisherDiffFunction[T] extends SecondOrderFunction[T, FisherMatrix[T]]

class FisherMatrix[T] extends AnyRef

trait IterableOptimizationPackage[Function, Vector, State] extends OptimizationPackage[Function, Vector]

case class L1Regularization(value: Double = 1.0) extends OptimizationOption with Product with Serializable

case class L2Regularization(value: Double = 1.0) extends OptimizationOption with Product with Serializable

class LBFGS[T] extends FirstOrderMinimizer[T, DiffFunction[T]] with SerializableLogging

trait LineSearch extends ApproximateLineSearch

class LineSearchFailed extends FirstOrderException

case class MaxIterations(num: Int) extends OptimizationOption with Product with Serializable

trait Minimizer[T, -F] extends AnyRef

trait MinimizingLineSearch extends AnyRef

class NaNHistory extends FirstOrderException

class OWLQN[K, T] extends LBFGS[T] with SerializableLogging

sealed trait OptimizationOption extends (OptParams) ⇒ OptParams

trait OptimizationPackage[Function, Vector] extends AnyRef

trait OptimizationPackageLowPriority extends AnyRef

class ProjectedQuasiNewton extends FirstOrderMinimizer[DenseVector[Double], DiffFunction[DenseVector[Double]]] with Projecting[DenseVector[Double]] with SerializableLogging

trait Projecting[T] extends AnyRef

trait SecondOrderFunction[T, H] extends DiffFunction[T]

class SpectralProjectedGradient[T, DF <: DiffFunction[T]] extends FirstOrderMinimizer[T, DF] with Projecting[T] with SerializableLogging

class StepSizeOverflow extends FirstOrderException

case class StepSizeScale(alpha: Double = 1.0) extends OptimizationOption with Product with Serializable

class StepSizeUnderflow extends FirstOrderException

class StochasticAveragedGradient[T] extends FirstOrderMinimizer[T, BatchDiffFunction[T]]

trait StochasticDiffFunction[T] extends (T) ⇒ Double

abstract class StochasticGradientDescent[T] extends FirstOrderMinimizer[T, StochasticDiffFunction[T]] with SerializableLogging

class StrongWolfeLineSearch extends CubicLineSearch

case class Tolerance(fvalTolerance: Double = 1.0E-5, gvalTolerance: Double = 1.0E-6) extends OptimizationOption with Product with Serializable

class TruncatedNewtonMinimizer[T, H] extends Minimizer[T, SecondOrderFunction[T, H]] with SerializableLogging

Value Members

object AdaptiveGradientDescent

object BatchDiffFunction

object DiffFunction

object EmpiricalHessian

object FirstOrderMinimizer extends Serializable

object FisherMatrix

object GradientTester extends SerializableLogging

object LBFGS extends Serializable

object LineSearch

object OptimizationOption

object OptimizationPackage

object PreferBatch extends OptimizationOption with Product with Serializable

object PreferOnline extends OptimizationOption with Product with Serializable

object ProjectedQuasiNewton extends Serializable

object SecondOrderFunction

object StochasticGradientDescent extends Serializable

package flow

def iterations[Objective, Vector, State](fn: Objective, init: Vector, options: OptimizationOption*)(implicit optimization: IterableOptimizationPackage[Objective, Vector, State]): Iterator[State]

package linear

def minimize[Objective, Vector](fn: Objective, init: Vector, options: OptimizationOption*)(implicit optimization: OptimizationPackage[Objective, Vector]): Vector

Inherited from AnyRef

Inherited from Any

Ungrouped