MatrixLBFGS

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
def axpy(a: Double, x: Vector, y: Array[Double]): Unit

Definition Classes
HasNetlibBlas
def axpy(a: Double, x: Array[Double], y: Array[Double]): Unit

Definition Classes
HasNetlibBlas
def blas: BLAS

Definition Classes
HasNetlibBlas
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
def computeGradient(data: Vector, label: Vector, weights: DenseMatrix, accumulatedGradient: DenseMatrix, accumulatedLoss: DenseVector): Unit

Compute the gradient and loss given the features of a single data point.
Compute the gradient and loss given the features of a single data point.
data
features for one data point
label
label for this data point
weights
weights/coefficients corresponding to features
returns
Loss vector for the current point.
def computeGradientAndLoss[T](data: RDD[(Vector, T)], currentWeights: DenseMatrix, batchSize: Int = 10, labelsAssigner: (Int, T, Array[Double]) ⇒ Unit): (DenseMatrix, DenseVector)

Computes cumulative gradient and loss for a set of samples
Computes cumulative gradient and loss for a set of samples
data
RDD with vectors of features and vectors of labels
currentWeights
Current weights matrix.
batchSize
Number of samples to collect in a batch before computing.
labelsAssigner
Routine for extracting labels vector (in some cases only part of the labels are needed)
returns
Tuple with cumulative gradient matrix and loss vector
def computeGradientMatrix(data: Array[Double], label: Array[Double], weights: DenseMatrix, accumulatedGradient: DenseMatrix, accumulatedLoss: DenseVector, marginCache: Array[Double], samples: Int): Unit

Computes gradient and loss for a batch containing data and labels from multiple samples.
Computes gradient and loss for a batch containing data and labels from multiple samples.
data
Samples matrix in row-major form (one row per sample)
label
Labels matrix in row-major form (one row per sample)
weights
Matrix with weights (column-major)
accumulatedGradient
Matrix with accumulated gradient
accumulatedLoss
Vector with accumulated loss
marginCache
Array used to cache margin calculations
samples
Number of samples in the batch
def copy(x: Array[Double], y: Array[Double]): Unit

Definition Classes
HasNetlibBlas
def dscal(a: Double, data: Array[Double]): Unit

Definition Classes
HasNetlibBlas
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def evaluateMaxRegularization(data: RDD[(Vector, Vector)], regulaizeLast: Boolean, numFeatures: Int, labelsMean: DenseVector, numExamples: Long): Vector

Evaluates upper bound for regularization param for each label based on estimation from http://jmlr.org/papers/volume8/koh07a/koh07a.pdf
Evaluates upper bound for regularization param for each label based on estimation from http://jmlr.org/papers/volume8/koh07a/koh07a.pdf
data
RDD with samples features -> labels.
regulaizeLast
Whenever to consider last feature as a subject for regularization (set to false to exclude intercept from regularization)
returns
A pair with instances count and regularization bounds.
def evaluateMaxRegularization(data: DataFrame, featuresColumn: String, labelColumn: String, regulaizeLast: Boolean): (Long, Vector)

Evaluates upper bound for regularization param for each label based on estimation from http://jmlr.org/papers/volume8/koh07a/koh07a.pdf
Evaluates upper bound for regularization param for each label based on estimation from http://jmlr.org/papers/volume8/koh07a/koh07a.pdf
data
Dataframewith samples.
featuresColumn
Name of the features column
labelColumn
Name of the labels column.
regulaizeLast
Whenever to consider last feature as a subject for regularization (set to false to exclude intercept from regularization)
returns
A pair with instances count and regularization bounds.
def f2jBLAS: BLAS

Definition Classes
HasNetlibBlas
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
def hashCode(): Int

Definition Classes
AnyRef → Any
def initializeLogIfNecessary(isInterpreter: Boolean): Unit

Attributes
protected
Definition Classes
Logging
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
def isTraceEnabled(): Boolean

Attributes
protected
Definition Classes
Logging
def log: Logger

Attributes
protected
Definition Classes
Logging
def logDebug(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logDebug(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logInfo(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logInfo(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logName: String

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logWarning(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logWarning(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def multiClassLBFGS(data: DataFrame, featuresColumn: String, labelColumn: String, numCorrections: Int, convergenceTol: Double, maxNumIterations: Int, batchSize: Int, regParam: Double = 0.0, regulaizeLast: Boolean = true): Map[String, Vector]

Implementation of the matrix LBFGS algorithm.
Implementation of the matrix LBFGS algorithm. Uses breeze implementation of the iterations and provides it with a specific cost function. The function batches requests for costs for different labels and converts to a single matrix pass.
data
Data fram to run on.
featuresColumn
Name of the column with features vector. Attribute group metadata is required
labelColumn
Name of the column with labels vector. Attribute group metadata is required
numCorrections
Number of corrections in LBFGS iteration
convergenceTol
Convergence tolerance for the iteration
maxNumIterations
Maximum number of iteration
batchSize
Number of samples to batch before calculating
returns
Map label -> trained weights vector
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
AnyRef → Any
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

Related Docs: class MatrixLBFGS | package odkl

object MatrixLBFGS extends Logging with HasNetlibBlas with Serializable

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

final def asInstanceOf[T0]: T0

def axpy(a: Double, x: Vector, y: Array[Double]): Unit

def axpy(a: Double, x: Array[Double], y: Array[Double]): Unit

def blas: BLAS

def clone(): AnyRef

def computeGradient(data: Vector, label: Vector, weights: DenseMatrix, accumulatedGradient: DenseMatrix, accumulatedLoss: DenseVector): Unit

def computeGradientAndLoss[T](data: RDD[(Vector, T)], currentWeights: DenseMatrix, batchSize: Int = 10, labelsAssigner: (Int, T, Array[Double]) ⇒ Unit): (DenseMatrix, DenseVector)

def computeGradientMatrix(data: Array[Double], label: Array[Double], weights: DenseMatrix, accumulatedGradient: DenseMatrix, accumulatedLoss: DenseVector, marginCache: Array[Double], samples: Int): Unit

def copy(x: Array[Double], y: Array[Double]): Unit

def dscal(a: Double, data: Array[Double]): Unit

final def eq(arg0: AnyRef): Boolean

def equals(arg0: Any): Boolean

def evaluateMaxRegularization(data: RDD[(Vector, Vector)], regulaizeLast: Boolean, numFeatures: Int, labelsMean: DenseVector, numExamples: Long): Vector

def evaluateMaxRegularization(data: DataFrame, featuresColumn: String, labelColumn: String, regulaizeLast: Boolean): (Long, Vector)

def f2jBLAS: BLAS

def finalize(): Unit

final def getClass(): Class[_]

def hashCode(): Int

def initializeLogIfNecessary(isInterpreter: Boolean): Unit

final def isInstanceOf[T0]: Boolean

def isTraceEnabled(): Boolean

def log: Logger

def logDebug(msg: ⇒ String, throwable: Throwable): Unit

def logDebug(msg: ⇒ String): Unit

def logError(msg: ⇒ String, throwable: Throwable): Unit

def logError(msg: ⇒ String): Unit

def logInfo(msg: ⇒ String, throwable: Throwable): Unit

def logInfo(msg: ⇒ String): Unit

def logName: String

def logTrace(msg: ⇒ String, throwable: Throwable): Unit

def logTrace(msg: ⇒ String): Unit

def logWarning(msg: ⇒ String, throwable: Throwable): Unit

def logWarning(msg: ⇒ String): Unit

def multiClassLBFGS(data: DataFrame, featuresColumn: String, labelColumn: String, numCorrections: Int, convergenceTol: Double, maxNumIterations: Int, batchSize: Int, regParam: Double = 0.0, regulaizeLast: Boolean = true): Map[String, Vector]

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

final def synchronized[T0](arg0: ⇒ T0): T0

def toString(): String

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Inherited from Serializable

Inherited from Serializable

Inherited from HasNetlibBlas

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Ungrouped