LASSO

java.lang.Object
- smile.regression.LASSO

All Implemented Interfaces:

java.io.Serializable, Regression<double[]>
```
public class LASSO
extends java.lang.Object
implements Regression<double[]>, java.io.Serializable
```
Lasso (least absolute shrinkage and selection operator) regression. The Lasso is a shrinkage and selection method for linear regression. It minimizes the usual sum of squared errors, with a bound on the sum of the absolute values of the coefficients (i.e. L₁-regularized). It has connections to soft-thresholding of wavelet coefficients, forward stage-wise regression, and boosting methods.
The Lasso typically yields a sparse solution, of which the parameter vector β has relatively few nonzero coefficients. In contrast, the solution of L₂-regularized least squares (i.e. ridge regression) typically has all coefficients nonzero. Because it effectively reduces the number of variables, the Lasso is useful in some contexts.
For over-determined systems (more instances than variables, commonly in machine learning), we normalize variables with mean 0 and standard deviation 1. For under-determined systems (less instances than variables, e.g. compressed sensing), we assume white noise (i.e. no intercept in the linear model) and do not perform normalization. Note that the solution is not unique in this case.
There is no analytic formula or expression for the optimal solution to the L₁-regularized least squares problems. Therefore, its solution must be computed numerically. The objective function in the L₁-regularized least squares is convex but not differentiable, so solving it is more of a computational challenge than solving the L₂-regularized least squares. The Lasso may be solved using quadratic programming or more general convex optimization methods, as well as by specific algorithms such as the least angle regression algorithm.
References
1. R. Tibshirani. Regression shrinkage and selection via the lasso. J. Royal. Statist. Soc B., 58(1):267-288, 1996.
2. B. Efron, I. Johnstone, T. Hastie, and R. Tibshirani. Least angle regression. Annals of Statistics, 2003
3. Seung-Jean Kim, K. Koh, M. Lustig, Stephen Boyd, and Dimitry Gorinevsky. An Interior-Point Method for Large-Scale L1-Regularized Least Squares. IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. 1, NO. 4, 2007.
See Also:

Serialized Form

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

static class LASSO.Trainer
Trainer for LASSO regression.

Nested Classes
Modifier and Type	Class and Description
`static class`	`LASSO.Trainer` Trainer for LASSO regression.

Constructor Summary

Constructors
Constructor and Description
`LASSO(double[][] x, double[] y, double lambda)` Constructor.
`LASSO(double[][] x, double[] y, double lambda, double tol, int maxIter)` Constructor.
`LASSO(smile.math.matrix.Matrix x, double[] y, double lambda)` Constructor.
`LASSO(smile.math.matrix.Matrix x, double[] y, double lambda, double tol, int maxIter)` Constructor.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`double`	`adjustedRSquared()` Returns adjusted R² statistic.
`double[]`	`coefficients()` Returns the linear coefficients.
`int`	`df()` Returns the degree-of-freedom of residual standard error.
`double`	`error()` Returns the residual standard error.
`double`	`ftest()` Returns the F-statistic of goodness-of-fit.
`double`	`intercept()` Returns the intercept.
`double`	`predict(double[] x)` Predicts the dependent variable of an instance.
`double`	`pvalue()` Returns the p-value of goodness-of-fit test.
`double[]`	`residuals()` Returns the residuals, that is response minus fitted values.
`double`	`RSquared()` Returns R² statistic.
`double`	`RSS()` Returns the residual sum of squares.
`double`	`shrinkage()` Returns the shrinkage parameter.
`java.lang.String`	`toString()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface smile.regression.Regression
predict

- Constructor Detail
  - LASSO
```
public LASSO(double[][] x,
             double[] y,
             double lambda)
```
    Constructor. Learn the L1-regularized least squares model.
    
    Parameters:
    
    x - a matrix containing the explanatory variables. NO NEED to include a constant column of 1s for bias.
    
    y - the response values.
    
    lambda - the shrinkage/regularization parameter.
  - LASSO
```
public LASSO(double[][] x,
             double[] y,
             double lambda,
             double tol,
             int maxIter)
```
    Constructor. Learn the L1-regularized least squares model.
    
    Parameters:
    
    x - a matrix containing the explanatory variables. NO NEED to include a constant column of 1s for bias.
    
    y - the response values.
    
    lambda - the shrinkage/regularization parameter.
    
    tol - the tolerance for stopping iterations (relative target duality gap).
    
    maxIter - the maximum number of IPM (Newton) iterations.
  - LASSO
```
public LASSO(smile.math.matrix.Matrix x,
             double[] y,
             double lambda)
```
    Constructor. Learn the L1-regularized least squares model.
    
    Parameters:
    
    x - a matrix containing the explanatory variables. The variables should be centered and standardized. NO NEED to include a constant column of 1s for bias.
    
    y - the response values.
    
    lambda - the shrinkage/regularization parameter.
  - LASSO
```
public LASSO(smile.math.matrix.Matrix x,
             double[] y,
             double lambda,
             double tol,
             int maxIter)
```
    Constructor. Learn the L1-regularized least squares model.
    
    Parameters:
    
    x - a matrix containing the explanatory variables. The variables should be centered and standardized. NO NEED to include a constant column of 1s for bias.
    
    y - the response values.
    
    lambda - the shrinkage/regularization parameter.
    
    tol - the tolerance for stopping iterations (relative target duality gap).
    
    maxIter - the maximum number of IPM (Newton) iterations.
- Method Detail
  - coefficients
```
public double[] coefficients()
```
    Returns the linear coefficients.
  - intercept
```
public double intercept()
```
    Returns the intercept.
  - shrinkage
```
public double shrinkage()
```
    Returns the shrinkage parameter.
  - predict
```
public double predict(double[] x)
```
    Description copied from interface: Regression
    
    Predicts the dependent variable of an instance.
    
    Specified by:
    
    predict in interface Regression<double[]>
    
    Parameters:
    
    x - the instance.
    
    Returns:
    
    the predicted value of dependent variable.
  - residuals
```
public double[] residuals()
```
    Returns the residuals, that is response minus fitted values.
  - RSS
```
public double RSS()
```
    Returns the residual sum of squares.
  - error
```
public double error()
```
    Returns the residual standard error.
  - df
```
public int df()
```
    Returns the degree-of-freedom of residual standard error.
  - RSquared
```
public double RSquared()
```
    Returns R² statistic. In regression, the R² coefficient of determination is a statistical measure of how well the regression line approximates the real data points. An R² of 1.0 indicates that the regression line perfectly fits the data.
    In the case of ordinary least-squares regression, R² increases as we increase the number of variables in the model (R² will not decrease). This illustrates a drawback to one possible use of R², where one might try to include more variables in the model until "there is no more improvement". This leads to the alternative approach of looking at the adjusted R².
  - adjustedRSquared
```
public double adjustedRSquared()
```
    Returns adjusted R² statistic. The adjusted R² has almost same explanation as R² but it penalizes the statistic as extra variables are included in the model.
  - ftest
```
public double ftest()
```
    Returns the F-statistic of goodness-of-fit.
  - pvalue
```
public double pvalue()
```
    Returns the p-value of goodness-of-fit test.
  - toString
```
public java.lang.String toString()
```
    Overrides:
    
    toString in class java.lang.Object

Class LASSO

References

Nested Class Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Methods inherited from interface smile.regression.Regression

Constructor Detail

LASSO

LASSO

LASSO

LASSO

Method Detail

coefficients

intercept

shrinkage

predict

residuals

RSS

error

df

RSquared

adjustedRSquared

ftest

pvalue

toString