smile.imputation

Class SVDImputation

java.lang.Object

smile.imputation.SVDImputation

All Implemented Interfaces:

MissingValueImputation

public class SVDImputation
extends Object
implements MissingValueImputation

Missing value imputation with singular value decomposition. Given SVD A = U Σ V^T, we use the most significant eigenvectors of V^T to linearly estimate missing values. Although it has been shown that several significant eigenvectors are sufficient to describe the data with small errors, the exact fraction of eigenvectors best for estimation needs to be determined empirically. Once k most significant eigenvectors from V^T are selected, we estimate a missing value j in row i by first regressing this row against the k eigenvectors and then use the coefficients of the regression to reconstruct j from a linear combination of the k eigenvectors. The j th value of row i and the j th values of the k eigenvectors are not used in determining these regression coefficients. It should be noted that SVD can only be performed on complete matrices; therefore we originally fill all missing values by other methods in matrix A, obtaining A'. We then utilize an expectation maximization method to arrive at the final estimate, as follows. Each missing value in A is estimated using the above algorithm, and then the procedure is repeated on the newly obtained matrix, until the total change in the matrix falls below the empirically determined threshold (say 0.01).

Author:

Haifeng Li

Constructor Summary

Constructors

Constructor and Description
SVDImputation(int k) Constructor.

Method Summary

Methods

Modifier and Type	Method and Description
void	impute(double[][] data) Impute missing values in the dataset.
void	impute(double[][] data, int maxIter) Impute missing values in the dataset.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SVDImputation

public SVDImputation(int k)

Constructor.

Parameters:

k - the number of eigenvectors used for imputation.

Method Detail

impute

public void impute(double[][] data)
            throws MissingValueImputationException

Description copied from interface: MissingValueImputation

Impute missing values in the dataset.

Specified by:

impute in interface MissingValueImputation

Parameters:

data - a data set with missing values (represented as Double.NaN). On output, missing values are filled with estimated values.

Throws:

MissingValueImputationException

impute

public void impute(double[][] data,
          int maxIter)
            throws MissingValueImputationException

Impute missing values in the dataset.

Parameters:

data - a data set with missing values (represented as Double.NaN). On output, missing values are filled with estimated values.

maxIter - the maximum number of iterations.

Throws:

MissingValueImputationException