Generalized SURE for optimal shrinkage of singular values in low-rank matrix denoising

We consider the problem of estimating a low-rank signal matrix from noisy measurements under the assumption that the distribution of the data matrix belongs to an exponential family. In this setting, we derive generalized Stein's unbiased risk estimation (SURE) formulas that hold for any spectral estimators which shrink or threshold the singular values of the data matrix. This leads to new data-driven spectral estimators, whose optimality is discussed using tools from random matrix theory and through numerical experiments. Under the spiked population model and in the asymptotic setting where the dimensions of the data matrix are let going to infinity, some theoretical properties of our approach are compared to recent results on asymptotically optimal shrinking rules for Gaussian noise. It also leads to new procedures for singular values shrinkage in finite-dimensional matrix denoising for Gamma-distributed and Poisson-distributed measurements.

[1]  L. Pastur,et al.  Matrices aléatoires: Statistique asymptotique des valeurs propres , 2003 .

[2]  J. W. Silverstein,et al.  Spectral Analysis of Large Dimensional Random Matrices , 2009 .

[3]  Gonzalo Mateos,et al.  Inference of Poisson count processes using low-rank tensor data , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[4]  Olivier Ledoit,et al.  Nonlinear Shrinkage Estimation of Large-Dimensional Covariance Matrices , 2011, 1207.5322.

[5]  J. W. Silverstein,et al.  Eigenvalues of large sample covariance matrices of spiked population models , 2004, math/0408165.

[6]  H. Akaike.  A new look at the statistical model identification , 1974 .

[7]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[8]  Adrian S. Lewis,et al.  Twice Differentiable Spectral Functions , 2001, SIAM J. Matrix Anal. Appl..

[9]  Yonina C. Eldar.  Generalized SURE for Exponential Families: Applications to Regularization , 2008, IEEE Transactions on Signal Processing.

[10]  B. Efron.  The Estimation of Prediction Error , 2004 .

[11]  D. Paul.  ASYMPTOTICS OF SAMPLE EIGENSTRUCTURE FOR A LARGE DIMENSIONAL SPIKED COVARIANCE MODEL , 2007 .

[12]  B. Efron,et al.  Empirical Bayes on vector observations: An extension of Stein's method , 1972 .

[13]  R. Tibshirani,et al.  Selecting the number of principal components: estimation of the true rank of a noisy matrix , 2014, 1410.8260.

[14]  Defeng Sun.  Nonsmooth Matrix Valued Functions Defined by Singular Values , 2002 .

[15]  J. W. Silverstein,et al.  On the empirical distribution of eigenvalues of large dimensional information-plus-noise-type matrices , 2007 .

[16]  T. Yanagimoto.  The Kullback-Leibler risk of the Stein estimator and the conditional MLE , 1994 .

[17]  Stephen P. Boyd,et al.  Generalized Low Rank Models , 2014, Found. Trends Mach. Learn..

[18]  Charles R. Johnson,et al.  Topics in Matrix Analysis , 1991 .

[19]  Yang Cao,et al.  Poisson Matrix Recovery and Completion , 2015, IEEE Transactions on Signal Processing.

[20]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[21]  Michael E. Wall,et al.  SVDMAN-singular value decomposition analysis of microarray data , 2001, Bioinform..

[22]  Jiang Du,et al.  Noise reduction in multiple-echo data sets using singular value decomposition. , 2006, Magnetic resonance imaging.

[23]  J. Goodman.  Some fundamental properties of speckle , 1976 .

[24]  Charles-Alban Deledalle,et al.  Estimation of Kullback-Leibler losses for noisy recovery problems within the exponential family , 2015, 1512.08191.

[25]  David L. Donoho,et al.  The Optimal Hard Threshold for Singular Values is 4/sqrt(3) , 2013, 1305.5870.

[26]  Jean Lafond,et al.  Low Rank Matrix Completion with Exponential Family Noise , 2015, COLT.

[27]  A. Girard.  A fast ‘Monte-Carlo cross-validation’ procedure for large least squares problems with noisy data , 1989 .

[28]  D. Botstein,et al.  Singular value decomposition for genome-wide expression data processing and modeling. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[29]  Raj Rao Nadakuditi,et al.  The singular values and vectors of low rank perturbations of large rectangular random matrices , 2011, J. Multivar. Anal..

[30]  Haipeng Shen,et al.  Analysis of call centre arrival data using singular value decomposition , 2005 .

[31]  David L. Donoho,et al.  Optimal Shrinkage of Singular Values , 2014, IEEE Transactions on Information Theory.

[32]  C. Stein.  Estimation of the Mean of a Multivariate Normal Distribution , 1981 .

[33]  B. Efron,et al.  Multivariate Empirical Bayes and Estimation of Covariance Matrices , 1976 .

[34]  F. Ulaby,et al.  Handbook of radar scattering statistics for terrain , 1989 .

[35]  Raj Rao Nadakuditi,et al.  OptShrink: An Algorithm for Improved Low-Rank Signal Matrix Denoising by Optimal, Data-Driven Singular Value Shrinkage , 2013, IEEE Transactions on Information Theory.

[36]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[37]  H. Hudson.  A Natural Identity for Exponential Families with Applications in Multiparameter Estimation , 1978 .

[38]  L. Brown.  Fundamentals of statistical exponential families: with applications in statistical decision theory , 1986 .

[39]  Norbert Schuff,et al.  Denoising diffusion-weighted MR magnitude image sequences using low rank and edge constraints , 2012, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI).

[40]  Julie Josse,et al.  Adaptive shrinkage of singular values , 2013, Statistics and Computing.

[41]  D. Donoho,et al.  Minimax risk of matrix denoising by singular value thresholding , 2013, 1304.2085.

[42]  P. Hall.  On Kullback-Leibler loss and density estimation , 1987 .

[43]  Minh N. Do,et al.  Spatiotemporal denoising of MR spectroscopic imaging data by low-rank approximations , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[44]  Thierry Blu,et al.  Monte-Carlo Sure: A Black-Box Optimization of Regularization Parameters for General Denoising Algorithms , 2008, IEEE Transactions on Image Processing.

[45]  Eero P. Simoncelli,et al.  Learning to be Bayesian without Supervision , 2006, NIPS.

[46]  Jan Hannig,et al.  On Poisson signal estimation under Kullback-Leibler discrepancy and squared risk , 2006 .

[47]  Emmanuel J. Candès,et al.  Unbiased Risk Estimates for Singular Value Thresholding and Spectral Estimators , 2012, IEEE Transactions on Signal Processing.

[48]  Mohamed-Jalal Fadili,et al.  Risk estimation for matrix recovery with spectral regularization , 2012, ICML 2012.

[49]  Andrew B. Nobel,et al.  Reconstruction of a low-rank matrix in the presence of Gaussian noise , 2010, J. Multivar. Anal..