Lower Bounds on the Mean-Squared Error of Low-Rank Matrix Reconstruction

We investigate the behavior of the mean-square error (MSE) of low-rank matrix reconstruction and its special case, matrix completion. We first derive the constrained Cramér-Rao bound (CRB) on the MSE matrix of any locally unbiased estimator, and then analyze the behavior of the constrained CRB when a subset of entries of the underlying matrix is randomly observed. We design an alternating minimization procedure to compute the maximum likelihood estimator (MLE) for the low-rank matrix, and demonstrate through numerical simulations that the performance of the MLE approaches the constrained CRB when the signal-to-noise ratio is high. Applying a Chapman-Robbins type Barankin bound allows us to derive lower bounds on the worst-case scalar MSE. We demonstrate that the worst-case scalar MSE is infinite even if the model is identifiable. However, the infinite scalar MSE is achieved only on a set of low-rank matrices with measure zero. We discuss the implications of these bounds and compare them with the empirical performance of the matrix LASSO estimator and the existing bounds in the literature.

[1]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[2]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[3]  Bernard C. Levy,et al.  Worst-case MSE precoder design for imperfectly known MIMO communications channels , 2005, IEEE Transactions on Signal Processing.

[4]  P. Dooren.  Matrix Mathematics: Theory, Facts, and Formulas with Application to Linear Systems Theory [Book Review] , 2006 .

[5]  M. Rudelson,et al.  Sparse reconstruction by convex relaxation: Fourier and Gaussian measurements , 2006, 2006 40th Annual Conference on Information Sciences and Systems.

[6]  Petre Stoica,et al.  MUSIC, maximum likelihood, and Cramer-Rao bound: further results and comparisons , 1990, IEEE Trans. Acoust. Speech Signal Process..

[7]  E. Candès.  The restricted isometry property and its implications for compressed sensing , 2008 .

[8]  Andreas Karlsson,et al.  Matrix Analysis for Statistics , 2007, Technometrics.

[9]  Yonina C. Eldar,et al.  Robust mean-squared error estimation in the presence of model uncertainties , 2005, IEEE Transactions on Signal Processing.

[10]  Alfred O. Hero,et al.  Lower bounds for parametric estimation with constraints , 1990, IEEE Trans. Inf. Theory.

[11]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[12]  Stephen P. Boyd,et al.  A rank minimization heuristic with application to minimum order system approximation , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[13]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[14]  Jeffrey A. Fessler,et al.  Cramér Rao bound analysis of joint B1/T1 mapping methods in MRI , 2010, 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[15]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[16]  M. Talagrand.  The Generic chaining : upper and lower bounds of stochastic processes , 2005 .

[17]  Yonina C. Eldar,et al.  The Cramér-Rao Bound for Estimating a Sparse Parameter Vector , 2010, IEEE Transactions on Signal Processing.

[18]  Yonina C. Eldar,et al.  Unbiased Estimation of a Sparse Vector in White Gaussian Noise , 2010, IEEE Transactions on Information Theory.

[19]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[20]  Laurent El Ghaoui,et al.  Rank Minimization under LMI constraints: A Framework for Output Feedback Problems , 2007 .

[21]  M. Talagrand,et al.  Probability in Banach Spaces: Isoperimetry and Processes , 1991 .

[22]  D. G. Chapman,et al.  Minimum Variance Estimation Without Regularity Assumptions , 1951 .

[23]  M. Rudelson,et al.  On sparse reconstruction from Fourier and Gaussian measurements , 2008 .

[24]  Thomas L. Marzetta,et al.  A simple derivation of the constrained multiple parameter Cramer-Rao bound , 1993, IEEE Trans. Signal Process..

[25]  J. Hammersley.  On Estimating Restricted Parameters , 1950 .

[26]  Emmanuel J. Candès,et al.  Tight Oracle Inequalities for Low-Rank Matrix Recovery From a Minimal Number of Noisy Random Measurements , 2011, IEEE Transactions on Information Theory.

[27]  Stephen Becker,et al.  Quantum state tomography via compressed sensing. , 2009, Physical review letters.

[28]  Patrick Mitran,et al.  On Bounds and Algorithms for Frequency Synchronization for Collaborative Communication Systems , 2008, IEEE Transactions on Signal Processing.

[29]  Yonina C. Eldar,et al.  On the Constrained CramÉr–Rao Bound With a Singular Fisher Information Matrix , 2009, IEEE Signal Processing Letters.

[30]  B. C. Ng,et al.  On the Cramer-Rao bound under parametric constraints , 1998, IEEE Signal Processing Letters.

[31]  Ronen Basri,et al.  Lambertian reflectance and linear subspaces , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[32]  Yonina C. Eldar.  Minimax MSE estimation of deterministic parameters with noise covariance uncertainties , 2006, IEEE Transactions on Signal Processing.

[33]  Martin D. Buhmann,et al.  Radial Basis Functions: Theory and Implementations: Preface , 2003 .

[34]  Shiqian Ma,et al.  Fixed point and Bregman iterative methods for matrix rank minimization , 2009, Math. Program..

[35]  Edward M. Hofstetter,et al.  Barankin Bounds on Parameter Estimation , 1971, IEEE Trans. Inf. Theory.

[36]  Jürgen Pilz,et al.  Minimax linear regression estimation with symmetric parameter restrictions , 1986 .