Statistical inferences of linear forms for noisy matrix completion

We introduce a flexible framework for making inferences about general linear forms of a large matrix based on noisy observations of a subset of its entries. In particular, under mild regularity conditions, we develop a universal procedure to construct asymptotically normal estimators of its linear forms through double-sample debiasing and low-rank projection whenever an entry-wise consistent estimator of the matrix is available. These estimators allow us to subsequently construct confidence intervals for and test hypotheses about the linear forms. Our proposal was motivated by a careful perturbation analysis of the empirical singular spaces under the noisy matrix completion model which might be of independent interest.

[1]  Yuling Yan,et al.  Noisy Matrix Completion: Understanding Statistical Guarantees for Convex Relaxation via Nonconvex Optimization , 2019, SIAM J. Optim..

[2]  Yuxin Chen,et al.  Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval, Matrix Completion, and Blind Deconvolution , 2017, Found. Comput. Math..

[3]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[4]  Dong Xia,et al.  Optimal estimation of low rank density matrices , 2015, J. Mach. Learn. Res..

[5]  Arlene K. H. Kim,et al.  An iterative hard thresholding estimator for low rank matrix recovery with explicit limiting distribution , 2015, 1502.04654.

[6]  Chandler Davis.  The rotation of eigenvectors by a perturbation , 1963 .

[7]  R. Nickl,et al.  Uncertainty Quantification for Matrix Compressed Sensing and Quantum Tomography Problems , 2015, Progress in Probability.

[8]  W. Kahan,et al.  The Rotation of Eigenvectors by a Perturbation. III , 1970 .

[9]  Adel Javanmard,et al.  Confidence intervals and hypothesis testing for high-dimensional regression , 2013, J. Mach. Learn. Res..

[10]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[11]  J. Robins,et al.  Double/Debiased Machine Learning for Treatment and Structural Parameters , 2017 .

[12]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[13]  Yi-Kai Liu,et al.  Universal low-rank matrix recovery from Pauli measurements , 2011, NIPS.

[14]  Andrea Montanari,et al.  Matrix Completion from Noisy Entries , 2009, J. Mach. Learn. Res..

[15]  Cun-Hui Zhang,et al.  Calibrated Elastic Regularization in Matrix Completion , 2012, NIPS.

[16]  V. Koltchinskii,et al.  Nuclear norm penalization and optimal rates for noisy low rank matrix completion , 2010, 1011.6256.

[17]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[18]  Zhaoran Wang,et al.  A Nonconvex Optimization Framework for Low Rank Matrix Estimation , 2015, NIPS.

[19]  Anru Zhang,et al.  ROP: Matrix Recovery via Rank-One Projections , 2013, ArXiv.

[20]  Martin J. Wainwright,et al.  Estimation of (near) low-rank matrices with noise and high-dimensional scaling , 2009, ICML.

[21]  David Gross,et al.  Recovering Low-Rank Matrices From Few Coefficients in Any Basis , 2009, IEEE Transactions on Information Theory.

[22]  Dong Xia.  Data-dependent Confidence Regions of Singular Subspaces , 2019, ArXiv.

[23]  S. Geer,et al.  On asymptotically optimal confidence regions and tests for high-dimensional models , 2013, 1303.0518.

[24]  T. Tony Cai,et al.  Matrix completion via max-norm constrained optimization , 2013, ArXiv.

[25]  T. Tony Cai,et al.  Confidence intervals for high-dimensional linear regression: Minimax rates and adaptivity , 2015, 1506.05539.

[26]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, 2009 IEEE International Symposium on Information Theory.

[27]  Martin J. Wainwright,et al.  Fast low-rank estimation by projected gradient descent: General statistical and algorithmic guarantees , 2015, ArXiv.

[28]  C. Esseen,et al.  A moment inequality with an application to the central limit theorem , 1956 .

[29]  Cun-Hui Zhang,et al.  Confidence intervals for low dimensional parameters in high dimensional linear models , 2011, 1110.2563.

[30]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[31]  P. Wedin.  Perturbation bounds in connection with singular value decomposition , 1972 .

[32]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[33]  John D. Lafferty,et al.  Convergence Analysis for Rectangular Matrix Completion Using Burer-Monteiro Factorization and Gradient Descent , 2016, ArXiv.

[34]  A. Pajor.  Metric Entropy of the Grassmann Manifold , 1998 .

[35]  O. Klopp.  Noisy low-rank matrix completion with general sampling distribution , 2012, 1203.0108.

[36]  R. Nickl,et al.  Adaptive confidence sets for matrix completion , 2016, Bernoulli.

[37]  Kenneth Y. Goldberg,et al.  Eigentaste: A Constant Time Collaborative Filtering Algorithm , 2001, Information Retrieval.

[38]  Tengyuan Liang,et al.  Geometric Inference for General High-Dimensional Linear Inverse Problems , 2014, 1404.4408.

[39]  Anru Zhang,et al.  Structured Matrix Completion with Applications to Genomic Data Integration , 2015, Journal of the American Statistical Association.

[40]  Yihong Wu,et al.  Volume ratio, sparsity, and minimaxity under unitarily invariant norms , 2013, 2013 IEEE International Symposium on Information Theory.

[41]  Yuling Yan,et al.  Inference and uncertainty quantification for noisy matrix completion , 2019, Proceedings of the National Academy of Sciences.

[42]  Xiaodong Li,et al.  Nonconvex Rectangular Matrix Completion via Gradient Descent without $\ell_{2,\infty}$ Regularization , 2019 .

[43]  Xiao Zhang,et al.  A Unified Computational and Statistical Framework for Nonconvex Low-rank Matrix Estimation , 2016, AISTATS.

[44]  Tengyu Ma,et al.  Matrix Completion has No Spurious Local Minimum , 2016, NIPS.

[45]  Ji Chen,et al.  Nonconvex Rectangular Matrix Completion via Gradient Descent Without ℓ₂,∞ Regularization , 2020, IEEE Transactions on Information Theory.

[46]  Joel A. Tropp,et al.  User-Friendly Tail Bounds for Sums of Random Matrices , 2010, Found. Comput. Math..

[47]  Chao Gao,et al.  Optimal Estimation and Completion of Matrices with Biclustering Structures , 2015, J. Mach. Learn. Res..

[48]  Dong Xia.  Normal approximation and confidence region of singular subspaces , 2021, Electronic Journal of Statistics.

[49]  A. C. Berry.  The accuracy of the Gaussian approximation to the sum of independent variates , 1941 .

[50]  Stanislav Minsker.  On Some Extensions of Bernstein's Inequality for Self-adjoint Operators , 2011, 1112.5448.

[51]  Frank R. Kschischang,et al.  2018 IEEE Information Theory Society Paper Award , 2019, IEEE Trans. Inf. Theory.

[52]  A. Tsybakov,et al.  Estimation of high-dimensional low-rank matrices , 2009, 0912.5338.

[53]  V. Koltchinskii.  Von Neumann Entropy Penalization and Low Rank Matrix Estimation , 2010, 1009.2439.

[54]  Ming Yuan,et al.  On Polynomial Time Methods for Exact Low-Rank Tensor Completion , 2017, Found. Comput. Math..