Random perturbation and matrix sparsification and completion

We discuss general perturbation inequalities when the perturbation is random. As applications, we obtain several new results concerning two important problems: matrix sparsification and matrix completion.

[1]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[2]  Nisheeth K. Vishnoi,et al.  Lx = b , 2013, Found. Trends Theor. Comput. Sci..

[3]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[4]  Van H. Vu,et al.  Eigenvectors of random matrices: A survey , 2016, J. Comb. Theory A.

[5]  Prateek Jain,et al.  Low-rank matrix completion using alternating minimization , 2012, STOC '13.

[6]  David P. Woodruff.  Sketching as a Tool for Numerical Linear Algebra , 2014, Found. Trends Theor. Comput. Sci..

[7]  Shang-Hua Teng,et al.  Spectral Sparsification of Graphs , 2008, SIAM J. Comput..

[8]  Sanjeev Arora,et al.  A Fast Random Sampling Algorithm for Sparsifying Matrices , 2006, APPROX-RANDOM.

[9]  Dimitris Achlioptas,et al.  Fast computation of low rank matrix approximations , 2001, STOC '01.

[10]  Jianqing Fan,et al.  ENTRYWISE EIGENVECTOR ANALYSIS OF RANDOM MATRICES WITH LOW EXPECTED RANK. , 2017, Annals of statistics.

[11]  Andrea Montanari,et al.  Matrix Completion from Noisy Entries , 2009, J. Mach. Learn. Res..

[12]  V. N. Bogaevski,et al.  Matrix Perturbation Theory , 1991 .

[13]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[14]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[15]  J. Bouchaud,et al.  Eigenvector dynamics under free addition , 2013, 1301.4939.

[16]  Shang-Hua Teng,et al.  Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems , 2003, STOC '04.

[17]  Nikhil Srivastava,et al.  Graph sparsification by effective resistances , 2008, SIAM J. Comput..

[18]  Benjamin Recht,et al.  A Simpler Approach to Matrix Completion , 2009, J. Mach. Learn. Res..

[19]  Chandler Davis.  The rotation of eigenvectors by a perturbation , 1963 .

[20]  Martin J. Wainwright,et al.  Fast low-rank estimation by projected gradient descent: General statistical and algorithmic guarantees , 2015, ArXiv.

[21]  Weichen Wang,et al.  An $\ell_{\infty}$ Eigenvector Perturbation Bound and Its Application , 2017, J. Mach. Learn. Res..

[22]  Robert D. Nowak,et al.  High-Rank Matrix Completion , 2012, AISTATS.

[23]  Rongrong Wang,et al.  Singular Vector Perturbation Under Gaussian Noise , 2012, SIAM J. Matrix Anal. Appl..

[24]  Mikhail Belkin,et al.  Unperturbed: spectral analysis beyond Davis-Kahan , 2017, ALT.

[25]  V. Vu,et al.  Random perturbation of low rank matrices: Improving classical bounds , 2013, 1311.2657.

[26]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[27]  Dong Xia,et al.  Perturbation of linear forms of singular vectors under Gaussian noise , 2015 .

[28]  Alex Gittens,et al.  Error Bounds for Random Matrix Approximation Schemes , 2009, 0911.4108.

[29]  R. Latala,et al.  The dimension-free structure of nonhomogeneous random matrices , 2017, Inventiones mathematicae.

[30]  Van H. Vu.  Singular vectors under random perturbation , 2011, Random Struct. Algorithms.

[31]  Anru R. Zhang,et al.  Rate-Optimal Perturbation Bounds for Singular Subspaces with Applications to High-Dimensional Statistics , 2016, 1605.00353.

[32]  W. Kahan,et al.  The Rotation of Eigenvectors by a Perturbation. III , 1970 .

[33]  P. Wedin.  Perturbation bounds in connection with singular value decomposition , 1972 .

[34]  Raj Rao Nadakuditi,et al.  The singular values and vectors of low rank perturbations of large rectangular random matrices , 2011, J. Multivar. Anal..

[35]  Shang-Hua Teng,et al.  Spectral sparsification of graphs: theory and algorithms , 2013, CACM.

[36]  S. Chatterjee,et al.  Matrix estimation by Universal Singular Value Thresholding , 2012, 1212.1247.

[37]  Nam H. Nguyen,et al.  Matrix sparsification via the Khintchine inequality , 2010 .

[38]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, 2009 IEEE International Symposium on Information Theory.

[39]  Petros Drineas,et al.  A note on element-wise matrix sparsification via a matrix-valued Bernstein inequality , 2010, Inf. Process. Lett..

[40]  Martin J. Wainwright,et al.  Restricted strong convexity and weighted matrix completion: Optimal bounds with noise , 2010, J. Mach. Learn. Res..

[41]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[42]  Raj Rao Nadakuditi,et al.  The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices , 2009, 0910.2120.

[43]  Van H. Vu,et al.  Random weighted projections, random quadratic forms and random eigenvectors , 2013, Random Struct. Algorithms.

[44]  Trac D. Tran,et al.  Tensor sparsification via a bound on the spectral norm of random tensors , 2010, ArXiv.

[45]  Yihua Li,et al.  Blind Regression: Nonparametric Regression for Latent Variable Models via Collaborative Filtering , 2016, NIPS.

[46]  V. Koltchinskii,et al.  Nuclear norm penalization and optimal rates for noisy low rank matrix completion , 2010, 1011.6256.

[47]  Abhisek Kundu,et al.  A Note on Randomized Element-wise Matrix Sparsification , 2014, ArXiv.

[48]  Joel A. Tropp,et al.  An Introduction to Matrix Concentration Inequalities , 2015, Found. Trends Mach. Learn..