User-Friendly Tail Bounds for Sums of Random Matrices

This paper presents new probability inequalities for sums of independent, random, self-adjoint matrices. These results place simple and easily verifiable hypotheses on the summands, and they deliver strong conclusions about the large-deviation behavior of the maximum eigenvalue of the sum. Tail bounds for the norm of a sum of random rectangular matrices follow as an immediate corollary. The proof techniques also yield some information about matrix-valued martingales.In other words, this paper provides noncommutative generalizations of the classical bounds associated with the names Azuma, Bennett, Bernstein, Chernoff, Hoeffding, and McDiarmid. The matrix inequalities promise the same diversity of application, ease of use, and strength of conclusion that have made the scalar inequalities so valuable.

[1]  C. McDiarmid.  Concentration , 1862, The Dental register.

[2]  H. Chernoff.  A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations , 1952 .

[3]  E. Lieb.  Convex trace functions and the Wigner-Yanase-Dyson conjecture , 1973 .

[4]  H. Epstein.  Remarks on two theorems of E. Lieb , 1973 .

[5]  G. Lindblad.  Expectations and entropy inequalities for finite quantum systems , 1974 .

[6]  N. Tomczak-Jaegermann.  The moduli of smoothness and convexity and the Rademacher averages of the trace classes $S_{p}$ (1≤p<∞) , 1974 .

[7]  D. Freedman.  On Tail Probabilities for Martingales , 1975 .

[8]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[9]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[10]  Y. Gordon.  Some inequalities for Gaussian processes and applications , 1985 .

[11]  V. Paulsen.  Completely bounded maps and dilations , 1987 .

[12]  M. Talagrand.  An isoperimetric theorem on the cube and the Kintchine-Kahane inequalities , 1988 .

[13]  Charles R. Johnson,et al.  Topics in Matrix Analysis , 1991 .

[14]  M. Talagrand,et al.  Probability in Banach Spaces: Isoperimetry and Processes , 1991 .

[15]  G. Pisier,et al.  Non commutative Khintchine and Paley inequalities , 1991 .

[16]  B. M. Fulk.  MATH , 1992 .

[17]  Y. Gordon.  Majorization of Gaussian processes and geometric applications , 1992 .

[18]  D. Petz.  A survey of certain trace inequalities , 1994 .

[19]  Rajeev Motwani,et al.  Randomized algorithms , 1996, CSUR.

[20]  R. Bhatia.  Matrix Analysis , 1996 .

[21]  M. Rudelson.  Random Vectors in the Isotropic Position , 1996, math/9608208.

[22]  G. Pisier,et al.  Non-Commutative Martingale Inequalities , 1997, math/9704209.

[23]  M. Ledoux.  On Talagrand's deviation inequalities for product measures , 1997 .

[24]  Amir Dembo,et al.  Large Deviations Techniques and Applications , 1998 .

[25]  G. Lugosi,et al.  On Concentration-of-Measure Inequalities , 1998 .

[26]  E. Giné,et al.  Decoupling: From Dependence to Independence , 1998 .

[27]  M. Habib.  Probabilistic methods for algorithmic discrete mathematics , 1998 .

[28]  Yoav Seginer,et al.  The Expected Norm of Random Matrices , 2000, Combinatorics, Probability and Computing.

[29]  B. Carl,et al.  An Elementary Approach to an Eigenvalue Estimate for Matrices , 2000 .

[30]  A. Buchholz.  Operator Khintchine inequality in non-commutative probability , 2001 .

[31]  Dimitris Achlioptas,et al.  Fast computation of low rank matrix approximations , 2001, STOC '01.

[32]  S. Szarek,et al.  Chapter 8 - Local Operator Theory, Random Matrices and Banach Spaces , 2001 .

[33]  M. Ruskai.  Inequalities for quantum entropy: A review with conditions for equality , 2002, quant-ph/0205064.

[34]  Rudolf Ahlswede,et al.  Strong converse for identification via quantum channels , 2000, IEEE Trans. Inf. Theory.

[35]  Rudolf Ahlswede,et al.  Addendum to "Strong converse for identification via quantum channels" , 2003, IEEE Trans. Inf. Theory.

[36]  V. Paulsen.  Completely Bounded Maps and Operator Algebras: Completely Bounded Multilinear Maps and the Haagerup Tensor Norm , 2003 .

[37]  M. Junge,et al.  Noncommutative Burkholder/Rosenthal inequalities , 2003 .

[38]  F. Hansen,et al.  Jensen's Operator Inequality , 2002, math/0204049.

[39]  R. Lata,et al.  SOME ESTIMATES OF NORMS OF RANDOM MATRICES , 2004 .

[40]  M. Ruskai.  Erratum: Inequalities for quantum entropy: A review with conditions for equality [J. Math. Phys. 43, 4358 (2002)] , 2005 .

[41]  A. Buchholz.  Optimal Constants in Khintchine Type Inequalities for Fermions, Rademachers and q-Gaussian Operators , 2005 .

[42]  M. Junge,et al.  On the Best Constants in Some Non‐Commutative Martingale Inequalities , 2005, math/0505309.

[43]  D. Spielman,et al.  Smoothed Analysis of the Condition Numbers and Growth Factors of Matrices , 2003, SIAM Journal on Matrix Analysis and Applications.

[44]  Mark Rudelson,et al.  Sampling from large matrices: An approach through geometric functional analysis , 2005, JACM.

[45]  E. Candès,et al.  Sparsity and incoherence in compressive sampling , 2006, math/0611957.

[46]  R. Bhatia.  Positive Definite Matrices , 2007 .

[47]  Arkadi Nemirovski,et al.  Sums of random symmetric matrices and quadratic optimization under orthogonality constraints , 2007, Math. Program..

[48]  M. Junge,et al.  Noncommutative Burkholder/Rosenthal inequalities II: Applications , 2007, 0705.1952.

[49]  Nicholas J. Higham,et al.  Functions of matrices - theory and computation , 2008 .

[50]  Klas Markström,et al.  Expansion properties of random Cayley graphs and vertex transitive graphs via matrix martingales , 2008, Random Struct. Algorithms.

[51]  N. Higham.  Functions of Matrices: Theory and Computation (Other Titles in Applied Mathematics) , 2008 .

[52]  J. Tropp.  On the conditioning of random subdictionaries , 2008 .

[53]  K. Markström,et al.  Expansion properties of random Cayley graphs and vertex transitive graphs via matrix martingales , 2008 .

[54]  Bernard Chazelle,et al.  The Fast Johnson--Lindenstrauss Transform and Approximate Nearest Neighbors , 2009, SIAM J. Comput..

[55]  E. Effros.  A matrix convexity approach to some celebrated quantum inequalities , 2008, Proceedings of the National Academy of Sciences.

[56]  M. Rudelson,et al.  Non-asymptotic theory of random matrices: extreme singular values , 2010, 1003.2990.

[57]  R. Oliveira.  Concentration of the adjacency matrix and of the Laplacian in random graphs with independent edges , 2009, 0911.0600.

[58]  R. Oliveira.  Sums of random Hermitian matrices and an inequality by Rudelson , 2010, 1004.3821.

[59]  Joel A. Tropp,et al.  From joint convexity of quantum relative entropy to a concavity theorem of Lieb , 2011, ArXiv.

[60]  J. Tropp.  User-Friendly Tail Bounds for Matrix Martingales , 2011 .

[61]  Joel A. Tropp,et al.  Improved Analysis of the subsampled Randomized Hadamard Transform , 2010, Adv. Data Sci. Adapt. Anal..

[62]  J. Tropp.  FREEDMAN'S INEQUALITY FOR MATRIX MARTINGALES , 2011, 1101.3039.

[63]  David Gross,et al.  Recovering Low-Rank Matrices From Few Coefficients in Any Basis , 2009, IEEE Transactions on Information Theory.

[64]  Anthony Man-Cho So,et al.  Moment inequalities for sums of random matrices and their applications in optimization , 2011, Math. Program..

[65]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[66]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[67]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[68]  Benjamin Recht,et al.  A Simpler Approach to Matrix Completion , 2009, J. Mach. Learn. Res..

[69]  W. Marsden.  I and J , 2012 .

[70]  隆志 佐野.  Rajendra Bhatia: Positive Definite Matrices, Princeton Ser. Appl. Math., Princeton Univ. Press, 2007年，x+254ページ． , 2013 .