Finite Size Corrections and Likelihood Ratio Fluctuations in the Spiked Wigner Model

In this paper we study principal components analysis in the regime of high dimensionality and high noise. Our model of the problem is a rank-one deformation of a Wigner matrix where the signal-to-noise ratio (SNR) is of constant order, and we are interested in the fundamental limits of detection of the spike. Our main goal is to gain a fine understanding of the asymptotics for the log-likelihood ratio process, also known as the free energy, as a function of the SNR. Our main results are twofold. We first prove that the free energy has a finite-size correction to its limit---the replica-symmetric formula---which we explicitly compute. This provides a formula for the Kullback-Leibler divergence between the planted and null models. Second, we prove that below the reconstruction threshold, where it becomes impossible to reconstruct the spike, the log-likelihood ratio has fluctuations of constant order and converges in distribution to a Gaussian under both the planted and (under restrictions) the null model. As a consequence, we provide a general proof of contiguity between these two distributions that holds up to the reconstruction threshold, and is valid for an arbitrary separable prior on the spike. Formulae for the total variation distance, and the Type-I and Type-II errors of the optimal test are also given. Our proofs are based on Gaussian interpolation methods and a rigorous incarnation of the cavity method, as devised by Guerra and Talagrand in their study of the Sherrington--Kirkpatrick spin-glass model.

[1]  Le Cam,et al.  Locally asymptotically normal families of distributions : certain approximations to families of distributions & thier use in the theory of estimation & testing hypotheses , 1960 .

[2]  D. Ruelle,et al.  Some rigorous results on the Sherrington-Kirkpatrick spin glass model , 1987 .

[3]  M. Mézard,et al.  Spin Glass Theory and Beyond , 1987 .

[4]  F. Comets,et al.  The Sherrington-Kirkpatrick model of spin glasses and stochastic calculus: The high temperature case , 1995 .

[5]  G. Parisi,et al.  Recipes for metastable states in spin glasses , 1995 .

[6]  H. Nishimori.  Statistical Physics of Spin Glasses and Information Processing , 2001 .

[7]  I. Johnstone.  On the distribution of the largest eigenvalue in principal components analysis , 2001 .

[8]  西森 秀稔.  Statistical physics of spin glasses and information processing : an introduction , 2001 .

[9]  F. Guerra,et al.  The Thermodynamic Limit in Mean Field Spin Glass Models , 2002, cond-mat/0204280.

[10]  Pisa,et al.  Central limit theorem for fluctuations in the high temperature region of the Sherrington-Kirkpatrick spin glass model , 2002, cond-mat/0201092.

[11]  F. Guerra.  Broken Replica Symmetry Bounds in the Mean Field Spin Glass Model , 2002, cond-mat/0205123.

[12]  M. Aizenman,et al.  Extended variational principle for the Sherrington-Kirkpatrick spin-glass model , 2003 .

[13]  S. Péché,et al.  Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices , 2004, math/0403022.

[14]  J. W. Silverstein,et al.  Eigenvalues of large sample covariance matrices of spiked population models , 2004, math/0408165.

[15]  S. Péché.  The largest eigenvalue of small rank perturbations of Hermitian random matrices , 2004, math/0411487.

[16]  M. Talagrand.  The parisi formula , 2006 .

[17]  D. Paul.  ASYMPTOTICS OF SAMPLE EIGENSTRUCTURE FOR A LARGE DIMENSIONAL SPIKED COVARIANCE MODEL , 2007 .

[18]  D. Féral,et al.  The Largest Eigenvalue of Rank One Deformation of Large Wigner Matrices , 2006, math/0605624.

[19]  M. Wainwright,et al.  High-dimensional analysis of semidefinite relaxations for sparse principal components , 2008, 2008 IEEE International Symposium on Information Theory.

[20]  C. Donati-Martin,et al.  The largest eigenvalues of finite rank deformation of large Wigner matrices: Convergence and nonuniversality of the fluctuations. , 2007, 0706.0136.

[21]  I. Johnstone,et al.  On Consistency and Sparsity for Principal Components Analysis in High Dimensions , 2009, Journal of the American Statistical Association.

[22]  B. Nadler.  Finite sample approximation results for principal component analysis: a matrix perturbation approach , 2009, 0901.3245.

[23]  Satish Babu Korada,et al.  Exact Solution of the Gauge Symmetric p-Spin Glass Model on a Complete Graph , 2009 .

[24]  Raj Rao Nadakuditi,et al.  The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices , 2009, 0910.2120.

[25]  Tim Austin.  Mean field models for spin glasses , 2012 .

[26]  Alexei Onatski,et al.  Signal detection in high dimension: The multispiked case , 2012, 1210.5663.

[27]  P. Rigollet,et al.  Optimal detection of sparse principal components in high dimension , 2012, 1202.5070.

[28]  Marcelo J. Moreira,et al.  Asymptotic power of sphericity tests for high-dimensional data , 2013, 1306.4867.

[29]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[30]  Andrea Montanari,et al.  Information-theoretically optimal sparse PCA , 2014, 2014 IEEE International Symposium on Information Theory.

[31]  R. Handel.  Probability in High Dimension , 2014 .

[32]  Florent Krzakala,et al.  Phase transitions in sparse PCA , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[33]  J. Baik,et al.  Fluctuations of the Free Energy of the Spherical Sherrington–Kirkpatrick Model , 2015, Journal of Statistical Physics.

[34]  J. Baik,et al.  Fluctuations of the Free Energy of the Spherical Sherrington–Kirkpatrick Model with Ferromagnetic Interaction , 2016, Annales Henri Poincaré.

[35]  Florent Krzakala,et al.  Mutual information in rank-one matrix estimation , 2016, 2016 IEEE Information Theory Workshop (ITW).

[36]  Andrea Montanari,et al.  Asymptotic mutual information for the binary stochastic block model , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[37]  E. Dobriban,et al.  Sharp detection in PCA under correlations: all eigenvalues matter , 2016, 1602.06896.

[38]  Ankur Moitra,et al.  Optimality and Sub-optimality of PCA for Spiked Random Matrices and Synchronization , 2016, ArXiv.

[39]  Nicolas Macris,et al.  Mutual information for symmetric rank-one matrix estimation: A proof of the replica formula , 2016, NIPS.

[40]  Marc Lelarge,et al.  Fundamental limits of symmetric low-rank matrix estimation , 2016, Probability Theory and Related Fields.

[41]  Florent Krzakala,et al.  Constrained low-rank matrix estimation: phase transitions, approximate message passing and applications , 2017, ArXiv.

[42]  Jess Banks,et al.  Information-theoretic bounds and phase transitions in clustering, sparse PCA, and submatrix localization , 2016, 2017 IEEE International Symposium on Information Theory (ISIT).