Estimation in the Spiked Wigner Model: A Short Proof of the Replica Formula

We consider the problem of estimating the rank-one perturbation of a Wigner matrix in a setting of low signal-to-noise ratio. This serves as a simple model for principal component analysis in high dimensions. The mutual information per variable between the spike and the observed matrix, or equivalently, the normalized Kullback-Leibler divergence between the planted and null models are known to converge to the so-called replica-symmetric formula, the properties of which determine the fundamental limits of estimation in this model. We provide in this note a short and transparent proof of this formula, based on simple executions of Gaussian interpolations and standard concentration-of-measure arguments. The Franz-Parisi potential, that is, the free entropy at a fixed overlap, plays an important role in our proof. Furthermore, our proof can be generalized straightforwardly to spiked tensor models of even order.

[1]  Florent Krzakala,et al.  Phase transitions in sparse PCA , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[2]  Nicolas Macris,et al.  Mutual information for symmetric rank-one matrix estimation: A proof of the replica formula , 2016, NIPS.

[3]  Marc Lelarge,et al.  Fundamental limits of symmetric low-rank matrix estimation , 2016, Probability Theory and Related Fields.

[4]  F. Guerra,et al.  The Thermodynamic Limit in Mean Field Spin Glass Models , 2002, cond-mat/0204280.

[5]  S. Péché.  The largest eigenvalue of small rank perturbations of Hermitian random matrices , 2004, math/0411487.

[6]  Michael I. Jordan,et al.  Finite Size Corrections and Likelihood Ratio Fluctuations in the Spiked Wigner Model , 2017, ArXiv.

[7]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[8]  Giorgio Parisi,et al.  EFFECTIVE POTENTIAL IN GLASSY SYSTEMS : THEORY AND SIMULATIONS , 1998 .

[9]  Andrea Montanari,et al.  Asymptotic mutual information for the binary stochastic block model , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[10]  Florent Krzakala,et al.  Mutual information in rank-one matrix estimation , 2016, 2016 IEEE Information Theory Workshop (ITW).

[11]  Nicolas Macris,et al.  The stochastic interpolation method: A simple scheme to prove replica formulas in Bayesian inference , 2017, ArXiv.

[12]  Satish Babu Korada,et al.  Exact Solution of the Gauge Symmetric p-Spin Glass Model on a Complete Graph , 2009 .

[13]  西森 秀稔.  Statistical physics of spin glasses and information processing : an introduction , 2001 .

[14]  M. Aizenman,et al.  Extended variational principle for the Sherrington-Kirkpatrick spin-glass model , 2003 .

[15]  Nicolas Macris,et al.  Phase Transitions, Optimal Errors and Optimality of Message-Passing in Generalized Linear Models , 2017, ArXiv.

[16]  N. Macris,et al.  The adaptive interpolation method: a simple scheme to prove replica formulas in Bayesian inference , 2018, Probability Theory and Related Fields.

[17]  Andrea Montanari,et al.  A statistical model for tensor PCA , 2014, NIPS.

[18]  Tim Austin.  Mean field models for spin glasses , 2012 .

[19]  G. Parisi,et al.  Recipes for metastable states in spin glasses , 1995 .

[20]  Florent Krzakala,et al.  Statistical and computational phase transitions in spiked tensor estimation , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[21]  Nicolas Macris,et al.  The layered structure of tensor estimation and its mutual information , 2017, 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[22]  Raj Rao Nadakuditi,et al.  The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices , 2009, 0910.2120.

[23]  I. Johnstone.  On the distribution of the largest eigenvalue in principal components analysis , 2001 .

[24]  S. Péché,et al.  Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices , 2004, math/0403022.