The overlap gap property and approximate message passing algorithms for $p$-spin models

We consider the algorithmic problem of finding a near ground state (near optimal solution) of a $p$-spin model. We show that for a class of algorithms broadly defined as Approximate Message Passing (AMP), the presence of the Overlap Gap Property (OGP), appropriately defined, is a barrier. We conjecture that when $p\ge 4$ the model does indeed exhibits OGP (and prove it for the space of binary solutions). Assuming the validity of this conjecture, as an implication, the AMP fails to find near ground states in these models, per our result. We extend our result to the problem of finding pure states by means of Thouless, Anderson and Palmer (TAP) based iterations, which is yet another example of AMP type algorithms. We show that such iterations fail to find pure states approximately, subject to the conjecture that the space of pure states exhibits the OGP, appropriately stated, when $p\ge 4$.

[1]  M. Mézard,et al.  Analytic and Algorithmic Solution of Random Satisfiability Problems , 2002, Science.

[2]  Aukosh Jagannath,et al.  Some properties of the phase diagram for mixed p-spin glasses , 2015, 1504.02731.

[3]  Roman Vershynin,et al.  High-Dimensional Probability , 2018 .

[4]  Andrea Montanari,et al.  Optimization of the Sherrington-Kirkpatrick Hamiltonian , 2018, 2019 IEEE 60th Annual Symposium on Foundations of Computer Science (FOCS).

[5]  Wei-Kuo Chen,et al.  On the energy landscape of the mixed even p-spin model , 2016, 1609.04368.

[6]  Andrea Montanari,et al.  The dynamics of message passing on dense graphs, with applications to compressed sensing , 2010, ISIT.

[7]  David Gamarnik,et al.  Computing the Partition Function of the Sherrington-Kirkpatrick Model is Hard on Average , 2018, 2020 IEEE International Symposium on Information Theory (ISIT).

[8]  Eliran Subag.  Following the Ground States of Full‐RSB Spherical Spin Glasses , 2018 .

[9]  David Gamarnik,et al.  Finding a large submatrix of a Gaussian random matrix , 2016, The Annals of Statistics.

[10]  Amin Coja-Oghlan,et al.  On independent sets in random graphs , 2010, SODA '11.

[11]  A. COJA-OGHLAN,et al.  Walksat Stalls Well Below Satisfiability , 2016, SIAM J. Discret. Math..

[12]  Andrea Montanari,et al.  Universality in Polytope Phase Transitions and Message Passing Algorithms , 2012, ArXiv.

[13]  Aukosh Jagannath,et al.  A Dynamic Programming Approach to the Parisi Functional , 2015, 1502.04398.

[14]  E. Bolthausen.  An Iterative Construction of Solutions of the TAP Equations for the Sherrington–Kirkpatrick Model , 2012, 1201.2891.

[15]  R. Adler,et al.  Random Fields and Geometry , 2007 .

[16]  Wei-Kuo Chen,et al.  The SK model is Full-step Replica Symmetry Breaking at zero temperature , 2017 .

[17]  Antonio Auffinger,et al.  On properties of Parisi measures , 2013, 1303.3573.

[18]  D. Panchenko.  The Sherrington-Kirkpatrick Model , 2013 .

[19]  David Gamarnik,et al.  The overlap gap property in principal submatrix recovery , 2019, Probability Theory and Related Fields.

[20]  Wei-Kuo Chen,et al.  Disorder chaos in some diluted spin glass models , 2017, The Annals of Applied Probability.

[21]  Y. Kabashima.  A CDMA multiuser detection algorithm on the basis of belief propagation , 2003 .

[22]  Madhu Sudan,et al.  Limits of local algorithms over sparse random graphs , 2013, ITCS.

[23]  Bálint Virág,et al.  Local algorithms for independent sets are half-optimal , 2014, ArXiv.

[24]  Dmitry Panchenko,et al.  Suboptimality of local algorithms for a class of max-cut problems , 2017, The Annals of Probability.

[25]  Aukosh Jagannath,et al.  Low Temperature Asymptotics of Spherical Mean Field Spin Glasses , 2016, 1602.00657.

[26]  Adel Javanmard,et al.  State Evolution for General Approximate Message Passing Algorithms, with Applications to Spatial Coupling , 2012, ArXiv.

[27]  G. B. Arous,et al.  Algorithmic thresholds for tensor PCA , 2018, The Annals of Probability.

[28]  G. Ben Arous,et al.  Spectral Gap Estimates in Mean Field Spin Glasses , 2017, 1705.04243.

[29]  M. Mézard,et al.  Spin Glass Theory and Beyond , 1987 .

[30]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[31]  Pascal Maillard,et al.  The algorithmic hardness threshold for continuous random energy models , 2018, Mathematical Statistics and Learning.

[32]  R. Palmer,et al.  Solution of 'Solvable model of a spin glass' , 1977 .

[33]  Andrea Montanari,et al.  Optimization of mean-field spin glasses , 2020, The Annals of Probability.

[34]  Amin Coja-Oghlan,et al.  Algorithmic Barriers from Phase Transitions , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[35]  David Gamarnik,et al.  High Dimensional Regression with Binary Coefficients. Estimating Squared Error and a Phase Transtition , 2017, COLT.

[36]  Aukosh Jagannath,et al.  Bounds on the complexity of Replica Symmetry Breaking for spherical spin glasses , 2016, 1607.02134.

[37]  O. Papaspiliopoulos.  High-Dimensional Probability: An Introduction with Applications in Data Science , 2020 .

[38]  David Gamarnik,et al.  Low-Degree Hardness of Random Optimization Problems , 2020, 2020 IEEE 61st Annual Symposium on Foundations of Computer Science (FOCS).

[39]  Wei-Kuo Chen,et al.  Parisi Formula, Disorder Chaos and Fluctuation for the Ground State Energy in the Spherical Mixed p-Spin Models , 2015, 1512.08492.

[40]  G. Teschl.  Ordinary Differential Equations and Dynamical Systems , 2012 .

[41]  Madhu Sudan,et al.  Performance of Sequential Local Algorithms for the Random NAE-K-SAT Problem , 2017, SIAM J. Comput..

[42]  Thierry Mora,et al.  Clustering of solutions in the random satisfiability problem , 2005, Physical review letters.

[43]  M. Talagrand.  Free energy of the spherical mean field model , 2006 .

[44]  Andrea Montanari,et al.  State Evolution for Approximate Message Passing with Non-Separable Functions , 2017, Information and Inference: A Journal of the IMA.

[45]  Reza Gheissari,et al.  Bounding Flows for Spherical Spin Glass Dynamics , 2018, Communications in Mathematical Physics.