Survey propagation at finite temperature: application to a Sourlas code as a toy model

In this paper we investigate a finite temperature generalization of survey propagation, by applying it to the problem of finite temperature decoding of a biased finite connectivity Sourlas code for temperatures lower than the Nishimori temperature. We observe that the result is a shift of the location of the dynamical critical channel noise to larger values than the corresponding dynamical transition for belief propagation, as suggested recently by Migliorini and Saad for LDPC codes. We show how the finite temperature 1RSB SP gives accurate results in the regime where competing approaches fail to converge or fail to recover the retrieval state.

[1]  M. Mézard,et al.  Survey propagation: An algorithm for satisfiability , 2005 .

[2]  R. Zecchina,et al.  Polynomial iterative algorithms for coloring and analyzing random graphs. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[3]  M. Mézard,et al.  The Cavity Method at Zero Temperature , 2002, cond-mat/0207121.

[4]  Andrea Montanari,et al.  Instability of one-step replica-symmetry-broken phase in satisfiability problems , 2003, ArXiv.

[5]  M. Mézard,et al.  Spin Glass Theory and Beyond , 1987 .

[6]  William T. Freeman,et al.  Learning Low-Level Vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[7]  A. Hasman,et al.  Probabilistic reasoning in intelligent systems: Networks of plausible inference , 1991 .

[8]  R. Kikuchi.  A Theory of Cooperative Phenomena , 1951 .

[9]  S. Franz,et al.  Dynamic phase transition for decoding algorithms. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[10]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[11]  Michael Horstein,et al.  Review of 'Low-Density Parity-Check Codes' (Gallager, R. G.; 1963) , 1964, IEEE Transactions on Information Theory.

[12]  Riccardo Zecchina,et al.  Minimizing energy below the glass thresholds. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[13]  Riccardo Zecchina,et al.  Exact solutions for diluted spin glasses and optimization problems , 2001, Physical review letters.

[14]  M. Mézard,et al.  Random K-satisfiability problem: from an analytic solution to an efficient algorithm. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[15]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[16]  B. Derrida.  Random-energy model: An exactly solvable model of disordered systems , 1981 .

[17]  David Saad,et al.  Improved message passing for inference in densely connected systems , 2005, ArXiv.

[18]  R Vicente,et al.  Finite-connectivity systems as error-correcting codes. , 1999, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[19]  Y. Kabashima.  Propagating beliefs in spin-glass models , 2002, cond-mat/0211500.

[20]  D. Thouless,et al.  Stability of the Sherrington-Kirkpatrick solution of a spin glass model , 1978 .

[21]  Yoshiyuki Kabashima.  Replicated Bethe Free Energy: A Variational Principle behind Survey Propagation , 2005 .

[22]  S. Kirkpatrick,et al.  Solvable Model of a Spin-Glass , 1975 .

[23]  O. Nicrosini,et al.  Higher-order QED corrections to single-W production in electron–positron collisions , 2000, hep-ph/0005121.

[24]  Monasson.  Structural glass transition and the entropy of the metastable states. , 1995, Physical Review Letters.

[25]  Jung-Fu Cheng,et al.  Turbo Decoding as an Instance of Pearl's "Belief Propagation" Algorithm , 1998, IEEE J. Sel. Areas Commun..

[26]  Nicolas Sourlas,et al.  Spin-glass models as error-correcting codes , 1989, Nature.

[27]  M. Mézard,et al.  The Bethe lattice spin glass revisited , 2000, cond-mat/0009418.

[28]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[29]  John W. Fisher,et al.  Loopy Belief Propagation: Convergence and Effects of Message Errors , 2005, J. Mach. Learn. Res..

[30]  M. Pretti.  A message-passing algorithm with damping , 2005 .

[31]  Yoshiyuki Kabashima,et al.  Belief propagation vs. TAP for decoding corrupted messages , 1998 .

[32]  Lane P. Hughston,et al.  Disordered and complex systems, London, United Kingdom, 10-14 July 2000 , 2001 .

[33]  M. Mézard,et al.  Analytic and Algorithmic Solution of Random Satisfiability Problems , 2002, Science.

[34]  Kazuyuki Tanaka,et al.  Cluster variation method and image restoration problem , 1995 .

[35]  H. Nishimori.  Statistical Physics of Spin Glasses and Information Processing , 2001 .

[36]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[37]  F. Krzakala,et al.  Spin glass models with ferromagnetically biased couplings on the Bethe lattice: analytic solutions and numerical simulations , 2004, cond-mat/0403053.

[38]  D. Saad,et al.  Statistical mechanics of low-density parity-check codes , 2004 .