The Bethe approximation for solving the inverse Ising problem: a comparison with other inference methods

The inverse Ising problem consists in inferring the coupling constants of an Ising model given the correlation matrix. The fastest methods for solving this problem are based on mean-field approximations, but which one performs better in the general case is still not completely clear. In the first part of this work, I summarize the formulas for several mean-field approximations and I derive new analytical expressions for the Bethe approximation, which allow one to solve the inverse Ising problem without running the susceptibility propagation algorithm (thus avoiding the lack of convergence). In the second part, I compare the accuracy of different mean-field approximations on several models (diluted ferromagnets and spin glasses) defined on random graphs and regular lattices, showing which one is in general more effective. A simple improvement over these approximations is proposed. Also a fundamental limitation is found in using methods based on TAP and Bethe approximations in the presence of an external field.

[1]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[2]  H. Bethe.  Statistical Theory of Superlattices , 1935 .

[3]  R. Palmer,et al.  Solution of 'Solvable model of a spin glass' , 1977 .

[4]  T. Plefka.  Convergence condition of the TAP equation for the infinite-ranged Ising spin glass model , 1982 .

[5]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[6]  J. Yedidia,et al.  How to expand around mean-field theory using high-temperature expansions , 1991 .

[7]  Hilbert J. Kappen,et al.  Efficient Learning in Boltzmann Machines Using Linear Response Theory , 1998, Neural Computation.

[8]  Toshiyuki TANAKA.  Mean-field theory of Boltzmann machine learning , 1998 .

[9]  M. Mézard,et al.  The Bethe lattice spin glass revisited , 2000, cond-mat/0009418.

[10]  V. Akila,et al.  Information , 2001, The Lancet.

[11]  H. J. Mclaughlin,et al.  Learn , 2002 .

[12]  Alan L. Yuille,et al.  CCCP Algorithms to Minimize the Bethe and Kikuchi Free Energies: Convergent Alternatives to Belief Propagation , 2002, Neural Computation.

[13]  M. Mézard,et al.  The Cavity Method at Zero Temperature , 2002, cond-mat/0207121.

[14]  Yee Whye Teh,et al.  Approximate inference in Boltzmann machines , 2003, Artif. Intell..

[15]  Ericka Stricklin-Parker,et al.  Ann , 2005 .

[16]  A. Montanari,et al.  How to compute loop corrections to the Bethe approximation , 2005, cond-mat/0506769.

[17]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[18]  Hilbert J. Kappen,et al.  Loop Corrections for Approximate Inference on Factor Graphs , 2007, J. Mach. Learn. Res..

[19]  Michael Chertkov,et al.  Approximate Inference on Planar Graphs using Loop Calculus and Belief Propagation , 2009, J. Mach. Learn. Res..

[20]  Riccardo Zecchina,et al.  Inference of sparse combinatorial-control networks from gene-expression data: a message passing approach , 2010, BMC Bioinformatics.

[21]  J. Hertz,et al.  Ising model for neural data: model quality and approximate methods for extracting functional connectivity. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[22]  Erik Aurell,et al.  Frontiers in Computational Neuroscience , 2022 .

[23]  Thierry Mora,et al.  Constraint satisfaction problems and neural networks: A statistical physics perspective , 2008, Journal of Physiology-Paris.

[24]  T. Hwa,et al.  Identification of direct residue contacts in protein–protein interaction by message passing , 2009, Proceedings of the National Academy of Sciences.

[25]  S. Leibler,et al.  Neuronal couplings between retinal ganglion cells inferred by efficient inverse statistical physics methods , 2009, Proceedings of the National Academy of Sciences.

[26]  R. Monasson,et al.  Small-correlation expansions for the inverse Ising problem , 2008, 0811.3574.

[27]  J. Lafferty,et al.  High-dimensional Ising model selection using ℓ1-regularized logistic regression , 2010, 1010.0311.

[28]  E. Marinari,et al.  Intrinsic limitations of the susceptibility propagation inverse inference for the mean field Ising spin glass , 2010 .

[29]  E. Aurell,et al.  Dynamics and performance of susceptibility propagation on synthetic data , 2010, 1005.3694.

[30]  Simona Cocco,et al.  Adaptive Cluster Expansion for Inferring Boltzmann Machines with Noisy Data , 2011, Physical review letters.

[31]  Federico Ricci-Tersenghi,et al.  Characterizing and improving generalized belief propagation algorithms on the 2D Edwards–Anderson model , 2011, ArXiv.

[32]  J. Berg,et al.  Bethe–Peierls approximation and the inverse Ising problem , 2011, 1112.3501.

[33]  J. Cardy,et al.  Logarithmic conformal field theories as limits of ordinary CFTs and some physical applications , 2013, 1302.4279.

[34]  In Proceedings of the Ninth Annual Conference on Uncertainty in Artificial Intelligence ( UAI-93 ) , .