Exact and Approximate Recursive Calculations for Binary Markov Random Fields Defined on Graphs

In this article, we propose computationally feasible approximations to binary Markov random fields (MRFs). The basis of the approximation is the forward-backward algorithm. The exact forward-backward algorithm is computationally feasible only for fields defined on small lattices. The forward part of the algorithm computes a series of joint marginal distributions by summing out each variable in turn. We represent these joint marginal distributions by interaction parameters of different orders. The approximation is defined by approximating to zero all interaction parameters that are sufficiently close to zero. In addition, an interaction parameter is approximated to zero whenever all associated lower-level interactions are (approximated to) zero. If sufficiently many interaction parameters are set to zero, the algorithm is computationally feasible both in terms of computation time and memory requirements. The resulting approximate forward part of the forward-backward algorithm defines an approximation to the intractable normalizing constant, and the corresponding backward part of the algorithm defines a computationally feasible approximation to the MRF. We present numerical examples demonstrating the quality of the approximation. The supplementary materials for this article, which are available online, include two appendices and R and C codes for the proposed recursive algorithms.

[1]  Francesco Bartolucci,et al.  A recursive algorithm for Markov random fields , 2002 .

[2]  Anthony N. Pettitt,et al.  Efficient calculation of the normalizing constant of the autologistic and related models on the cylinder and lattice , 2003 .

[3]  S. T. Buckland,et al.  An autologistic model for the spatial distribution of wildlife , 1996 .

[4]  F. Huang,et al.  Generalized Pseudo-Likelihood Estimates for Markov Random Fields on Lattice , 2002 .

[5]  D. M. Titterington,et al.  Computational Bayesian Analysis of Hidden Markov Models , 1998 .

[6]  Anthony N. Pettitt,et al.  Bayesian Inference in Hidden Markov Random Fields for Binary Data Defined on Large Lattices , 2009 .

[7]  Xiao-Li Meng,et al.  Simulating Normalizing Constants: From Importance Sampling to Bridge Sampling to Path Sampling , 1998 .

[8]  Håvard Rue,et al.  Recursive computing and simulation-free inference for general factorizable models , 2007 .

[9]  J. Laurie Snell,et al.  Markov Random Fields and Their Applications , 1980 .

[10]  Mike Rees,et al.  5. Statistics for Spatial Data , 1993 .

[11]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[12]  Jennifer A. Hoeting,et al.  An Improved Model for Spatially Correlated Binary Responses , 2000 .

[13]  H. Rue.  Fast sampling of Gaussian Markov random fields , 2000 .

[14]  S. L. Scott.  Bayesian Methods for Hidden Markov Models , 2002 .

[15]  J. Besag.  On the Statistical Analysis of Dirty Pictures , 1986 .

[16]  Peter Clifford,et al.  Markov Random Fields in Statistics , 2012 .

[17]  David Bruce Wilson,et al.  Exact sampling with coupled Markov chains and applications to statistical mechanics , 1996, Random Struct. Algorithms.

[18]  Robert Haining,et al.  Statistics for spatial data: by Noel Cressie, 1991, John Wiley & Sons, New York, 900 p., ISBN 0-471-84336-9, US $89.95 , 1993 .

[19]  Håkon Tjelmeland,et al.  Markov Random Fields with Higher‐order Interactions , 1998 .

[20]  J. Møller,et al.  An efficient Markov chain Monte Carlo method for distributions with intractable normalising constants , 2006 .

[21]  Hans Kiinsch,et al.  State Space and Hidden Markov Models , 2000 .

[22]  Goodman,et al.  Multigrid Monte Carlo method. Conceptual foundations. , 1989, Physical review. D, Particles and fields.

[23]  Iain Murray.  Advances in Markov chain Monte Carlo methods , 2007 .

[24]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[25]  J. Heikkinen,et al.  Fully Bayesian Approach to Image Restoration with an Application in Biogeography , 1994 .

[26]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[27]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[28]  E GriffinJim,et al.  Advances in Markov chain Monte Carlo , 2013 .

[29]  Gerhard Tutz,et al.  State Space and Hidden Markov Models , 2001 .

[30]  J. Besag,et al.  Bayesian image restoration, with two applications in spatial statistics , 1991 .

[31]  Anthony N. Pettitt,et al.  Efficient recursions for general factorisable models , 2004 .

[32]  David A. Elston,et al.  Empirical models for the spatial distribution of wildlife , 1993 .

[33]  C. Geyer,et al.  Annealing Markov chain Monte Carlo with applications to ancestral inference , 1995 .

[34]  N. Cressie,et al.  Image analysis with partially ordered Markov models , 1998 .