Graphical Models as Block-Tree Graphs

We introduce block-tree graphs as a framework for deriving efficient algorithms on graphical models. We define block-tree graphs as a tree-structured graph where each node is a cluster of nodes such that the clusters in the graph are disjoint. This differs from junction-trees, where two clusters connected by an edge always have at least one common node. When compared to junction-trees, we show that constructing block-tree graphs is faster, and finding optimal block-tree graphs has a much smaller search space. Applying our block-tree graph framework to graphical models, we show that, for some graphs, e.g., grid graphs, using block-tree graphs for inference is computationally more efficient than using junction-trees. For graphical models with boundary conditions, the block-tree graph framework transforms the boundary valued problem into an initial value problem. For Gaussian graphical models, the block-tree graph framework leads to a linear state-space representation. Since exact inference in graphical models can be computationally intractable, we propose to use spanning block-trees to derive approximate inference algorithms. Experimental results show the improved performance in using spanning block-trees versus using spanning trees for approximate estimation over Gaussian graphical models.

[1]  Steffen L. Lauritzen,et al.  Bayesian updating in causal probabilistic networks by local computations , 1990 .

[2]  Rina Dechter,et al.  Bucket Elimination: A Unifying Framework for Reasoning , 1999, Artif. Intell..

[3]  José M. F. Moura,et al.  Telescoping Recursive Representations and Estimation of Gauss–Markov Random Fields , 2009, IEEE Transactions on Information Theory.

[4]  Martin J. Wainwright,et al.  Tree-based reparameterization framework for analysis of sum-product and related algorithms , 2003, IEEE Trans. Inf. Theory.

[5]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[6]  Nevin L. Zhang,et al.  A simple approach to Bayesian network computations , 1994 .

[7]  Nikhil Balram,et al.  Recursive structure of noncausal Gauss-Markov random fields , 1992, IEEE Trans. Inf. Theory.

[8]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[9]  Richard G. Baraniuk,et al.  Robust Distributed Estimation Using the Embedded Subgraphs Algorithm , 2006, IEEE Transactions on Signal Processing.

[10]  Dafna Shahaf,et al.  Learning Thin Junction Trees via Graph Cuts , 2009, AISTATS.

[11]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[12]  JOHN w. WOODS,et al.  Kalman filtering in two dimensions , 1977, IEEE Trans. Inf. Theory.

[13]  Venkat Chandrasekaran,et al.  Estimation in Gaussian Graphical Models Using Tractable Subgraphs: A Walk-Sum Analysis , 2008, IEEE Transactions on Signal Processing.

[14]  J. Besag,et al.  On the estimation and testing of spatial interaction in Gaussian lattice processes , 1975 .

[15]  Thomas Kailath,et al.  A further note on backwards Markovian models (Corresp.) , 1979, IEEE Trans. Inf. Theory.

[16]  A. Hasman,et al.  Probabilistic reasoning in intelligent systems: Networks of plausible inference , 1991 .

[17]  Rama Chellappa,et al.  Texture synthesis using 2-D noncausal autoregressive models , 1985, IEEE Trans. Acoust. Speech Signal Process..

[18]  K. C. Chou,et al.  Multiscale systems, Kalman filters, and Riccati equations , 1994, IEEE Trans. Autom. Control..

[19]  R. Prim.  Shortest connection networks and some generalizations , 1957 .

[20]  K. C. Chou,et al.  Multiscale recursive estimation, data fusion, and regularization , 1994, IEEE Trans. Autom. Control..

[21]  Prakash P. Shenoy,et al.  Probability propagation , 1990, Annals of Mathematics and Artificial Intelligence.

[22]  Martin J. Wainwright,et al.  MAP estimation via agreement on (hyper)trees: Message-passing and linear programming , 2005, ArXiv.

[23]  A. Willsky,et al.  Solution and linear estimation of 2-D nearest-neighbor models , 1990 .

[24]  Martin J. Wainwright,et al.  Embedded trees: estimation of Gaussian Processes on graphs with cycles , 2004, IEEE Transactions on Signal Processing.

[25]  N. Wermuth,et al.  Graphical Models for Associations between Variables, some of which are Qualitative and some Quantitative , 1989 .

[26]  Derek G. Corneil,et al.  Complexity of finding embeddings in a k -tree , 1987 .

[27]  H. Künsch.  Gaussian Markov random fields , 1979 .

[28]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[29]  W. Freeman,et al.  Bethe free energy, Kikuchi approximations, and belief propagation algorithms , 2001 .

[30]  Martin J. Wainwright,et al.  MAP estimation via agreement on trees: message-passing and linear programming , 2005, IEEE Transactions on Information Theory.

[31]  Paul D. Seymour,et al.  Graph Minors. II. Algorithmic Aspects of Tree-Width , 1986, J. Algorithms.

[32]  Arie M. C. A. Koster,et al.  Treewidth computations I. Upper bounds , 2010, Inf. Comput..

[33]  大西 仁,et al.  Pearl, J. (1988, second printing 1991). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan-Kaufmann. , 1994 .

[34]  Gregory F. Cooper,et al.  NESTOR: A Computer-Based Medical Diagnostic Aid That Integrates Causal and Probabilistic Knowledge. , 1984 .

[35]  Vladimir Kolmogorov,et al.  Convergent Tree-Reweighted Message Passing for Energy Minimization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Frank Jensen,et al.  Optimal junction Trees , 1994, UAI.

[37]  David Madigan,et al.  An Alternative Markov Property for Chain Graphs , 1996, UAI.

[38]  M. Frydenberg.  The chain graph Markov property , 1990 .

[39]  Venkat Chandrasekaran,et al.  Adaptive Embedded Subgraph Algorithms using Walk-Sum Analysis , 2007, NIPS.

[40]  J. Kruskal.  On the shortest spanning subtree of a graph and the traveling salesman problem , 1956 .

[41]  L. Scharf,et al.  Statistical Signal Processing: Detection, Estimation, and Time Series Analysis , 1991 .

[42]  Michael I. Jordan,et al.  Loopy Belief Propagation for Approximate Inference: An Empirical Study , 1999, UAI.

[43]  Dmitry M. Malioutov,et al.  Walk-Sums and Belief Propagation in Gaussian Graphical Models , 2006, J. Mach. Learn. Res..

[44]  Rama Chellappa,et al.  Estimation and choice of neighbors in spatial-interaction models of images , 1983, IEEE Trans. Inf. Theory.

[45]  José M. F. Moura,et al.  Gauss-Markov random fields (CMrf) with continuous indices , 1997, IEEE Trans. Inf. Theory.

[46]  Martin J. Wainwright,et al.  Stochastic processes on graphs with cycles: geometric and variational approaches , 2002 .

[47]  Yun Peng,et al.  Plausibility of Diagnostic Hypotheses: The Nature of Simplicity , 1986, AAAI.

[48]  Gregory F. Cooper,et al.  The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks , 1990, Artif. Intell..

[49]  F. Cozman,et al.  Generalizing variable elimination in Bayesian networks , 2000 .