Pushing the Power of Stochastic Greedy Ordering Schemes for Inference in Graphical Models

We study iterative randomized greedy algorithms for generating (elimination) orderings with small induced width and state space size - two parameters known to bound the complexity of inference in graphical models. We propose and implement the Iterative Greedy Variable Ordering (IGVO) algorithm, a new variant within this algorithm class. An empirical evaluation using different ranking functions and conditions of randomness, demonstrates that IGVO finds significantly better orderings than standard greedy ordering implementations when evaluated within an anytime framework. Additional order of magnitude improvements are demonstrated on a multi-core system, thus further expanding the set of solvable graphical models. The experiments also confirm the superiority of the MinFill heuristic within the iterative scheme.

[1]  Yair Weiss,et al.  Approximate Inference and Protein-Folding , 2002, NIPS.

[2]  Rina Dechter,et al.  Mini-buckets: a general scheme for approximating inference , 2002 .

[3]  Adnan Darwiche,et al.  Recursive conditioning , 2001, Artif. Intell..

[4]  Rina Dechter,et al.  Experimental Evaluation of Preprocessing Techniques in Constraint Satisfaction Problems , 1989, IJCAI.

[5]  Jacques Carlier,et al.  Heuristic and metaheuristic methods for computing graph treewidth , 2004, RAIRO Oper. Res..

[6]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[7]  B. Mohar,et al.  Graph Minors , 2009 .

[8]  Pedro Larrañaga,et al.  Decomposing Bayesian networks: triangulation of the moral graph with genetic algorithms , 1997, Stat. Comput..

[9]  Paul D. Seymour,et al.  Graph minors. I. Excluding a forest , 1983, J. Comb. Theory, Ser. B.

[10]  Andrew Gelfand,et al.  BEEM : Bucket Elimination with External Memory , 2010, UAI.

[11]  Vibhav Gogate,et al.  AND/OR Importance Sampling , 2008, UAI.

[12]  Uffe Kjærulff.  Optimal decomposition of probabilistic networks by simulated annealing , 1992 .

[13]  Derek G. Corneil,et al.  Complexity of finding embeddings in a k -tree , 1987 .

[14]  Rina Dechter,et al.  AND/OR search spaces for graphical models , 2007, Artif. Intell..

[15]  Hans L. Bodlaender,et al.  Kernelization: New Upper and Lower Bound Techniques , 2009, IWPEC.

[16]  Vibhav Gogate,et al.  A Complete Anytime Algorithm for Treewidth , 2004, UAI.

[17]  Toniann Pitassi,et al.  Value Elimination: Bayesian Interence via Backtracking Search , 2002, UAI.

[18]  Rina Dechter,et al.  Mini-buckets: A general scheme for bounded inference , 2003, JACM.

[19]  Dan Geiger,et al.  Optimizing exact genetic linkage computations , 2003, RECOMB '03.

[20]  Hans L. Bodlaender,et al.  Treewidth: Structure and Algorithms , 2007, SIROCCO.