Memory intensive AND/OR search for combinatorial optimization in graphical models

In this paper we explore the impact of caching during search in the context of the recent framework of AND/OR search in graphical models. Specifically, we extend the depth-first AND/OR Branch-and-Bound tree search algorithm to explore an AND/OR search graph by equipping it with an adaptive caching scheme similar to good and no-good recording. Furthermore, we present best-first search algorithms for traversing the same underlying AND/OR search graph and compare both algorithms empirically. We focus on two common optimization problems in graphical models: finding the Most Probable Explanation (MPE) in belief networks and solving Weighted CSPs (WCSP). In an extensive empirical evaluation we demonstrate conclusively the superiority of the memory intensive AND/OR search algorithms on a variety of benchmarks.

[1]  Rina Dechter,et al.  Bayesian Inference in the Presence of Determinism , 2003, AISTATS.

[2]  Rina Dechter,et al.  Mini-buckets: A general scheme for bounded inference , 2003, JACM.

[3]  Simon de Givry,et al.  Existential arc consistency: Getting closer to full arc consistency in weighted CSPs , 2005, IJCAI.

[4]  Alberto Martelli,et al.  Additive AND/OR Graphs , 1973, IJCAI.

[5]  Thomas Schiex,et al.  MendelSoft: Mendelian error detection in complex pedigree using weighted constraint satisfaction techniques. , 2006 .

[6]  Philippe Jégou,et al.  Decomposition and Good Recording for Solving Max-CSPs , 2004, ECAI.

[7]  R. Dechter,et al.  Unifying Cluster-Tree Decompositions for Reasoning in Graphical models ∗ , 2005 .

[8]  James D. Park.  Using weighted MAX-SAT engines to solve MPE , 2002, AAAI/IAAI.

[9]  Javier Larrosa,et al.  Pseudo-tree Search with Soft Constraints , 2002, ECAI.

[10]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[11]  Nils J. Nilsson,et al.  Principles of Artificial Intelligence , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Thomas Stützle,et al.  Efficient Stochastic Local Search for MPE Solving , 2005, IJCAI.

[13]  Rina Dechter,et al.  Hybrid Processing of Beliefs and Constraints , 2001, UAI.

[14]  Rina Dechter,et al.  Resolution versus Search: Two Strategies for SAT , 2000, Journal of Automated Reasoning.

[15]  Rina Dechter,et al.  AND/OR Branch-and-Bound for Graphical Models , 2005, IJCAI.

[16]  Jegou Philippe,et al.  Decomposition and good recording for solving Max-CSPs , 2004 .

[17]  Richard E. Korf,et al.  Linear-Space Best-First Search , 1993, Artif. Intell..

[18]  Michael J. Quinn,et al.  Taking Advantage of Stable Sets of Variables in Constraint Satisfaction Problems , 1985, IJCAI.

[19]  Javier Larrosa,et al.  Unifying tree decompositions for reasoning in graphical models , 2005, Artif. Intell..

[20]  Rina Dechter,et al.  And/or search strategies for combinatorial optimization in graphical models , 2008 .

[21]  David Allen,et al.  New Advances in Inference by Recursive Conditioning , 2002, UAI.

[22]  Adnan Darwiche,et al.  Uncertainty in artificial intelligence : proceedings of the nineteenth conference (2003), August 7-10, 2003, Acapulco, Mexico , 2003 .

[23]  Dan Geiger,et al.  Maximum Likelihood Haplotyping for General Pedigrees , 2005, Human Heredity.

[24]  Edward P. K. Tsang,et al.  Guided Local Search for Solving SAT and Weighted MAX-SAT Problems , 2000, Journal of Automated Reasoning.

[25]  Tuomas Sandholm,et al.  An algorithm for optimal winner determination in combinatorial auctions , 1999, IJCAI 1999.

[26]  Rina Dechter,et al.  Bucket Elimination: A Unifying Framework for Reasoning , 1999, Artif. Intell..

[27]  Eric A. Hansen,et al.  External-Memory Pattern Databases Using Structured Duplicate Detection , 2005, AAAI.

[28]  Simon de Givry,et al.  Solving Max-SAT as Weighted CSP , 2003, CP.

[29]  Rina Dechter,et al.  Memory Intensive Branch-and-Bound Search for Graphical Models , 2006, AAAI.

[30]  Dan Geiger,et al.  Exact genetic linkage computations for general pedigrees , 2002, ISMB.

[31]  Henry A. Kautz,et al.  Performing Bayesian Inference by Weighted Model Counting , 2005, AAAI.

[32]  Rina Dechter,et al.  Best-First AND/OR Search for Most Probable Explanations , 2007, UAI.

[33]  Rina Dechter,et al.  Constraint Processing , 1995, Lecture Notes in Computer Science.

[34]  Laveen N. Kanal.  Book review: Search in Artificial Intelligence Ed. by Laveen Kanal and Vipin Kumar (Springer-Verlag, New York, 1988) , 1991, SGAR.

[35]  Rina Dechter,et al.  A general scheme for automatic generation of search heuristics from specification dependencies , 2001, Artif. Intell..

[36]  Eric A. Hansen,et al.  Structured Duplicate Detection in External-Memory Graph Search , 2004, AAAI.

[37]  Simon de Givry,et al.  Radio Link Frequency Assignment , 1999, Constraints.

[38]  Manfred Jaeger,et al.  Compiling relational Bayesian networks for exact inference , 2006, Int. J. Approx. Reason..

[39]  Judea Pearl,et al.  Heuristics : intelligent search strategies for computer problem solving , 1984 .

[40]  Francesca Rossi,et al.  Semiring-based constraint satisfaction and optimization , 1997, JACM.

[41]  Simon de Givry,et al.  Exploiting Tree Decomposition and Soft Local Consistency In Weighted CSP , 2006, AAAI.

[42]  M. Spence,et al.  Analysis of human genetic linkage , 1986 .

[43]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[44]  Gérard Verfaillie,et al.  Earth Observation Satellite Management , 1999, Constraints.

[45]  P. P. Chakrabarti,et al.  Heuristic Search in Restricted Memory , 1989, Artif. Intell..

[46]  Simon de Givry,et al.  Combining constraint processing and pattern matching to describe and locate structured motifs in genomic sequences. , 2005, IJCAI 2005.

[47]  Thomas Schiex,et al.  Russian Doll Search for Solving Constraint Optimization Problems , 1996, AAAI/IAAI, Vol. 1.

[48]  Rina Dechter,et al.  AND/OR Cutset Conditioning , 2005, IJCAI.

[49]  Francesca Rossi,et al.  Semiring-based constraint solving and optimization , 1997 .

[50]  Toby Walsh,et al.  SAT v CSP , 2000, CP.

[51]  Rina Dechter,et al.  AND/OR search spaces for graphical models , 2007, Artif. Intell..

[52]  Daniel P. Miranker,et al.  A Complexity Analysis of Space-Bounded Learning Algorithms for the Constraint Satisfaction Problem , 1996, AAAI/IAAI, Vol. 1.

[53]  Rina Dechter,et al.  Mini-buckets: a general scheme for approximating inference , 2002 .

[54]  Adnan Darwiche,et al.  Recursive conditioning , 2001, Artif. Intell..

[55]  Rina Dechter,et al.  Dynamic Orderings for AND/OR Branch-and-Bound Search in Graphical Models , 2006, ECAI.

[56]  Daniel P. Miranker,et al.  On the Space-Time Trade-off in Solving Constraint Satisfaction Problems , 1995, IJCAI.

[57]  Rina Dechter,et al.  Enhancement Schemes for Constraint Processing: Backjumping, Learning, and Cutset Decomposition , 1990, Artif. Intell..

[58]  Henry A. Kautz,et al.  Solving Bayesian Networks by Weighted Model Counting , 2005 .

[59]  Sharad Malik,et al.  Chaff: engineering an efficient SAT solver , 2001, Proceedings of the 38th Design Automation Conference (IEEE Cat. No.01CH37232).

[60]  Toniann Pitassi,et al.  Value Elimination: Bayesian Interence via Backtracking Search , 2002, UAI.

[61]  John R. Gilbert,et al.  Approximating Treewidth, Pathwidth, and Minimum Elimination Tree Height , 1991, WG.

[62]  Rina Dechter,et al.  Best-First AND/OR Search for Graphical Models , 2007, AAAI.

[63]  Shmuel Katz,et al.  On the Feasibility of Distributed Constraint Satisfaction , 1991, IJCAI.

[64]  Rina Dechter,et al.  AND/OR Branch-and-Bound search for combinatorial optimization in graphical models , 2009, Artif. Intell..

[65]  Rina Dechter,et al.  Generalized best-first search strategies and the optimality of A* , 1985, JACM.

[66]  Rina Dechter,et al.  Mixtures of Deterministic-Probabilistic Networks and their AND/OR Search Space , 2004, UAI.

[67]  Simon de Givry,et al.  Mendelian Error Detection in Complex Pedigrees Using Weighted Constraint Satisfaction Techniques , 2007, Constraints.