Scaling up Hybrid Probabilistic Inference with Logical and Arithmetic Constraints via Message Passing

Weighted model integration (WMI) is a very appealing framework for probabilistic inference: it allows to express the complex dependencies of real-world problems where variables are both continuous and discrete, via the language of Satisfiability Modulo Theories (SMT), as well as to compute probabilistic queries with complex logical and arithmetic constraints. Yet, existing WMI solvers are not ready to scale to these problems. They either ignore the intrinsic dependency structure of the problem at all, or they are limited to too restrictive structures. To narrow this gap, we derive a factorized formalism of WMI enabling us to devise a scalable WMI solver based on message passing, MP-WMI. Namely, MP-WMI is the first WMI solver which allows to: 1) perform exact inference on the full class of tree-structured WMI problems; 2) compute all marginal densities in linear time; 3) amortize inference inter query. Experimental results show that our solver dramatically outperforms the existing WMI solvers on a large set of benchmarks.

[1]  Dan Klein,et al.  Learning Accurate, Compact, and Interpretable Tree Annotation , 2006, ACL.

[2]  N. Wermuth,et al.  Graphical Models for Associations between Variables, some of which are Qualitative and some Quantitative , 1989 .

[3]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[4]  Alex Zelikovsky,et al.  Improved Steiner tree approximation in graphs , 2000, SODA '00.

[5]  Henry A. Kautz,et al.  Performing Bayesian Inference by Weighted Model Counting , 2005, AAAI.

[6]  Andrea Passerini,et al.  Efficient Weighted Model Integration via SMT-Based Predicate Abstraction , 2017, IJCAI.

[7]  Paolo Morettin,et al.  Learning Weighted Model Integration Distributions , 2020, AAAI.

[8]  Cesare Tinelli,et al.  Handbook of Satisfiability , 2021, Handbook of Satisfiability.

[9]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[10]  Pradeep Ravikumar,et al.  Mixed Graphical Models via Exponential Families , 2014, AISTATS.

[11]  Prakash P. Shenoy,et al.  A Comparison of Lauritzen-Spiegelhalter, Hugin, and Shenoy-Shafer Architectures for Computing Marginals of Probability Distributions , 1998, UAI.

[12]  Luc De Raedt,et al.  Exact and Approximate Weighted Model Integration with Probability Density Functions Using Knowledge Compilation , 2019, AAAI.

[13]  Yordan Zaykov,et al.  TrueSkill 2: An improved Bayesian skill rating system , 2018 .

[14]  Devavrat Shah,et al.  Belief propagation for min-cost network flow: convergence & correctness , 2010, SODA '10.

[15]  Vibhav Gogate,et al.  Approximate Inference Algorithms for Hybrid Bayesian Networks with Discrete Constraints , 2005, UAI.

[16]  Ronald A. DeVore,et al.  Fast computation in adaptive tree approximation , 2004, Numerische Mathematik.

[17]  David Heckerman,et al.  Learning Bayesian Networks: A Unification for Discrete and Gaussian Domains , 1995, UAI.

[18]  Paul Penfield,et al.  Signal Delay in RC Tree Networks , 1981, 18th Design Automation Conference.

[19]  Guy Van den Broeck,et al.  Probabilistic Program Abstractions , 2017, UAI.

[20]  Shweta Shinde,et al.  A model counter for constraints over unbounded strings , 2014, PLDI.

[21]  Guy Van den Broeck,et al.  Sound Abstraction and Decomposition of Probabilistic Programs , 2018, ICML.

[22]  W E Vesely,et al.  Fault Tree Handbook , 1987 .

[23]  Guy Van den Broeck,et al.  Component Caching in Hybrid Domains with Piecewise Polynomial Densities , 2016, AAAI.

[24]  Prakash P. Shenoy,et al.  Inference in hybrid Bayesian networks using mixtures of polynomials , 2011, Int. J. Approx. Reason..

[25]  Luc De Raedt,et al.  How to Exploit Structure while Solving Weighted Model Integration Problems , 2019, UAI.

[26]  Ciaran O'Reilly,et al.  Probabilistic Inference Modulo Theories , 2016, IJCAI.

[27]  Adnan Darwiche,et al.  On probabilistic inference by weighted model counting , 2008, Artif. Intell..

[28]  Linda C. van der Gaag,et al.  Probabilistic Graphical Models , 2014, Lecture Notes in Computer Science.

[29]  Guy Van den Broeck,et al.  Efficient Search-Based Weighted Model Integration , 2019, UAI.

[30]  A. Oskooi Molecular Evolution and Phylogenetics , 2008 .

[31]  Nicholas Ruozzi,et al.  Marginal Inference in Continuous Markov Random Fields Using Mixtures , 2019, AAAI.

[32]  Marko Samer,et al.  Algorithms for propositional model counting , 2007, J. Discrete Algorithms.

[33]  Martin E. Dyer,et al.  On the Complexity of Computing the Volume of a Polyhedron , 1988, SIAM J. Comput..

[34]  Kristian Kersting,et al.  Automatic Bayesian Density Analysis , 2018, AAAI.

[35]  Aws Albarghouthi,et al.  Weighted Model Integration with Orthogonal Transformations , 2017, IJCAI.

[36]  Guy Van den Broeck,et al.  Probabilistic Inference in Hybrid Domains by Weighted Model Integration , 2015, IJCAI.

[37]  Luc De Raedt,et al.  The pywmi Framework and Toolbox for Probabilistic Inference using Weighted Model Integration , 2019, IJCAI.

[38]  Scott Sanner,et al.  Efficient Symbolic Integration for Probabilistic Inference , 2018, IJCAI.

[39]  Dilin Wang,et al.  Stein Variational Message Passing for Continuous Graphical Models , 2017, ICML.

[40]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[41]  L. H.,et al.  Communication Networks , 1936, Nature.

[42]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[43]  Cesare Tinelli,et al.  The SMT-LIB Initiative and the Rise of SMT - (HVC 2010 Award Talk) , 2010, Haifa Verification Conference.

[44]  Andrea Passerini,et al.  Advanced SMT techniques for weighted model integration , 2019, Artif. Intell..

[45]  Cesare Tinelli,et al.  Satisfiability Modulo Theories , 2021, Handbook of Satisfiability.

[46]  Kristian Kersting,et al.  Mixed Sum-Product Networks: A Deep Architecture for Hybrid Domains , 2018, AAAI.

[47]  Adnan Darwiche,et al.  Inference in belief networks: A procedural guide , 1996, Int. J. Approx. Reason..

[48]  Rina Dechter,et al.  AND/OR search spaces for graphical models , 2007, Artif. Intell..