Smoothing Structured Decomposable Circuits

We study the task of smoothing a circuit, i.e., ensuring that all children of a plus-gate mention the same variables. Circuits serve as the building blocks of state-of-the-art inference algorithms on discrete probabilistic graphical models and probabilistic programs. They are also important for discrete density estimation algorithms. Many of these tasks require the input circuit to be smooth. However, smoothing has not been studied in its own right yet, and only a trivial quadratic algorithm is known. This paper studies efficient smoothing for structured decomposable circuits. We propose a near-linear time algorithm for this task and explore lower bounds for smoothing decomposable circuits, using existing results on range-sum queries. Further, for the important case of All-Marginals, we show a more efficient linear-time algorithm. We validate experimentally the performance of our methods.

[1]  Andrew Chi-Chih Yao,et al.  Space-time tradeoff for answering range queries (Extended Abstract) , 1982, STOC '82.

[2]  Pedro M. Domingos,et al.  Learning the Structure of Sum-Product Networks , 2013, ICML.

[3]  Guy Van den Broeck,et al.  Probabilistic Sentential Decision Diagrams , 2014, KR.

[4]  Kewei Tu,et al.  Maximum A Posteriori Inference in Sum-Product Networks , 2018, AAAI.

[5]  Rain,et al.  Towards Compact Interpretable Models : Shrinking of Learned Probabilistic Sentential Decision Diagrams , 2017 .

[6]  Adnan Darwiche,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence SDD: A New Canonical Representation of Propositional Knowledge Bases , 2022 .

[7]  Adnan Darwiche,et al.  On the Tractable Counting of Theory Models and its Application to Truth Maintenance and Belief Revision , 2001, J. Appl. Non Class. Logics.

[8]  Franz Pernkopf,et al.  On the Latent Variable Interpretation in Sum-Product Networks , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Adnan Darwiche,et al.  On probabilistic inference by weighted model counting , 2008, Artif. Intell..

[10]  Robert E. Tarjan,et al.  Efficiency of a Good But Not Linear Set Union Algorithm , 1972, JACM.

[11]  Umut Oztok,et al.  A Top-Down Compiler for Sentential Decision Diagrams , 2015, IJCAI.

[12]  Sholeh Forouzan,et al.  Approximate Inference in Graphical Models , 2015 .

[13]  Luc De Raedt,et al.  Inference and learning in probabilistic logic programs using weighted Boolean formulas , 2013, Theory and Practice of Logic Programming.

[14]  Floriana Esposito,et al.  Simplifying, Regularizing and Strengthening Sum-Product Network Structure Learning , 2015, ECML/PKDD.

[15]  Luc De Raedt,et al.  Algebraic model counting , 2012, J. Appl. Log..

[16]  Daniel Lowd,et al.  Learning Sum-Product Networks with Direct and Indirect Variable Interactions , 2014, ICML.

[17]  Fabrizio Riguzzi,et al.  Expectation maximization over binary decision diagrams for probabilistic logic programs , 2013, Intell. Data Anal..

[18]  Guy Van den Broeck,et al.  Skolemization for Weighted First-Order Model Counting , 2013, KR.

[19]  Pedro M. Domingos,et al.  The Sum-Product Theorem: A Foundation for Learning Tractable Models , 2016, ICML.

[20]  Adnan Darwiche,et al.  Tractable Operations for Arithmetic Circuits of Probabilistic Models , 2016, NIPS.

[21]  Adnan Darwiche,et al.  New Compilation Languages Based on Structured Decomposability , 2008, AAAI.

[22]  Pedro M. Domingos,et al.  Sum-product networks: A new deep architecture , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[23]  Guy Van den Broeck,et al.  Tractable Learning for Complex Probability Queries , 2015, NIPS.

[24]  Randal E. Bryant,et al.  Graph-Based Algorithms for Boolean Function Manipulation , 1986, IEEE Transactions on Computers.

[25]  Dan Suciu,et al.  Recent Trends in Knowledge Compilation (Dagstuhl Seminar 17381) , 2017, Dagstuhl Reports.

[26]  Bernard Chazelle,et al.  Computing partial sums in multidimensional arrays , 1989, SCG '89.

[27]  Adnan Darwiche,et al.  On Relaxing Determinism in Arithmetic Circuits , 2017, ICML.

[28]  Guy Van den Broeck,et al.  Learning the Structure of Probabilistic Sentential Decision Diagrams , 2017, UAI.

[29]  Henry A. Kautz,et al.  Performing Bayesian Inference by Weighted Model Counting , 2005, AAAI.

[30]  Guy Van den Broeck,et al.  Approximate Knowledge Compilation by Online Collapsed Importance Sampling , 2018, NeurIPS.

[31]  Guy Van den Broeck,et al.  A Semantic Loss Function for Deep Learning with Symbolic Knowledge , 2017, ICML.

[32]  Friedrich Slivovsky,et al.  Expander CNFs have Exponential DNNF Size , 2014, ArXiv.