Discriminative Structure Learning of Arithmetic Circuits

The biggest limitation of probabilistic graphical models is the complexity of inference, which is often intractable. An appealing alternative is to use tractable probabilistic models, such as arithmetic circuits (ACs) and sum-product networks (SPNs), in which marginal and conditional queries can be answered efficiently. In this paper, we present the first discriminative structure learning algorithm for ACs, DACLearn (Discriminative AC Learner), which optimizes conditional log-likelihood. Based on our experiments, DACLearn learns models that are more accurate and compact than other tractable generative and discriminative baselines.

[1]  Pedro M. Domingos,et al.  Sum-product networks: A new deep architecture , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[2]  Pedro M. Domingos,et al.  Bottom-Up Learning of Markov Network Structure , 2010, ICML.

[3]  Andrew McCallum,et al.  Efficiently Inducing Features of Conditional Random Fields , 2002, UAI.

[4]  Jesse Davis,et al.  Markov Network Structure Learning: A Randomized Feature Generation Approach , 2012, AAAI.

[5]  Pedro M. Domingos,et al.  Learning the Structure of Sum-Product Networks , 2013, ICML.

[6]  Adnan Darwiche,et al.  A differential approach to inference in Bayesian networks , 2000, JACM.

[7]  Carlos Guestrin,et al.  Efficient Principled Learning of Thin Junction Trees , 2007, NIPS.

[8]  Elad Eban,et al.  Learning Max-Margin Tree Predictors , 2013, UAI.

[9]  Daniel Lowd,et al.  Learning Tractable Graphical Models Using Mixture of Arithmetic Circuits , 2013, AAAI.

[10]  Daniel Lowd,et al.  The Libra toolkit for probabilistic models , 2015, J. Mach. Learn. Res..

[11]  Pedro M. Domingos,et al.  Discriminative Learning of Sum-Product Networks , 2012, NIPS.

[12]  Milos Hauskrecht,et al.  A Mixtures-of-Trees Framework for Multi-Label Classification , 2014, CIKM.

[13]  Dafna Shahaf,et al.  Learning Thin Junction Trees via Graph Cuts , 2009, AISTATS.

[14]  Joseph K. Bradley,et al.  Learning Tree Conditional Random Fields , 2010, ICML.

[15]  Daniel Lowd,et al.  Learning Markov Networks With Arithmetic Circuits , 2013, AISTATS.

[16]  Vibhav Gogate,et al.  Cutset Networks: A Simple, Tractable, and Scalable Approach for Improving the Accuracy of Chow-Liu Trees , 2014, ECML/PKDD.

[17]  Michael I. Jordan,et al.  Thin Junction Trees , 2001, NIPS.

[18]  Daniel Lowd,et al.  Learning Sum-Product Networks with Direct and Indirect Variable Interactions , 2014, ICML.

[19]  Adnan Darwiche,et al.  Compiling Bayesian Networks with Local Structure , 2005, IJCAI.

[20]  Michael I. Jordan,et al.  Learning with Mixtures of Trees , 2001, J. Mach. Learn. Res..

[21]  Pedro M. Domingos,et al.  Learning Arithmetic Circuits , 2008, UAI.