Learning Chordal Markov Networks by Constraint Satisfaction

We investigate the problem of learning the structure of a Markov network from data. It is shown that the structure of such networks can be described in terms of constraints which enables the use of existing solver technology with optimization capabilities to compute optimal networks starting from initial scores computed from the data. To achieve efficient encodings, we develop a novel characterization of Markov network structure using a balancing condition on the separators between cliques forming the network. The resulting translations into propositional satisfiability and its extensions such as maximum satisfiability, satisfiability modulo theories, and answer set programming, enable us to prove optimal certain networks which have been previously found by stochastic search.

[1]  John D. Lafferty,et al.  Inducing Features of Random Fields , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Ilkka Niemelä,et al.  Compact Translations of Non-disjunctive Answer Set Programs to Propositional Clauses , 2011, Logic Programming, Knowledge Representation, and Nonmonotonic Reasoning.

[3]  M. Golumbic Algorithmic graph theory and perfect graphs , 1980 .

[4]  P. Green,et al.  Decomposable graphical Gaussian model determination , 1999 .

[5]  Cesare Tinelli,et al.  Handbook of Satisfiability , 2021, Handbook of Satisfiability.

[6]  D. Madigan,et al.  Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window , 1994 .

[7]  Mark W. Schmidt,et al.  Learning Graphical Model Structure Using L1-Regularization Paths , 2007, AAAI.

[8]  Martin Gebser,et al.  Conflict-driven answer set solving: From theory to practice , 2012, Artif. Intell..

[9]  James Cussens,et al.  Advances in Bayesian Network Learning using Integer Programming , 2013, UAI.

[10]  Cesare Tinelli,et al.  Satisfiability Modulo Theories , 2021, Handbook of Satisfiability.

[11]  Ruben Martins Parallel search for maximum satisfiability , 2015, Constraints.

[12]  Miroslaw Truszczynski,et al.  Answer set programming at a glance , 2011, Commun. ACM.

[13]  Carlos Gon,et al.  Parallel Search for Maximum Satisfiability , 2013 .

[14]  R. Möhring Algorithmic graph theory and perfect graphs , 1986 .

[15]  Martin Gebser,et al.  Domain-Specific Heuristics in Answer Set Programming , 2013, AAAI.

[16]  N. Wermuth,et al.  Graphical Models for Associations between Variables, some of which are Qualitative and some Quantitative , 1989 .

[17]  Toby Walsh,et al.  Handbook of Satisfiability: Volume 185 Frontiers in Artificial Intelligence and Applications , 2009 .

[18]  Mikko Koivisto,et al.  Exact Bayesian Structure Discovery in Bayesian Networks , 2004, J. Mach. Learn. Res..

[19]  Daphne Koller,et al.  Efficient Structure Learning of Markov Networks using L1-Regularization , 2006, NIPS.

[20]  James Cussens,et al.  Bayesian network learning by compiling to weighted MAX-SAT , 2008, UAI.

[21]  Carsten Sinz,et al.  Towards an Optimal CNF Encoding of Boolean Cardinality Constraints , 2005, CP.

[22]  Daniel Le Berre,et al.  The Sat4j library, release 2.2 , 2010, J. Satisf. Boolean Model. Comput..

[23]  Roberto Sebastiani,et al.  Optimization in SMT with LA(Q) Cost Functions , 2012 .

[24]  Theo Tryfonas,et al.  Frontiers in Artificial Intelligence and Applications , 2009 .

[25]  T. Speed,et al.  Markov Fields and Log-Linear Interaction Models for Contingency Tables , 1980 .

[26]  Ronald L. Graham,et al.  On the History of the Minimum Spanning Tree Problem , 1985, Annals of the History of Computing.

[27]  Robert Tibshirani,et al.  Estimation of Sparse Binary Pairwise Markov Networks using Pseudo-likelihoods , 2009, J. Mach. Learn. Res..

[28]  P. Dellaportas,et al.  Markov chain Monte Carlo model determination for hierarchical and graphical log-linear models , 1999 .

[29]  KoivistoMikko,et al.  Exact Bayesian Structure Discovery in Bayesian Networks , 2004 .

[30]  James Cussens,et al.  Bayesian network learning with cutting planes , 2011, UAI.

[31]  Jukka Corander,et al.  Parallell interacting MCMC for learning of topologies of graphical models , 2008, Data Mining and Knowledge Discovery.

[32]  Paolo Giudici,et al.  Improving Markov Chain Monte Carlo Model Search for Data Mining , 2004, Machine Learning.

[33]  M. Golummc Algorithmic graph theory and perfect graphs , 1980 .

[34]  Felip Manyà,et al.  MaxSAT, Hard and Soft Constraints , 2021, Handbook of Satisfiability.

[35]  Frank Jensen,et al.  Optimal junction Trees , 1994, UAI.

[36]  Yukio Shibata,et al.  On the tree representation of chordal graphs , 1988, J. Graph Theory.

[37]  Jukka Corander,et al.  Bayesian graphical model determination using decision theory , 2003 .