Finding Consensus Bayesian Network Structures

Suppose that multiple experts (or learning algorithms) provide us with alternative Bayesian network (BN) structures over a domain, and that we are interested in combining them into a single consensus BN structure. Specifically, we are interested in that the consensus BN structure only represents independences all the given BN structures agree upon and that it has as few parameters associated as possible. In this paper, we prove that there may exist several non-equivalent consensus BN structures and that finding one of them is NP-hard. Thus, we decide to resort to heuristics to find an approximated consensus BN structure. In this paper, we consider the heuristic proposed by Matzkevich and Abramson, which builds upon two algorithms, called Methods A and B, for efficiently deriving the minimal directed independence map of a BN structure relative to a given node ordering. Methods A and B are claimed to be correct although no proof is provided (a proof is just sketched). In this paper, we show that Methods A and B are not correct and propose a correction of them.

[1]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[2]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[3]  Fanica Gavril,et al.  Some NP-complete problems on graphs , 2011, CISS 2011.

[4]  David Maxwell Chickering,et al.  A Transformational Characterization of Equivalent Bayesian Network Structures , 1995, UAI.

[5]  José M. Peña,et al.  On Local Optima in Learning Bayesian Networks , 2003, UAI.

[6]  Tommi S. Jaakkola,et al.  Combining Location and Expression Data for Principled Discovery of Genetic Regulatory Network Models , 2001, Pacific Symposium on Biocomputing.

[7]  Srinivas Aluru,et al.  Consensus genetic maps: a graph theoretic approach , 2005, 2005 IEEE Computational Systems Bioinformatics Conference (CSB'05).

[8]  Bruce Abramson,et al.  Some Complexity Considerations in the Combination of Belief Networks , 1993, UAI.

[9]  David Maxwell Chickering,et al.  Optimal Structure Identification With Greedy Search , 2003, J. Mach. Learn. Res..

[10]  David Maxwell Chickering,et al.  Finding Optimal Bayesian Networks , 2002, UAI.

[11]  David Maxwell Chickering,et al.  Large-Sample Learning of Bayesian Networks is NP-Hard , 2002, J. Mach. Learn. Res..

[12]  Jose M. Peña,et al.  Featuring Multiple Local Optima to Assist the User in the Interpretation of Induced Bayesian Network Models. , 2004 .

[13]  Dan Geiger,et al.  Identifying independence in bayesian networks , 1990, Networks.

[14]  Milan Studený.  Bayesian Networks from the Point of View of Chain Graphs , 1998, UAI.

[15]  Nir Friedman,et al.  Being Bayesian About Network Structure. A Bayesian Approach to Structure Discovery in Bayesian Networks , 2004, Machine Learning.

[16]  Bruce Abramson,et al.  The Topological Fusion of Bayes Nets , 1992, UAI.

[17]  Jesper Tegnér,et al.  Towards scalable and data efficient learning of Markov boundaries , 2007, Int. J. Approx. Reason..

[18]  Matthew Richardson,et al.  Learning with Knowledge from Multiple Experts , 2003, ICML.

[19]  Michael P. Wellman,et al.  Graphical Representations of Consensus Belief , 1999, UAI.

[20]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[21]  Pedrito Maynard-Reid,et al.  Aggregating Learned Probabilistic Beliefs , 2001, UAI.

[22]  Keung-Chi Ng,et al.  Probabilistic multi-knowledge-base systems , 1994, Applied Intelligence.

[23]  Simon Parsons,et al.  An application of formal argumentation: Fusing Bayesian networks in multi-agent systems , 2007, Artif. Intell..

[24]  Christian Genest,et al.  Combining Probability Distributions: A Critique and an Annotated Bibliography , 1986 .

[25]  Bruce Abramson,et al.  Deriving A Minimal itI-map of a Belief Network Relative to a Target Ordering of its Nodes , 1993, UAI.

[26]  M. Studený,et al.  On chain graph models for description of conditional independence structures , 1998 .

[27]  Serafín Moral,et al.  Qualitative combination of Bayesian networks , 2003, Int. J. Intell. Syst..

[28]  C. Meek,et al.  Graphical models: selecting causal and statistical models , 1997 .