Triangulation of Bayesian networks with recursive estimation of distribution algorithms

Bayesian networks can be used as a model to make inferences in domains with intrinsic uncertainty, that is, to determine the probability distribution of a set of variables given the instantiation of another set. The inference is an NP-hard problem. There are several algorithms to make exact and approximate inference. One of the most popular, and that is also an exact method, is the evidence propagation algorithm of Lauritzen and Spiegelhalter [S.L. Lauritzen, D.J. Spiegelhalter, Local computations with probabilities on graphical structures and their application on expert systems, Journal of the Royal Statistical Society B 50 (2) (1988) 157-224], improved later by Jensen et al. [F.V. Jensen, S.L. Lauritzen, K.G. Olesen, Bayesian updating in causal probalistic networks by local computations, In Computational Statistics Quaterly 4 (1990) 269-282]. This algorithm needs an ordering of the variables in order to make the triangulation of the moral graph associated with the original Bayesian network structure. The effectiveness of the inference depends on the variable ordering. In this paper, we will use a new paradigm for evolutionary computation, the estimation of distribution algorithms (EDAs), to get the optimal ordering of the variables to obtain the most efficient triangulation. We will also present a new type of evolutionary algorithm, the recursive EDAs (REDAs). We will prove that REDAs improve the behaviour of EDAs in this particular problem, and that their results are competitive with other triangulation techniques.

[1]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[2]  C. Robert Kenley,et al.  Gaussian influence diagrams , 1989 .

[3]  Mark Hopkins,et al.  Using Recursive Decomposition to Construct Elimination Orders, Jointrees, and Dtrees , 2001, ECSQARU.

[4]  D. Rose.  A GRAPH-THEORETIC STUDY OF THE NUMERICAL SOLUTION OF SPARSE POSITIVE DEFINITE SYSTEMS OF LINEAR EQUATIONS , 1972 .

[5]  Neil Robertson,et al.  Graph Minors .XIII. The Disjoint Paths Problem , 1995, J. Comb. Theory B.

[6]  Andrew P. Sage,et al.  Uncertainty in Artificial Intelligence , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[7]  Jose Miguel Puerta,et al.  Searching for the best elimination sequence in Bayesian networks by using ant colony optimization , 2002, Pattern Recognit. Lett..

[8]  BayesiannetworksPedro,et al.  Combinatorial optimization by learning and simulation of , 2000 .

[9]  Heinz Mühlenbein,et al.  The Equation for Response to Selection and Its Use for Prediction , 1997, Evolutionary Computation.

[10]  Pedro Larrañaga,et al.  Learning Bayesian networks in the space of structures by estimation of distribution algorithms , 2003, Int. J. Intell. Syst..

[11]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[12]  José A. Gámez,et al.  Triangulation of Bayesian networks by retriangulation , 2003, Int. J. Intell. Syst..

[13]  Endika Bengoetxea,et al.  Inexact Graph Matching Using Estimation of Distribution Algorithms , 2002 .

[14]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[15]  Enrique F. Castillo,et al.  Expert Systems and Probabilistic Network Models , 1996, Monographs in Computer Science.

[16]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[17]  H. Mühlenbein,et al.  Gene Pool Recombination in Genetic Algorithms , 1996 .

[18]  Pedro Larrañaga,et al.  Decomposing Bayesian networks: triangulation of the moral graph with genetic algorithms , 1997, Stat. Comput..

[19]  Pedro Larrañaga,et al.  Learning Bayesian Networks In The Space Of Orderings With Estimation Of Distribution Algorithms , 2004, Int. J. Pattern Recognit. Artif. Intell..

[20]  Dan Geiger,et al.  A sufficiently fast algorithm for finding close to optimal junction trees , 1996, UAI.

[21]  Lakhmi C. Jain,et al.  Introduction to Bayesian Networks , 2008 .

[22]  Serafín Moral,et al.  Heuristic Algorithms for the Triangulation of Graphs , 1994, IPMU.

[23]  José A. Gámez,et al.  Partial abductive inference in Bayesian belief networks using a genetic algorithm , 1999, Pattern Recognit. Lett..

[24]  Detlef Seese,et al.  Easy Problems for Tree-Decomposable Graphs , 1991, J. Algorithms.

[25]  Isabelle Bloch,et al.  Inexact graph matching by means of estimation of distribution algorithms , 2002, Pattern Recognit..

[26]  Pedro Larrañaga,et al.  Unsupervised Learning Of Bayesian Networks Via Estimation Of Distribution Algorithms: An Application To Gene Expression Data Clustering , 2004, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[27]  Derek G. Corneil,et al.  Complexity of finding embeddings in a k -tree , 1987 .

[28]  José Manuel Gutiérrez,et al.  Expert Systems and Probabiistic Network Models , 1996 .

[29]  Wilson X. Wen,et al.  Optimal decomposition of belief networks , 1990, UAI.

[30]  Uffe Kjærulff.  Optimal decomposition of probabilistic networks by simulated annealing , 1992 .

[31]  Dan Wu,et al.  Triangulation of Bayesian Networks: A Relational Database Perspective , 2002, Rough Sets and Current Trends in Computing.

[32]  Steffen L. Lauritzen,et al.  Bayesian updating in causal probabilistic networks by local computations , 1990 .

[33]  Paul A. Viola,et al.  MIMIC: Finding Optima by Estimating Probability Densities , 1996, NIPS.

[34]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[35]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[36]  J. A. Lozano,et al.  Towards a New Evolutionary Computation: Advances on Estimation of Distribution Algorithms (Studies in Fuzziness and Soft Computing) , 2006 .

[37]  Uue Kjjrull.  Triangulation of Graphs { Algorithms Giving Small Total State Space Triangulation of Graphs { Algorithms Giving Small Total State Space , 1990 .

[38]  José A. Gámez,et al.  A Review on Distinct Methods and Approaches to Perform Triangulation for Bayesian Networks , 2007 .

[39]  Linda C. van der Gaag,et al.  Pre-processing for Triangulation of Probabilistic Networks , 2001, UAI.